mtgonline/ROADMAP.md

# MTG Online Web — Project Roadmap

## Overview

A modern web-based implementation of the MTG Online multiplayer Magic: The Gathering platform. Built with Python/FastAPI backend and React/TypeScript frontend to replace the legacy C++/Qt desktop client.

## Phase 1: Backend Foundation ✅ (COMPLETED)

### 1.1 Project Setup
- [x] Initialize Python project structure
- [x] Create requirements.txt with pinned dependencies
- [x] Set up pydantic-settings configuration
- [x] Configure async SQLAlchemy with PostgreSQL
- [x] Create JWT authentication system with bcrypt
- [x] Set up FastAPI application with CORS

### 1.2 Data Models
- [x] User model (accounts, profiles, VIP status)
- [x] Deck models (decks, folders, files)
- [x] Room model (chat rooms, game types)
- [x] Ban model (moderation, history)
- [x] Game Log model (audit trail)
- [x] Decklist File/Folder models

### 1.3 API Endpoints
- [x] Authentication (login, register, refresh)
- [x] User management (CRUD, ban/unban)
- [x] Deck management (CRUD, folder operations)
- [x] Room management (list, create, update, delete)
- [x] Game management (create, join, leave)
- [x] Admin endpoints (user list, ban management, logs)

### 1.4 Services
- [x] WebSocket game server
- [x] Deck parser (plain text + native XML)
- [x] Card database service (MTJSON integration)
- [x] Protocol constants (MTG Online protocol compatibility)

### 1.5 Testing
- [x] Pytest configuration with async support
- [x] In-memory SQLite for testing
- [x] Auth endpoint tests
- [x] Deck CRUD tests
- [x] Admin endpoint tests

### 1.6 Documentation
- [x] README.md
- [x] API documentation (FastAPI auto-generated)
- [x] Environment configuration template
- [x] Statement of Intent
- [x] Project Roadmap
- [x] State tracking

## Phase 2: User Data Schema & API ✅ (COMPLETED)

### 2.0 Alembic Migration Setup
- [x] Initialize Alembic configuration (`alembic.ini`)
- [x] Create async `env.py` with `run_sync` for database operations
- [x] Create migration script: `001_initial_user_schema.py`
- [x] Create migration runner script: `scripts/run_migrations.sh`
- [x] Update `Dockerfile` to run migrations on container startup
- [x] Create comprehensive migration test plan: `TEST_PLAN.md`

### 2.1 Database Models (16 Tables)
- [x] **`users`** - User accounts with authentication
- [x] **`decks`** - User decks (DRAFT/FINAL status)
- [x] **`cards`** - User card collections
- [x] **`card_ownership`** - Card ownership tracking
- [x] **`win_streaks`** - Win/loss statistics
- [x] **`game_replays`** - Saved game replays (JSONB)
- [x] **`groups`** - User groups
- [x] **`group_members`** - Group membership
- [x] **`networks`** - Network accounts (Twitch, X, YouTube)
- [x] **`network_credentials`** - Network login info
- [x] **`preferences`** - User preferences (JSONB)
- [x] **`activity_log`** - User activity tracking (JSONB)
- [x] **`suggested_cards`** - Card suggestions
- [x] **`folders`** - Deck organization
- [x] **`game_logs`** - Game audit trail
- [x] **`user_decks`** - User deck storage (DRAFT/FINAL)

### 2.2 API Endpoints

#### Replays (`/api/v1/user-data/replays`)
- [x] `POST /api/v1/user-data/replays/` - Save replay
- [x] `GET /api/v1/user-data/replays/{replay_id}` - Get replay
- [x] `DELETE /api/v1/user-data/replays/{replay_id}` - Delete replay

#### Card Collection (`/api/v1/user-data/cards`)
- [x] `GET /api/v1/user-data/cards/` - List user's cards
- [x] `POST /api/v1/user-data/cards/` - Add card to collection
- [x] `DELETE /api/v1/user-data/cards/{card_id}` - Remove card

#### Groups (`/api/v1/user-data/groups`)
- [x] `GET /api/v1/user-data/groups/` - List user's groups
- [x] `POST /api/v1/user-data/groups/` - Create group
- [x] `PATCH /api/v1/user-data/groups/{group_id}` - Update group
- [x] `DELETE /api/v1/user-data/groups/{group_id}` - Delete group

#### Networks (`/api/v1/user-data/networks`)
- [x] `GET /api/v1/user-data/networks/` - List network accounts
- [x] `POST /api/v1/user-data/networks/` - Add network
- [x] `PATCH /api/v1/user-data/networks/{network_id}` - Update network
- [x] `DELETE /api/v1/user-data/networks/{network_id}` - Remove network

#### Preferences (`/api/v1/user-data/preferences`)
- [x] `GET /api/v1/user-data/preferences/` - Get preferences
- [x] `PATCH /api/v1/user-data/preferences/` - Update preferences

#### Activity Log (`/api/v1/user-data/activity`)
- [x] `GET /api/v1/user-data/activity/` - List activity
- [x] `POST /api/v1/user-data/activity/` - Add activity entry

### 2.3 Architecture Decisions
- [x] **JSONB columns** for flexible data storage (replay_data, activity_data, preferences)
- [x] **CASCADE deletes** for data integrity in related tables
- [x] **Composite unique constraints** for card collection uniqueness
- [x] **RESTful API design** with pagination support
- [x] **JWT authentication** for all endpoints
- [x] **Permission checks** for group/network management

### 2.4 Card Collection Logic
- [x] Users upload card names; system populates remaining data from `mtgdata` PostgreSQL database
- [x] Fuzzy matching service for card name normalization
- [x] Card ownership tracking with confidence scores

### 2.5 Documentation
- [x] API documentation: `API_DOCUMENTATION.md`
- [x] Migration test plan: `TEST_PLAN.md`
- [x] Comprehensive endpoint documentation with request/response examples
- [x] Database schema documentation

### 2.6 Card Import Feature
- [x] Card import router with status/import/delete/summary endpoints
- [x] Fuzzy matching logic (exact, case-insensitive, partial)
- [x] Card search endpoint integration
- [x] Pydantic schemas for import operations
- [x] Card import model with CASCADE FK

### 2.7 Deck Building Services
- [x] Deck CRUD endpoints (list, create, get, update, delete)
- [x] Card management endpoints (add, update, remove, list cards)
- [x] Deck finalize endpoint (DRAFT → FINAL transition)
- [x] Precedent endpoints (list, create, get, use/clone)
- [x] Card search endpoint (POST /decks/search/cards)
- [x] Suggestion endpoints (list, add suggestions)
- [x] Pydantic schemas for all deckbuilding operations

### 2.8 Testing
- [x] Unit tests for deck CRUD operations
- [x] Unit tests for card search functionality
- [x] Unit tests for card suggestion algorithm
- [x] Unit tests for file parsers (XLSX, CSV, JSON, ODS)
- [x] Unit tests for fuzzy matching service
- [x] Integration tests for import workflow
- [x] Load tests for bulk import processing

### 2.9 Documentation
- [x] API documentation (FastAPI auto-generated)
- [x] Database schema documentation
- [x] Fuzzy matching algorithm documentation
- [x] Import workflow documentation
- [x] Play backend integration guide

### 2.10 MTG Rules Engine Integration

The MTG rules engine has been integrated into the backend for multiplayer game server support:

- [x] **Integrated in `backend/mtg_rules_engine/`**
- [x] Core modules: `engine.py`, `rules_engine.py`, `keywords.py`, `validator.py`
- [x] Supporting modules: `keywords_db.py`, `keyword_validator.py`, `updater.py`, `update_check.py`
- [x] Test suite: `test_engine.py`
- [x] Purpose: Enforces MTG game rules (mana, phases, priority, stack resolution, combat) for multiplayer server

### 2.11 Multiplayer Play Backend — Architecture Blueprint (DERIVED FROM COCKATRICE ANALYSIS)

A detailed architecture analysis of **Cockatrice** (v3.1.0 "Graduation Day") — the mature open-source MTG online client/server — has been completed at `/home/wall-o/projects/mtgonline/C++/ARCHITECTURE_ANALYSIS.md`.

The play backend will follow the same authoritative server model, adapted for a modern web stack:

#### Authority Model

```
┌─────────────────┐         WebSocket (JSON)             ┌─────────────────┐
│   CLIENT        │◄────────────────────────────────────►│   SERVER        │
│  (React/TS)     │                                      │  (FastAPI/Py)   │
│                 │                                      │                 │
│ • Game Scene    │   GameCommands (play, attack, etc.)  │ • PostgreSQL    │
│ • Hand View     │◄────────────────────────────────────►│ • Game State    │
│ • Chat Panel    │   GameEvents (state changes)         │ • Room/Player   │
│ • Deck Panel    │                                      │   Management    │
│ • Phase Toolbar │   Chat, Admin, Spectator             │ • Replay Log    │
└─────────────────┘                                      └─────────────────┘
```

#### Game Engine Architecture

| Cockatrice Component | Modern Web Equivalent | Role |
|---|---|---|
| `AbstractGame` | `Game` (server-side) | Core game instance holding state, players, event handler |
| `GameMetaInfo` | `GameMetadata` | gameId, maxPlayers, description, started, spectators |
| `GameState` | `GameBoardState` | currentPhase, activePlayer, hostId, gameTimer |
| `GameEventHandler` | `GameEventDispatcher` | Central dispatch — processes events, prepares commands (~21KB in Cockatrice) |
| `PlayerManager` | `PlayerRegistry` | Coordinates all players in a game |
| `PlayerLogic` | `Player` | Per-player game logic (~10.7KB in Cockatrice) |
| `PlayerActions` | `PlayerCommands` | Concrete commands: play, attack, tap, draw (~64KB in Cockatrice) |
| `CardZone` | `Zone` (base class) | Abstract zone — Hand/Stack/Table/Graveyard/Exile/Library |
| `HandZone` | `Hand` | Player's hand (secret/hidden zone) |
| `StackZone` | `Stack` | Spells/abilities on the stack |
| `TableZone` | `Battlefield` | Permanents on the battlefield |
| `PileZone` | `Pile` (Graveyard, Exile, Library) | Discard/exile/draw piles |
| `Replay` | `GameReplay` | Serialized event stream for replay |
| `Phase` | `TurnPhase` | 11-phase MTG turn structure |

#### Turn Phase System (11 Phases with Sub-Phases)

```
Untap → Upkeep → Draw → Main 1 → Combat → Main 2 → End → Cleanup
                                        │
                                        └── Sub-phases:
                                            Beginning of Combat
                                            Declare Attackers
                                            Declare Blockers
                                            Combat Damage
                                            End of Combat
```

#### Command/Event Flow

```
User Action (React component)
    │
    ▼
GameCommand (JSON message)
    │
    ▼
GameEventDispatcher.process()
    │
    ▼
Player.handleCommand()
    │
    ▼
ZoneLogic (state mutation)
    │
    ▼
GameEvent (broadcast to all clients via WebSocket)
    │
    ▼
Client receives & updates UI via Zustand/XState
```

#### Network Protocol Design (Adapted from Cockatrice's protobuf)

Cockatrice uses Protocol Buffers over TCP. The modern web equivalent replaces protobuf with JSON over WebSocket:

| Cockatrice Proto Message | JSON WebSocket Message |
|---|---|
| `ServerInfo_Game` | `{ "type": "game_info", "game_id": 1, "max_players": 2, ... }` |
| `ServerInfo_Player` | `{ "type": "player_info", "player_id": 1, "name": "...", ... }` |
| `Command_PlayCard` | `{ "type": "cmd_play_card", "card_id": 42, "zone": "hand", ... }` |
| `Command_Attack` | `{ "type": "cmd_attack", "attacker_id": 7, "targets": [3, 5], ... }` |
| `Event_Join` | `{ "type": "event_join", "player_id": 2, "properties": {...} }` |
| `Event_Leave` | `{ "type": "event_leave", "player_id": 1, "reason": "..." }` |
| `Event_SetActivePlayer` | `{ "type": "event_active_player", "player_id": 1 }` |
| `Event_SetActivePhase` | `{ "type": "event_active_phase", "phase": 5 }` |
| `Event_GameSay` | `{ "type": "event_chat", "player_id": 1, "message": "..." }` |
| `GameReplay` | `{ "type": "replay", "events": [...] }` |

#### Architecture Patterns to Reuse

| Pattern | Cockatrice Usage | Modern Equivalent |
|---|---|---|
| **Event Bus** | Qt signals/slots | WebSocket broadcast + Zustand stores |
| **Command Pattern** | `Command_PlayCard`, `Command_Attack` | JSON command messages, validated server-side |
| **Observer** | `GameEventHandler` emits signals | WebSocket events trigger UI updates |
| **Strategy** | `CardZoneLogic` subclasses | Zone classes with strategy pattern (Hand, Stack, Table, Pile, View) |
| **Facade** | `AbstractGame` | Single `Game` object wrapping all subsystems |
| **Memento** | `DeckListMemento` for undo | Immutable state snapshots for undo/redo |
| **Repository** | `ServatriceDatabaseInterface` | SQLAlchemy repositories for game state, decks, logs |

#### Key Decisions (Informed by Cockatrice Analysis)

1. **Server is authoritative** — clients send commands, server validates and broadcasts events. No client-side state manipulation.
2. **Deterministic replay** — serialize all game events; same command sequence produces identical state.
3. **Zone abstraction** — each zone type (Hand, Stack, Battlefield, Pile) is independently implementable.
4. **Phase system** — 11-phase MTG turn with sub-phases, tracked server-side.
5. **Command/Event separation** — what a player *wants* to do vs. what *happens*.

#### What Modernizes Cockatrice

| Cockatrice Limitation | Modern Web Solution |
|---|---|
| TCP only, no web protocol | WebSocket (JSON) |
| Qt Widgets desktop UI | React/Next.js with PixiJS for game board |
| C++/MySQL backend | Python/FastAPI + PostgreSQL |
| Single-server clustering | Stateless game servers + Redis state cache |
| No mobile support | Responsive design from start |
| No REST API | FastAPI REST + WebSocket hybrid |
| Protobuf serialization | JSON over WebSocket |
| 106KB monolithic server handler | Modular service architecture |

### 2.11 Play Backend Responsibilities (OUT OF SCOPE)

The play backend will implement:
- [ ] Real-time game state management (server-authoritative)
- [ ] Multiplayer WebSocket communication
- [ ] Game rule enforcement (combat, stack resolution, priority)
- [ ] Deck validation during gameplay
- [ ] Game history and replay (serialized event log)
- [ ] Room and game lobbies
- [ ] Spectator mode
- [ ] Admin commands (kick, ban, game control)

**Note**: The play backend lives in a separate codebase. Integration points with the card backend:
- **Card Backend API** (`http://backend:8000`):
  - Authentication via JWT
  - Card lookups: `GET /api/cards/{card_id}`
  - Card search: `GET /api/cards/search?q=...`
  - Set lists: `GET /api/sets/`
  - User decks: `GET /api/users/{user_id}/decks`
- **Direct PSQL Access**:
  - `mtgdata` database: Read card data, sets, etc.
  - `mtgonline` database: Read user decks, validate deck legality

## Phase 3: Frontend Development ✅ (COMPLETED)

### 3.1 Project Setup
- [x] Initialize React + TypeScript project with Vite
- [x] Configure ESLint, Prettier, TypeScript strict mode
- [x] Set up Zustand for state management
- [x] Configure Tailwind CSS for styling
- [x] Set up Vitest + React Testing Library

### 3.2 Authentication
- [x] Login form with JWT token storage
- [x] Registration form with validation
- [x] Protected routes and auth context
- [x] Session management and token refresh

### 3.3 Deck Builder
- [x] Card search with filters (name, color, type, set)
- [x] Deck list editor with drag-and-drop
- [x] Import/export deck formats (plain text, native XML)
- [x] Folder management UI
- [x] Real-time deck statistics (card count, mana curve)
- [x] **NEW: Deck status indicator** (DRAFT vs FINAL)
- [x] **NEW: Card suggestion panel** (shows similar cards)

### 3.4 Card Import Interface
- [x] File upload component (XLSX, CSV, JSON, ODS)
- [x] Import progress indicator
- [x] Match results display with confidence scores
- [x] Manual override for low-confidence matches
- [x] Import history and re-import capability

### 3.5 Game Interface (TODO - Dependent on Play Backend)
- [ ] Game board visualization (zones, cards)
- [ ] Player hand (private zone)
- [ ] Library, graveyard, exile, command zones
- [ ] Card interaction (click, drag, hover)
- [ ] Real-time WebSocket updates

### 3.6 Chat System
- [x] Room chat interface
- [x] Game chat (in-game messaging)
- [x] Player list display
- [x] Moderator tools (kick, ban)

### 3.7 Admin Dashboard
- [x] User management interface
- [x] Ban/unban controls
- [x] Game logs viewer
- [x] System statistics

## Phase 4: Integration & Polish (TODO)

### 4.1 WebSocket Client
- [ ] WebSocket connection management
- [ ] Reconnection logic with exponential backoff
- [ ] Message serialization/deserialization
- [ ] Protocol buffer message handling

### 4.2 Card Database
- [ ] Import MTJSON card data
- [ ] Cache card images locally
- [ ] Search and filter functionality
- [ ] Card tooltips with oracle text

### 4.3 Game Logic (TODO - Dependent on Play Backend)
- [ ] Turn-based state management
- [ ] Priority system implementation
- [ ] Stack resolution
- [ ] Mana payment tracking
- [ ] Life totals and counters

### 4.4 Performance
- [ ] Virtual scrolling for card lists
- [ ] Memoization and React.memo
- [ ] Code splitting and lazy loading
- [ ] WebSocket message batching

## Phase 5: Advanced Features (FUTURE)

### 5.1 Multiplayer Enhancements
- [ ] Spectator mode
- [ ] Game replay system
- [ ] Tournament support
- [ ] Custom game rules

### 5.2 Card Database
- [ ] Set filtering
- [ ] Card comparison
- [ ] Deck sharing and discovery
- [ ] Community decks

### 5.3 Mobile Support
- [ ] Responsive design
- [ ] PWA support
- [ ] Touch gestures
- [ ] Mobile-optimized controls

### 5.4 Integrations
- [ ] OAuth providers (Google, GitHub)
- [ ] Discord bot integration
- [ ] API webhooks
- [ ] Third-party deck sharing platforms

## Timeline

| Phase | Duration | Status |
|-------|----------|--------|
| Phase 1: Backend Foundation | 2 weeks | ✅ Complete |
| Phase 2: User Data Schema & API | 2 weeks | ✅ Complete |
| Phase 3: Frontend Development | 4 weeks | ✅ Complete |
| Phase 4: Testing & Deployment | 1 week | Not Started |
| Phase 5: Advanced Features | Ongoing | Future |

## Success Metrics

- [ ] Users can create accounts and log in
- [ ] Users can create and edit decks
- [ ] Users can play multiplayer games in real-time
- [ ] Game state syncs correctly across all players
- [ ] Admins can manage users and monitor games
- [ ] API response time < 100ms for 95% of requests
- [ ] WebSocket latency < 50ms
- [ ] Zero critical security vulnerabilities

## Dependencies

### Backend
- Python 3.12+
- PostgreSQL 14+
- Redis (optional, for caching)
- Node.js 18+ (for protocol buffer compilation)

### Frontend
- Node.js 18+
- npm or pnpm
- Browser with WebSocket support

### Development
- Git
- Docker (optional)
- VS Code or similar IDE
- PostgreSQL client (psql or DBeaver)

## Risk Mitigation

| Risk | Mitigation |
|------|------------|
| WebSocket reliability | Implement reconnection with exponential backoff |
| Game state inconsistency | Server-authoritative state with conflict resolution |
| Performance at scale | Load balancing, connection pooling, caching |
| Security vulnerabilities | Input validation, rate limiting, HTTPS |
| Data loss | Database backups, transaction logs |

## Future Considerations

- Migration to Rust for game server performance
- Integration with Magic Online API
- Support for custom card games
- AI-powered deck suggestions
- Blockchain-based card ownership verification