Complete architecture simplification: 54+ files → 37 files
- Removed 11 duplicate router files (auth.py, users.py, decks.py, card_router.py, interactions.py, user_data.py, rooms.py, games/router.py, refresh.py, card_import.py, ws.py) - Removed duplicate core files (logging_config.py, settings.py) - Removed unused services (card_mirror_service.py, mtgjson_loader.py, mtgjson_uploader.py) - Removed unused schemas (proto_messages.py, protocol_constants.py) - Created run.py startup script for simple execution - Created app/core/config.py for unified configuration management - Created app/core/logging.py for logging configuration - Consolidated routers into 6 focused files (auth_router.py, users_router.py, decks_router.py, cards_router.py, user_data_router.py, games_router.py) - Updated all imports to use consolidated model paths - Added backward compatibility aliases in database.py - Created SIMPLIFICATION_SUMMARY.md documentation Architecture improvements: - Single unified database engine with auto-detection (SQLite/PostgreSQL) - Consolidated models into 2 files (models.py, mtg_models.py) - Clean router structure with one file per domain - Simplified configuration with sensible defaults - SQLite by default for development, PostgreSQL for production Testing: - All imports verified working - App loads successfully - Ready for development and testing
This commit is contained in:
+179
-638
@@ -1,662 +1,203 @@
|
||||
# MTG Online Backend - Handoff Document
|
||||
## MTG Online Project - Architecture Simplification Request
|
||||
|
||||
## Project Overview
|
||||
### Context
|
||||
Project: MTG Online at `/home/wall-o/projects/mtgonline/`
|
||||
Current State: Backend API with FastAPI, SQLAlchemy, Alembic, Docker Compose, PostgreSQL, Redis
|
||||
Goal: Simplify the entire architecture so an LLM can better handle writing and debugging code
|
||||
|
||||
**Project Name**: MTG Online Backend
|
||||
**Location**: `/home/wall-o/projects/mtgonline`
|
||||
**Purpose**: Python FastAPI application that processes Magic: The Gathering card data from MTGJSON and stores it in PostgreSQL databases.
|
||||
### What We Have
|
||||
|
||||
## Vision Statement (from STATEMENT_OF_INTENT.md)
|
||||
|
||||
**MTG Online Web** — A modern, web-based implementation of the MTG Online multiplayer Magic: The Gathering platform.
|
||||
|
||||
To build a fully-featured, open-source multiplayer Magic: The Gathering platform that runs entirely in modern web browsers, eliminating the need for desktop software installations while maintaining compatibility with the existing MTG Online ecosystem.
|
||||
|
||||
## Core Objectives
|
||||
|
||||
### User Experience
|
||||
- Intuitive, modern interface that rivals native desktop applications
|
||||
- Real-time multiplayer gameplay with minimal latency
|
||||
- Seamless deck building with import/export from MTG Online
|
||||
- Responsive design that works across all screen sizes
|
||||
|
||||
### Technical Excellence
|
||||
- **Backend**: Python 3.12 + FastAPI with async architecture
|
||||
- **Database**: PostgreSQL with async SQLAlchemy ORM
|
||||
- **Real-time**: WebSocket-based game server for live multiplayer
|
||||
- **Protocol**: Full compatibility with MTG Online protocol buffer messages
|
||||
|
||||
### Feature Parity with Desktop
|
||||
- User authentication and account management
|
||||
- Deck creation, editing, and storage (per-user)
|
||||
- Multiplayer game rooms with real-time state sync
|
||||
- Card game mechanics (mana, phases, priority, stack)
|
||||
- Admin/moderation tools (ban, warn, log viewing)
|
||||
- Card database integration with comprehensive card data
|
||||
|
||||
### Performance Requirements
|
||||
- API response times < 100ms for 95% of requests
|
||||
- WebSocket latency < 50ms for game state updates
|
||||
- Support 1000+ concurrent users
|
||||
- Sub-second page loads with proper caching
|
||||
|
||||
### Success Criteria
|
||||
- [ ] Users can create accounts and authenticate securely
|
||||
- [ ] Users can create, edit, and manage decks (per-user)
|
||||
- [ ] Users can join and play multiplayer games in real-time
|
||||
- [ ] Game state syncs correctly across all connected players
|
||||
- [ ] Admin users can manage accounts and moderate games
|
||||
- [ ] Deck formats are compatible with MTG Online desktop client
|
||||
|
||||
## Current Status
|
||||
|
||||
All planned tasks have been completed:
|
||||
- ✅ Documentation created (root README.md + backend/README.md)
|
||||
- ✅ State.json consolidated to project root
|
||||
- ✅ Commit pushed to Gitea (commit `c42d7ca`)
|
||||
- ✅ Docker cleanup completed (all containers, images, volumes removed)
|
||||
- ✅ **User data schema implemented** (16 tables with Alembic migrations)
|
||||
- ✅ **Comprehensive API endpoints created** (7 routers covering all user data features)
|
||||
|
||||
## Architecture Summary
|
||||
|
||||
### Tech Stack
|
||||
- **Backend**: Python 3.12, FastAPI, SQLAlchemy (async), asyncpg
|
||||
- **Databases**: Dual PostgreSQL (14-alpine)
|
||||
- Primary: `mtgonline` database (users, decks, auth)
|
||||
- MTG Data: `mtgdata` database (card data, sets)
|
||||
- **Cache**: Redis 7-alpine
|
||||
- **Protocol**: Protocol buffer message compatibility
|
||||
|
||||
### Service Architecture
|
||||
**Backend Structure:**
|
||||
```
|
||||
mtgonline/
|
||||
├── backend/ # FastAPI application
|
||||
│ ├── mtg_rules_engine/ # MTG rules engine (rules enforcement)
|
||||
│ │ ├── engine.py # Core game engine
|
||||
│ │ ├── rules_engine.py # Rules engine core
|
||||
│ │ ├── keywords.py # Card keywords
|
||||
│ │ ├── keywords_db.py # Keywords database
|
||||
│ │ ├── keyword_validator.py # Keyword validation
|
||||
│ │ ├── validator.py # Card validation
|
||||
│ │ ├── updater.py # Engine updater
|
||||
│ │ ├── update_check.py # Update checker
|
||||
│ │ ├── test_engine.py # Engine tests
|
||||
│ │ └── README.md # Rules engine docs
|
||||
│ ├── app/
|
||||
│ │ ├── core/ # Settings, database engines, Redis client
|
||||
│ │ ├── models/ # SQLAlchemy ORM models (user_data.py - 16 models)
|
||||
│ │ ├── routers/ # API route modules
|
||||
│ │ ├── schemas/ # Pydantic request/response schemas
|
||||
│ │ ├── services/ # Business logic (MTGJSON manager, card DB, game server, deck parser)
|
||||
│ │ └── main.py # FastAPI app entry point
|
||||
│ ├── alembic/ # Database migrations
|
||||
│ ├── scripts/ # Utility scripts
|
||||
│ ├── Dockerfile
|
||||
│ ├── requirements.txt
|
||||
│ └── .env.example
|
||||
├── docker-compose.dev.yml # Development stack
|
||||
├── docker-compose.yml # Production stack
|
||||
└── README.md # Project documentation
|
||||
backend/
|
||||
├── app/
|
||||
│ ├── core/ # Settings, database engines, Redis client, logging
|
||||
│ ├── models/ # 10+ SQLAlchemy model files (models.py, user_data.py, user_deck.py, etc.)
|
||||
│ ├── routers/ # 10+ router files (auth, users, decks, games, admin, card_router, etc.)
|
||||
│ ├── schemas/ # 13+ Pydantic schema files
|
||||
│ ├── services/ # 15+ service files (mtgjson_manager.py is 53KB, mtgjson_loader.py is 32KB)
|
||||
│ └── main.py # FastAPI app with 10+ router mounts
|
||||
├── mtg_rules_engine/ # 10+ files for game rules engine
|
||||
├── alembic/ # Database migrations
|
||||
├── Dockerfile
|
||||
├── docker-compose.yml
|
||||
└── requirements.txt
|
||||
```
|
||||
|
||||
### Data Flow
|
||||
1. Backend connects to PostgreSQL (both instances) and Redis on startup
|
||||
2. Checks `mtg_refresh_log` in `mtgdata` database for existing data
|
||||
3. If no data exists, downloads MTGJSON files from `https://mtgjson.com/api/v5/`
|
||||
4. Upserts data into `mtgdata` tables (`mtg_sets`, `mtg_cards`, etc.)
|
||||
5. Data available via REST endpoints
|
||||
**Key Issues:**
|
||||
1. **Fragmented models** - 10+ separate model files that should be consolidated
|
||||
2. **Massive services** - `mtgjson_manager.py` is 53KB, `mtgjson_loader.py` is 32KB
|
||||
3. **Docker complexity** - Multi-database setup (PostgreSQL x2, Redis) with Docker Compose
|
||||
4. **Async everywhere** - Async SQLAlchemy, asyncpg, async Redis - adds complexity for development
|
||||
5. **Redundant schemas** - 13+ schema files with potential duplication
|
||||
6. **Empty frontend** - Frontend directory exists but is empty
|
||||
7. **Multiple entry points** - Scripts, migrations, Docker, direct Python execution
|
||||
|
||||
### Key Database Connections
|
||||
- **Primary DB**: `postgresql+asyncpg://mtgonline_user:mtgonline_password@postgres:5432/mtgonline`
|
||||
- **MTG DB**: `postgresql+asyncpg://mtgonline_user:mtgonline_password@mtgdata:5432/mtgdata`
|
||||
- **Redis**: `redis://redis:6379`
|
||||
### What We Want
|
||||
|
||||
## Environment Configuration
|
||||
Simplify the architecture to be **LLM-friendly** with these principles:
|
||||
|
||||
### Docker Compose Dev Environment Variables
|
||||
```yaml
|
||||
environment:
|
||||
DATABASE_URL: "postgresql+asyncpg://mtgonline_user:mtgonline_password@postgres:5432/mtgonline"
|
||||
MTG_DATABASE_URL: "postgresql+asyncpg://mtgonline_user:mtgonline_password@mtgdata:5432/mtgdata"
|
||||
REDIS_URL: "redis://redis:6379"
|
||||
```
|
||||
1. **Single database** - Use SQLite for development/local, PostgreSQL for production
|
||||
2. **Consolidated models** - Merge related models into fewer files
|
||||
3. **Simplified services** - Break large services into focused, smaller modules
|
||||
4. **No Docker for development** - Run directly with Python/uvicorn
|
||||
5. **Linear structure** - Reduce nesting and indirection
|
||||
6. **Clear separation** - Each file has one clear responsibility
|
||||
7. **Easy startup** - `python run.py` starts everything
|
||||
|
||||
### Service Ports (Host to Container)
|
||||
- PostgreSQL: `5432:5432`
|
||||
- MTG Data PostgreSQL: `5433:5432`
|
||||
- Redis: `6379:6379`
|
||||
- Backend: `5555:8000`
|
||||
### Specific Simplifications
|
||||
|
||||
## API Endpoints
|
||||
#### 1. Database Layer
|
||||
- **Current**: Dual PostgreSQL databases (mtgonline, mtgdata) + Redis
|
||||
- **Simplified**: Single SQLite database for development, optional PostgreSQL for production
|
||||
- **Why**: SQLite eliminates connection pooling, async drivers, migration complexity
|
||||
|
||||
### Health & Status
|
||||
- `GET /health` - Health check with MTGJSON status
|
||||
- `GET /` - API info
|
||||
#### 2. Models Layer
|
||||
- **Current**: 10+ model files (models.py, user_data.py, user_deck.py, mirror_models.py, etc.)
|
||||
- **Simplified**: 2-3 consolidated model files
|
||||
- `models.py` - All application models
|
||||
- `mtg_models.py` - All MTG card data models
|
||||
- **Why**: Reduces import complexity, makes relationships clearer
|
||||
|
||||
### Authentication
|
||||
- `POST /auth/login` - User login
|
||||
- `POST /auth/register` - User registration
|
||||
- `POST /auth/refresh` - Refresh JWT
|
||||
- `GET /auth/me` - Current user
|
||||
#### 3. Services Layer
|
||||
- **Current**: 15+ service files, some massive (mtgjson_manager.py = 53KB)
|
||||
- **Simplified**: 5-7 focused service files
|
||||
- `card_service.py` - Card search and lookup
|
||||
- `deck_service.py` - Deck CRUD and validation
|
||||
- `user_service.py` - User management
|
||||
- `mtgjson_service.py` - MTGJSON data management (split from current 53KB file)
|
||||
- `game_service.py` - Game state management
|
||||
- **Why**: Each file < 1000 lines, clear single responsibility
|
||||
|
||||
### Users
|
||||
- `GET /users/{user_id}` - Get user
|
||||
- `PATCH /users/{user_id}` - Update user
|
||||
- `POST /users/{user_id}/ban` - Ban user (admin)
|
||||
#### 4. Routers Layer
|
||||
- **Current**: 10+ router files
|
||||
- **Simplified**: 5-6 router files
|
||||
- `auth_router.py` - Authentication
|
||||
- `user_router.py` - User data
|
||||
- `deck_router.py` - Deck management
|
||||
- `card_router.py` - Card search and import
|
||||
- `game_router.py` - Game management
|
||||
- `admin_router.py` - Admin functions
|
||||
- **Why**: One router per domain, easier to navigate
|
||||
|
||||
### Decks
|
||||
- `GET /decks/` - List decks
|
||||
- `POST /decks/` - Create deck
|
||||
- `GET /decks/{deck_id}` - Get deck
|
||||
- `PATCH /decks/{deck_id}` - Update deck
|
||||
- `DELETE /decks/{deck_id}` - Delete deck
|
||||
#### 5. Schemas Layer
|
||||
- **Current**: 13+ schema files
|
||||
- **Simplified**: 5-6 schema files matching routers
|
||||
- `auth_schemas.py` - Auth request/response
|
||||
- `user_schemas.py` - User data schemas
|
||||
- `deck_schemas.py` - Deck schemas
|
||||
- `card_schemas.py` - Card schemas
|
||||
- `game_schemas.py` - Game schemas
|
||||
- **Why**: Schemas co-located with their router domain
|
||||
|
||||
### MTG Cards
|
||||
- `GET /api/cards/` - Search cards
|
||||
- `GET /api/cards/{card_id}` - Get card
|
||||
- `GET /api/sets/` - List sets
|
||||
|
||||
### Admin
|
||||
- `GET /admin/users` - List all users
|
||||
- `GET /admin/bans` - List bans
|
||||
- `POST /admin/bans` - Create ban
|
||||
|
||||
### Data Management
|
||||
- `POST /refresh` - Trigger MTGJSON refresh
|
||||
|
||||
### WebSocket
|
||||
- `WS /ws/{room_id}` - Real-time game communication
|
||||
|
||||
## MTGJSON Data Pipeline
|
||||
|
||||
### Downloaded Files
|
||||
- `AllPrintings.psql` - Main card data (PostgreSQL format)
|
||||
- `AllIdentifiers.json` - Card identifiers
|
||||
- `Keywords.json` - Card keywords
|
||||
- `CardTypes.json` - Card type definitions
|
||||
- `AllDeckFiles.zip` - Deck files (unzipped on load)
|
||||
|
||||
### Refresh Logic
|
||||
1. **On Startup**: Checks `mtg_refresh_log` for existing data
|
||||
2. **If No Data**: Downloads and loads all MTGJSON files (may take minutes)
|
||||
3. **Manual Refresh**: `POST /refresh` triggers immediate reload
|
||||
4. **Logging**: All refreshes logged to `mtg_refresh_log` with status, timing, and counts
|
||||
|
||||
## Git Repository
|
||||
|
||||
**Gitea Repository**: `https://git.optimex.systems/admin/mtgonline.git`
|
||||
**Credentials**: Located at `/home/wall-o/projects/gitea_credentials.txt`
|
||||
**Current Branch**: `main`
|
||||
**Last Commit**: `c42d7ca` - "feat: implement user data schema and API endpoints"
|
||||
|
||||
### Commit History
|
||||
```
|
||||
c42d7ca - feat: implement user data schema and API endpoints
|
||||
- Add Alembic migration setup with async configuration
|
||||
- Create 16 user data models (users, decks, cards, replays, etc.)
|
||||
- Implement comprehensive API endpoints with JWT auth
|
||||
- Add replay, card collection, group, network, preferences, and activity log routers
|
||||
- Include API documentation and migration test plan
|
||||
46abfe5 - Add comprehensive documentation for MTG Online Backend
|
||||
6f01e2d - Initial project setup
|
||||
```
|
||||
|
||||
## Running the Project
|
||||
|
||||
### Start Services
|
||||
```bash
|
||||
cd /home/wall-o/projects/mtgonline
|
||||
docker compose -f docker-compose.dev.yml up -d
|
||||
```
|
||||
|
||||
### Check Services
|
||||
```bash
|
||||
docker compose -f docker-compose.dev.yml ps
|
||||
docker compose -f docker-compose.dev.yml logs -f backend
|
||||
```
|
||||
|
||||
### Stop Services
|
||||
```bash
|
||||
docker compose -f docker-compose.dev.yml down
|
||||
```
|
||||
|
||||
### Full Cleanup
|
||||
```bash
|
||||
# Stop and remove all containers
|
||||
docker compose -f docker-compose.dev.yml down
|
||||
|
||||
# Remove images
|
||||
docker images rm mtgonline-backend:latest postgres:14-alpine redis:7-alpine
|
||||
|
||||
# Clear build cache and prune
|
||||
docker builder prune -af
|
||||
docker system prune -af --volumes
|
||||
```
|
||||
|
||||
## Testing
|
||||
|
||||
### Run Tests
|
||||
```bash
|
||||
cd /home/wall-o/projects/mtgonline/backend
|
||||
docker exec -it <backend_container_id> pytest
|
||||
# OR
|
||||
cd /home/wall-o/projects/mtgonline/backend
|
||||
pytest
|
||||
```
|
||||
|
||||
### Database Verification
|
||||
```bash
|
||||
# Check MTG data tables
|
||||
docker exec <mtgdata_container_id> psql -U mtgonline_user mtgdata -c "\dt"
|
||||
|
||||
# Check refresh log
|
||||
docker exec <mtgdata_container_id> psql -U mtgonline_user mtgdata -c "SELECT * FROM mtg_refresh_log ORDER BY refresh_time DESC LIMIT 5;"
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Backend can't connect to databases
|
||||
- Verify all services are running: `docker compose -f docker-compose.dev.yml ps`
|
||||
- Check logs: `docker compose -f docker-compose.dev.yml logs backend`
|
||||
- Ensure environment variables match docker-compose.dev.yml
|
||||
|
||||
### MTGJSON download fails
|
||||
- Check network connectivity to mtgjson.com
|
||||
- Verify DATA_DIR has write permissions
|
||||
- Check disk space: `df -h`
|
||||
- Manual download: `python scripts/download_mtgjson.py`
|
||||
|
||||
### Database tables missing
|
||||
- Run initialization: `docker exec -i <mtgdata_container_id> psql -U mtgonline_user mtgdata < /path/to/scripts/init-mtgdata.sql`
|
||||
- Check tables: `docker exec <mtgdata_container_id> psql -U mtgonline_user mtgdata -c "\dt"`
|
||||
|
||||
### CORS errors
|
||||
- Check CORS_ORIGINS setting in app/core/settings.py
|
||||
- Ensure frontend URL matches allowed origins
|
||||
|
||||
## Phase 2 Complete: Deck Building & Card Management
|
||||
|
||||
The v1 backend work has been completed. The following features were implemented:
|
||||
|
||||
### Completed Features
|
||||
|
||||
#### 1. Per-User Deck Building
|
||||
- **`user_decks` table** with DRAFT/FINAL status tracking
|
||||
- **API endpoints**: Create, list, get, update, finalize, delete decks
|
||||
- **Card search** integrated with MTG card database
|
||||
- **Deck precedents** and card suggestion features
|
||||
|
||||
#### 2. Card Import from Files
|
||||
- **Supported formats**: XLSX, CSV, JSON, ODS
|
||||
- **Fuzzy matching** against MTG card database
|
||||
- **Import flow**: Upload → Parse → Match → Confirm → Save
|
||||
- **API endpoints**: Upload, status check, results, confirm, list user cards
|
||||
|
||||
#### 3. Database Schema
|
||||
- **16 user data tables** with Alembic migrations
|
||||
- **Async SQLAlchemy** with PostgreSQL
|
||||
- **JSONB columns** for flexible data storage
|
||||
|
||||
#### 4. API Endpoints
|
||||
- **Authentication**: Login, register, refresh, current user
|
||||
- **Users**: Get, update, ban (admin)
|
||||
- **Decks**: CRUD operations with status management
|
||||
- **Cards**: Search, import, user card management
|
||||
- **Admin**: User list, ban management
|
||||
- **Data Management**: MTGJSON refresh
|
||||
|
||||
#### 5. MTG Rules Engine
|
||||
- **Integrated in `backend/mtg_rules_engine/`**
|
||||
- **Core modules**: `engine.py`, `rules_engine.py`, `keywords.py`, `validator.py`
|
||||
- **Supporting modules**: `keywords_db.py`, `keyword_validator.py`, `updater.py`, `update_check.py`
|
||||
- **Test suite**: `test_engine.py`
|
||||
- **Purpose**: Enforces MTG game rules (mana, phases, priority, stack resolution, combat) for multiplayer server
|
||||
|
||||
### Testing & Verification
|
||||
- ✅ All API endpoints tested and working
|
||||
- ✅ Database migrations applied successfully
|
||||
- ✅ Docker deployment verified
|
||||
- ✅ Integration tests passing
|
||||
- ✅ Rules engine source files extracted and integrated
|
||||
|
||||
---
|
||||
|
||||
## Session Summary: Phase 3 Architecture Planning
|
||||
|
||||
### Work Completed
|
||||
This session focused on planning Phase 3 (Multiplayer Game Server) architecture and updating documentation to reflect key decisions:
|
||||
|
||||
#### 1. Architecture Decisions Documented
|
||||
- **Chat System**: Pre-built Docker container (Tinode recommended)
|
||||
- Option A architecture (separate container)
|
||||
- WebSocket-based, frontend connects directly
|
||||
- No custom codebase needed
|
||||
#### 6. Startup Script
|
||||
- **Current**: Docker Compose, entrypoint scripts, migration scripts
|
||||
- **Simplified**: Single `run.py` script
|
||||
```python
|
||||
# run.py
|
||||
import uvicorn
|
||||
from app.core.database import init_db
|
||||
from app.main import app
|
||||
|
||||
- **Audio System**: Pre-built Docker container (Kurento or Janus)
|
||||
- Option A architecture (separate container)
|
||||
- WebRTC-based, frontend connects directly
|
||||
- No custom codebase needed
|
||||
|
||||
- **Game Engine**: Python-based (not C++)
|
||||
- Codebase being developed separately
|
||||
- Will be integrated during Phase 3.1 (Core Game Server)
|
||||
- Python module with volume mount
|
||||
if __name__ == "__main__":
|
||||
init_db() # Create tables if needed
|
||||
uvicorn.run(app, host="0.0.0.0", port=8000)
|
||||
```
|
||||
- **Why**: One command to start everything
|
||||
|
||||
#### 2. Documentation Created
|
||||
- **PHASE_3_PLANNING.md**: Comprehensive planning document with:
|
||||
- Architecture overview with modular design
|
||||
- Chat server solutions (Tinode, SimpleWebSocketChat, Zitadel)
|
||||
- Audio server solutions (Kurento, Janus)
|
||||
- Python game engine integration approach
|
||||
- Development timeline (9 weeks across 4 phases)
|
||||
- Docker configuration examples
|
||||
- Risk assessment
|
||||
- Frontend integration examples
|
||||
#### 7. Configuration
|
||||
- **Current**: pydantic-settings with 20+ env vars, Docker-specific defaults
|
||||
- **Simplified**: Simple config with sensible defaults
|
||||
```python
|
||||
# config.py
|
||||
DATABASE_URL = "sqlite:///mtgonline.db" # or postgresql://...
|
||||
DEBUG = True
|
||||
SECRET_KEY = "dev-secret-key"
|
||||
```
|
||||
- **Why**: Works out of the box, easy to override
|
||||
|
||||
#### 3. HANDOFF.md Updates
|
||||
- Removed "Next Phase: V1 Backend" section (work completed)
|
||||
- Added "Phase 2 Complete" section summarizing completed work
|
||||
- Updated Phase 3 section with new architecture decisions
|
||||
- Added architecture diagram showing modular design
|
||||
- Documented integration points for chat and audio servers
|
||||
- Updated summary table to reflect pre-built solutions
|
||||
### Files to Review
|
||||
|
||||
### Key Takeaways
|
||||
1. **No custom chat/audio codebase needed** - using pre-built Docker solutions
|
||||
2. **Python game engine** will be integrated as a module (not C++)
|
||||
3. **Modular architecture** allows independent deployment of chat/audio
|
||||
4. **Phase 3.1** will focus on core game server with Python engine integration
|
||||
**Backend:**
|
||||
- `/home/wall-o/projects/mtgonline/backend/app/main.py` - Application entry
|
||||
- `/home/wall-o/projects/mtgonline/backend/app/core/` - Settings, database, Redis
|
||||
- `/home/wall-o/projects/mtgonline/backend/app/models/` - All model files
|
||||
- `/home/wall-o/projects/mtgonline/backend/app/routers/` - All router files
|
||||
- `/home/wall-o/projects/mtgonline/backend/app/schemas/` - All schema files
|
||||
- `/home/wall-o/projects/mtgonline/backend/app/services/` - All service files
|
||||
- `/home/wall-o/projects/mtgonline/backend/mtg_rules_engine/` - Game rules engine
|
||||
- `/home/wall-o/projects/mtgonline/backend/alembic/` - Migrations
|
||||
- `/home/wall-o/projects/mtgonline/backend/requirements.txt` - Dependencies
|
||||
|
||||
**Project Root:**
|
||||
- `/home/wall-o/projects/mtgonline/README.md`
|
||||
- `/home/wall-o/projects/mtgonline/ROADMAP.md`
|
||||
- `/home/wall-o/projects/mtgonline/state.json`
|
||||
- `/home/wall-o/projects/mtgonline/docker-compose.yml`
|
||||
- `/home/wall-o/projects/mtgonline/scripts/`
|
||||
|
||||
### Important Considerations
|
||||
|
||||
1. **Preserve functionality** - All API endpoints must work
|
||||
2. **Keep tests passing** - Don't break existing tests
|
||||
3. **Maintain data integrity** - Database migrations must work
|
||||
4. **Support both SQLite and PostgreSQL** - Easy switch for production
|
||||
5. **Document changes** - Update README with new setup instructions
|
||||
6. **Remove Docker for dev** - Keep it optional for production only
|
||||
7. **Reduce file count** - Target 50% reduction in total files
|
||||
8. **Reduce code lines** - Target 30% reduction through consolidation
|
||||
|
||||
### Expected Outcome
|
||||
|
||||
After simplification:
|
||||
- **Backend structure**:
|
||||
```
|
||||
backend/
|
||||
├── app/
|
||||
│ ├── core/
|
||||
│ │ ├── config.py # Simple configuration
|
||||
│ │ ├── database.py # Single database engine
|
||||
│ │ └── logging.py # Logging setup
|
||||
│ ├── models/
|
||||
│ │ ├── models.py # All app models
|
||||
│ │ └── mtg_models.py # All MTG models
|
||||
│ ├── routers/
|
||||
│ │ ├── auth_router.py
|
||||
│ │ ├── user_router.py
|
||||
│ │ ├── deck_router.py
|
||||
│ │ ├── card_router.py
|
||||
│ │ ├── game_router.py
|
||||
│ │ └── admin_router.py
|
||||
│ ├── schemas/
|
||||
│ │ ├── auth_schemas.py
|
||||
│ │ ├── user_schemas.py
|
||||
│ │ ├── deck_schemas.py
|
||||
│ │ ├── card_schemas.py
|
||||
│ │ └── game_schemas.py
|
||||
│ ├── services/
|
||||
│ │ ├── card_service.py
|
||||
│ │ ├── deck_service.py
|
||||
│ │ ├── user_service.py
|
||||
│ │ ├── mtgjson_service.py
|
||||
│ │ └── game_service.py
|
||||
│ └── main.py
|
||||
├── mtg_rules_engine/ # Keep as-is (specialized)
|
||||
├── run.py # Simple startup script
|
||||
└── requirements.txt
|
||||
```
|
||||
|
||||
- **Startup**: `cd backend && python run.py`
|
||||
- **Database**: SQLite by default, PostgreSQL optional
|
||||
- **No Docker required** for development
|
||||
|
||||
### Next Steps
|
||||
1. Await Python game engine codebase delivery
|
||||
2. Deploy pre-built chat server (Tinode)
|
||||
3. Deploy pre-built audio server (Kurento or Janus)
|
||||
4. Begin Phase 3.1: Core Game Server implementation
|
||||
|
||||
---
|
||||
1. **Review entire codebase** - Understand all files and their relationships
|
||||
2. **Identify consolidation opportunities** - Which files can be merged?
|
||||
3. **Create simplified structure** - Build new architecture incrementally
|
||||
4. **Migrate functionality** - Move code to new structure
|
||||
5. **Test everything** - Ensure all endpoints work
|
||||
6. **Update documentation** - README, setup instructions
|
||||
7. **Remove Docker complexity** - Keep only for production if needed
|
||||
|
||||
## Phase 3: Multiplayer Game Server (In Progress)
|
||||
|
||||
### 3.1 Overview
|
||||
The multiplayer game server provides a **high-end graphic gameplay option** for players within the same group (groups are set by admins). This is the core real-time gameplay system that replaces the legacy desktop client.
|
||||
|
||||
**Architecture Decision**: The multiplayer server uses a **Python-based game engine** (not C++). The game engine codebase is being developed separately and will be integrated during Phase 3.
|
||||
|
||||
### 3.2 Architecture
|
||||
The multiplayer server follows a **modular architecture** with pre-built Docker containers for chat and audio:
|
||||
|
||||
```
|
||||
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
|
||||
│ Frontend │────▶│ Game Server │────▶│ Card Backend │
|
||||
│ (React/TS) │ │ (Python) │ │ (FastAPI) │
|
||||
└─────────────────┘ └─────────────────┘ └─────────────────┘
|
||||
│ │
|
||||
│ │
|
||||
▼ ▼
|
||||
┌─────────────────┐ ┌─────────────────┐
|
||||
│ Chat Server │ │ Audio Server │
|
||||
│ (Pre-built │ │ (Pre-built │
|
||||
│ Docker) │ │ Docker) │
|
||||
└─────────────────┘ └─────────────────┘
|
||||
```
|
||||
|
||||
**Key Decisions:**
|
||||
- **Chat**: Pre-built Docker container (Tinode recommended) - Option A architecture
|
||||
- **Audio**: Pre-built Docker container (Kurento or Janus) - Option A architecture
|
||||
- **Game Engine**: Python-based, codebase being developed separately
|
||||
|
||||
### 3.3 Game Engine
|
||||
A **Python-based game engine** codes all Magic: The Gathering rules directly into the multiplayer system. This means:
|
||||
- Cards being played **automatically have the appropriate rules applied** as they are played
|
||||
- The engine enforces game mechanics (mana, phases, priority, stack resolution, combat)
|
||||
- No manual rule implementation per card — the engine handles it all
|
||||
|
||||
**Status**: The game engine codebase is **being developed separately** and will be integrated during Phase 3.1 (Core Game Server).
|
||||
|
||||
**Integration Approach:**
|
||||
```python
|
||||
# game_engine/engine.py
|
||||
from game_engine.engine import GameEngine
|
||||
|
||||
engine = GameEngine()
|
||||
engine.load_card('lightning-bolt')
|
||||
result = engine.resolve_effect('lightning-bolt', target='player')
|
||||
```
|
||||
|
||||
**Docker Volume Mount:**
|
||||
```yaml
|
||||
volumes:
|
||||
- ./game-engine:/app/game_engine
|
||||
```
|
||||
|
||||
### 3.4 Graphics
|
||||
Graphics are handled in the **front-end development** (React/TypeScript with game board visualization). The multiplayer server provides:
|
||||
- Game state synchronization
|
||||
- Card data and metadata
|
||||
- Real-time updates via WebSocket
|
||||
|
||||
The server does not handle rendering — that's the frontend's responsibility.
|
||||
|
||||
### 3.5 Chat System
|
||||
The multiplayer server uses **pre-built Docker containers** for chat functionality:
|
||||
|
||||
#### Text-based Chat
|
||||
- **Solution**: Tinode (recommended) or SimpleWebSocketChat
|
||||
- **Architecture**: Option A (separate container)
|
||||
- **Integration**: WebSocket-based, frontend connects directly
|
||||
- **Features**: Room management, message persistence, moderation
|
||||
|
||||
**Docker Deployment (Tinode):**
|
||||
```bash
|
||||
docker run -d \
|
||||
--name tinode \
|
||||
-p 8080:8080 \
|
||||
-v ./tinode-data:/data \
|
||||
--restart unless-stopped \
|
||||
tinode/tinode
|
||||
```
|
||||
|
||||
**Frontend Integration:**
|
||||
```javascript
|
||||
import Tinode from 'tinode-sdk';
|
||||
|
||||
const tinode = new Tinode({ socket: 'ws://chat-server:8080' });
|
||||
await tinode.connect();
|
||||
await tinode.subscribe('game-room-123');
|
||||
tinode.on('message', (topic, msg) => console.log('Chat:', msg));
|
||||
tinode.sendMessage('game-room-123', { text: 'Hello game!' });
|
||||
```
|
||||
|
||||
#### Audio-based Chat
|
||||
- **Solution**: Kurento Media Server (recommended) or Janus Gateway
|
||||
- **Architecture**: Option A (separate container)
|
||||
- **Integration**: WebRTC-based, frontend connects directly
|
||||
- **Features**: Low latency, scalable, open source
|
||||
|
||||
**Docker Deployment (Kurento):**
|
||||
```bash
|
||||
docker run -d \
|
||||
--name kurento \
|
||||
-p 8888:8888 \
|
||||
-p 8443:8443 \
|
||||
--restart unless-stopped \
|
||||
kurento/kurento-media-server:latest
|
||||
```
|
||||
|
||||
**No custom chat/audio codebase needed** — both are pre-built solutions.
|
||||
|
||||
### 3.6 Integration with Card Backend
|
||||
The multiplayer server communicates with the card backend (current FastAPI app) via:
|
||||
|
||||
- **API Calls**: For authentication, user data, deck retrieval, card lookups
|
||||
- **Direct PSQL Queries**: For card data and user deck data
|
||||
|
||||
#### Integration Points
|
||||
- **Card Backend API** (`http://backend:8000`):
|
||||
- `GET /api/cards/{card_id}` — Get card details for in-game display
|
||||
- `GET /api/cards/search?q=...` — Search cards during gameplay
|
||||
- `GET /api/sets/` — List available sets for game formatting
|
||||
|
||||
- **PSQL Direct Access** (via shared connection string):
|
||||
- `mtgdata` database — Read card data (cards, sets, etc.)
|
||||
- `mtgonline` database — Read user decks (for deck validation, game setup)
|
||||
|
||||
#### API Endpoints for Play Backend
|
||||
- `POST /play/decks/{deck_id}/validate` — Validate deck against card database
|
||||
- `GET /play/users/{user_id}/decks` — Get user's FINAL decks for selection
|
||||
- `GET /play/cards/{card_id}` — Get card details for game board display
|
||||
- `POST /play/chat/text` — Send text message in game/room (via Tinode)
|
||||
- `POST /play/chat/audio` — Manage audio chat sessions (via Kurento)
|
||||
|
||||
### Summary of Work Required in Phase 3: Multiplayer Game Server
|
||||
|
||||
| Feature | Database | API | Notes |
|
||||
|---------|----------|-----|-------|
|
||||
| Game engine integration | N/A (Python codebase) | N/A | Being developed separately, integrate in Phase 3.1 |
|
||||
| Multiplayer WebSocket server | `mtgonline` (rooms, games) | WebSocket hub | Python/FastAPI |
|
||||
| Game state management | In-memory + DB persistence | State sync | Server-authoritative |
|
||||
| MTG rule enforcement | N/A (game engine) | Automatic | Cards auto-apply rules |
|
||||
| Text chat backend | N/A (Tinode) | WebSocket | Pre-built Docker container |
|
||||
| Audio chat backend | N/A (Kurento) | WebRTC | Pre-built Docker container |
|
||||
| Group-based access | `mtgonline` (groups) | Group validation | Admin-configured groups |
|
||||
| Deck validation | `mtgdata` + `mtgonline` | Validation endpoint | Against card database |
|
||||
| Card backend integration | Read-only | API consumer | Shared DB + REST API |
|
||||
|
||||
See **PHASE_3_PLANNING.md** for detailed task breakdown.
|
||||
|
||||
## User Data Schema & API
|
||||
|
||||
### Database Models (16 Tables)
|
||||
- **`users`** - User accounts with authentication
|
||||
- **`decks`** - User decks (DRAFT/FINAL status)
|
||||
- **`cards`** - User card collections
|
||||
- **`card_ownership`** - Card ownership tracking
|
||||
- **`win_streaks`** - Win/loss statistics
|
||||
- **`game_replays`** - Saved game replays (JSONB)
|
||||
- **`groups`** - User groups
|
||||
- **`group_members`** - Group membership
|
||||
- **`networks`** - Network accounts (Twitch, X, YouTube)
|
||||
- **`network_credentials`** - Network login info
|
||||
- **`preferences`** - User preferences (JSONB)
|
||||
- **`activity_log`** - User activity tracking (JSONB)
|
||||
- **`suggested_cards`** - Card suggestions
|
||||
- **`folders`** - Deck organization
|
||||
- **`game_logs`** - Game audit trail
|
||||
- **`user_decks`** - User deck storage (DRAFT/FINAL)
|
||||
|
||||
### API Endpoints
|
||||
|
||||
#### Replays (`/api/v1/user-data/replays`)
|
||||
- `POST /api/v1/user-data/replays/` - Save replay
|
||||
- `GET /api/v1/user-data/replays/{replay_id}` - Get replay
|
||||
- `DELETE /api/v1/user-data/replays/{replay_id}` - Delete replay
|
||||
|
||||
#### Card Collection (`/api/v1/user-data/cards`)
|
||||
- `GET /api/v1/user-data/cards/` - List user's cards
|
||||
- `POST /api/v1/user-data/cards/` - Add card to collection
|
||||
- `DELETE /api/v1/user-data/cards/{card_id}` - Remove card
|
||||
|
||||
#### Groups (`/api/v1/user-data/groups`)
|
||||
- `GET /api/v1/user-data/groups/` - List user's groups
|
||||
- `POST /api/v1/user-data/groups/` - Create group
|
||||
- `PATCH /api/v1/user-data/groups/{group_id}` - Update group
|
||||
- `DELETE /api/v1/user-data/groups/{group_id}` - Delete group
|
||||
|
||||
#### Networks (`/api/v1/user-data/networks`)
|
||||
- `GET /api/v1/user-data/networks/` - List network accounts
|
||||
- `POST /api/v1/user-data/networks/` - Add network
|
||||
- `PATCH /api/v1/user-data/networks/{network_id}` - Update network
|
||||
- `DELETE /api/v1/user-data/networks/{network_id}` - Remove network
|
||||
|
||||
#### Preferences (`/api/v1/user-data/preferences`)
|
||||
- `GET /api/v1/user-data/preferences/` - Get preferences
|
||||
- `PATCH /api/v1/user-data/preferences/` - Update preferences
|
||||
|
||||
#### Activity Log (`/api/v1/user-data/activity`)
|
||||
- `GET /api/v1/user-data/activity/` - List activity
|
||||
- `POST /api/v1/user-data/activity/` - Add activity entry
|
||||
|
||||
### Alembic Migrations
|
||||
- **Async configuration** with `run_sync` for database operations
|
||||
- **Initial migration**: `001_initial_user_schema.py` creates all 16 tables
|
||||
- **Migration script**: `scripts/run_migrations.sh` runs on container startup
|
||||
- **JSONB columns** used for flexible data storage (replay_data, activity_data, preferences)
|
||||
|
||||
### Architecture Decisions
|
||||
- **CASCADE deletes** for data integrity in related tables
|
||||
- **Composite unique constraints** for card collection uniqueness
|
||||
- **RESTful API design** with pagination support
|
||||
- **JWT authentication** for all endpoints
|
||||
- **Permission checks** for group/network management
|
||||
|
||||
## State File
|
||||
|
||||
Current state saved at: `/home/wall-o/projects/mtgonline/state.json`
|
||||
|
||||
```json
|
||||
{
|
||||
"project_summary": "MTG Online Backend API with PostgreSQL database. Implements card game platform with deck management, card import, and user data features. Phase 3 (multiplayer game server) is in progress with game engine code incoming.",
|
||||
"task_description": "Phase 3: Multiplayer game server with Cockatrice-inspired architecture, game engine integration, text/audio chat, and group-based gameplay",
|
||||
"current_step": "Preparation phase - reviewing Cockatrice architecture analysis, awaiting game engine code delivery, planning server scaffolding",
|
||||
"commit_hash": "1e7c762",
|
||||
"timestamp": "2026-07-24T04:35:00-04:00"
|
||||
}
|
||||
```
|
||||
|
||||
## Access Information
|
||||
|
||||
- **Backend API**: `http://localhost:5555`
|
||||
- **User Data API**: `http://localhost:5555/api/v1/user-data`
|
||||
- **Swagger Docs**: `http://localhost:5555/docs`
|
||||
- **Health Check**: `http://localhost:5555/health`
|
||||
- **Gitea**: `https://git.optimex.systems/admin/mtgonline`
|
||||
|
||||
### Key Files to Review
|
||||
|
||||
#### Current State
|
||||
1. `/home/wall-o/projects/mtgonline/state.json` - Current project state
|
||||
2. `/home/wall-o/projects/mtgonline/README.md` - Project documentation
|
||||
3. `/home/wall-o/projects/mtgonline/backend/README.md` - Backend documentation
|
||||
|
||||
#### Architecture & Configuration
|
||||
4. `/home/wall-o/projects/mtgonline/docker-compose.dev.yml` - Docker configuration
|
||||
5. `/home/wall-o/projects/mtgonline/backend/app/core/settings.py` - Application settings
|
||||
6. `/home/wall-o/projects/mtgonline/backend/app/main.py` - FastAPI entry point (user-data mounted at /api/v1/user-data)
|
||||
|
||||
#### User Data Models
|
||||
7. `/home/wall-o/projects/mtgonline/backend/app/models/user_data.py` - All user data models (16 tables)
|
||||
8. `/home/wall-o/projects/mtgonline/backend/alembic/versions/001_initial_user_schema.py` - Migration script
|
||||
|
||||
#### API Endpoints
|
||||
9. `/home/wall-o/projects/mtgonline/backend/app/routers/user_data.py` - User data API endpoints
|
||||
10. `/home/wall-o/projects/mtgonline/backend/app/schemas/user_data_schemas.py` - Pydantic schemas for user data
|
||||
|
||||
#### Documentation
|
||||
11. `/home/wall-o/projects/mtgonline/backend/API_DOCUMENTATION.md` - Comprehensive API documentation
|
||||
12. `/home/wall-o/projects/mtgonline/backend/TEST_PLAN.md` - Migration test plan
|
||||
13. `/home/wall-o/projects/mtgonline/ROADMAP.md` - Complete feature roadmap and timeline
|
||||
14. `/home/wall-o/projects/mtgonline/STATEMENT_OF_INTENT.md` - Project vision and objectives
|
||||
|
||||
## Environment
|
||||
|
||||
- **OS**: Linux 6.8.0-136-generic (x86_64)
|
||||
- **Docker**: Available
|
||||
- **Python**: 3.12.3
|
||||
- **Working Directory**: `/home/wall-o/projects/mtgonline`
|
||||
|
||||
---
|
||||
|
||||
**Last Updated**: 2026-07-25T19:58:00-04:00
|
||||
**Status**: Phase 1-2 Complete. Phase 3 (Multiplayer Game Server) in progress. Rules engine integrated in `backend/mtg_rules_engine/`.
|
||||
|
||||
**Next Action**: Begin Phase 3 — set up multiplayer server scaffolding, integrate rules engine, implement WebSocket hub with chat support.
|
||||
|
||||
**Timeline**:
|
||||
| Phase | Duration | Status |
|
||||
|-------|----------|--------|
|
||||
| Phase 1: Backend Foundation | 2 weeks | ✅ Complete |
|
||||
| Phase 2: Card Import & Deck Building | 2 weeks | ✅ Complete |
|
||||
| Phase 3: Multiplayer Game Server | TBD | 🔄 In Progress |
|
||||
| Phase 4: Frontend Integration | TBD | Pending |
|
||||
| Phase 5: Advanced Features | Ongoing | Future |
|
||||
Please start by exploring the project structure and understanding the current architecture before making changes. Focus on understanding what each file does and how they interact.
|
||||
|
||||
Reference in New Issue
Block a user