mtgonline/HANDOFF.md

MTG Online Backend - Handoff Document

Project Overview

Project Name: MTG Online Backend Location: /home/wall-o/projects/mtgonline Purpose: Python FastAPI application that processes Magic: The Gathering card data from MTGJSON and stores it in PostgreSQL databases.

Vision Statement (from STATEMENT_OF_INTENT.md)

MTG Online Web — A modern, web-based implementation of the MTG Online multiplayer Magic: The Gathering platform.

To build a fully-featured, open-source multiplayer Magic: The Gathering platform that runs entirely in modern web browsers, eliminating the need for desktop software installations while maintaining compatibility with the existing MTG Online ecosystem.

Core Objectives

User Experience

• Intuitive, modern interface that rivals native desktop applications
• Real-time multiplayer gameplay with minimal latency
• Seamless deck building with import/export from MTG Online
• Responsive design that works across all screen sizes

Technical Excellence

• Backend: Python 3.12 + FastAPI with async architecture
• Database: PostgreSQL with async SQLAlchemy ORM
• Real-time: WebSocket-based game server for live multiplayer
• Protocol: Full compatibility with MTG Online protocol buffer messages

Feature Parity with Desktop

• User authentication and account management
• Deck creation, editing, and storage (per-user)
• Multiplayer game rooms with real-time state sync
• Card game mechanics (mana, phases, priority, stack)
• Admin/moderation tools (ban, warn, log viewing)
• Card database integration with comprehensive card data

Performance Requirements

• API response times < 100ms for 95% of requests
• WebSocket latency < 50ms for game state updates
• Support 1000+ concurrent users
• Sub-second page loads with proper caching

Success Criteria

• Users can create accounts and authenticate securely
• Users can create, edit, and manage decks (per-user)
• Users can join and play multiplayer games in real-time
• Game state syncs correctly across all connected players
• Admin users can manage accounts and moderate games
• Deck formats are compatible with MTG Online desktop client

Current Status

All planned tasks have been completed:

• ✅ Documentation created (root README.md + backend/README.md)
• ✅ State.json updated
• ✅ Commit pushed to Gitea (commit 46abfe5)
• ✅ Docker cleanup completed (all containers, images, volumes removed)

Architecture Summary

Tech Stack

• Backend: Python 3.12, FastAPI, SQLAlchemy (async), asyncpg
• Databases: Dual PostgreSQL (14-alpine)
  • Primary: mtgonline database (users, decks, auth)
  • MTG Data: mtgdata database (card data, sets)
• Cache: Redis 7-alpine
• Protocol: Protocol buffer message compatibility

Service Architecture

mtgonline/
├── backend/                    # FastAPI application
│   ├── app/
│   │   ├── core/              # Settings, database engines, Redis client
│   │   ├── models/            # SQLAlchemy ORM models
│   │   ├── routers/           # API route modules (auth, users, decks, rooms, games, admin, cards, interactions, refresh, ws)
│   │   ├── schemas/           # Pydantic request/response schemas
│   │   ├── services/          # Business logic (MTGJSON manager, card DB, game server, deck parser)
│   │   └── main.py            # FastAPI app entry point
│   ├── scripts/               # Utility scripts (downloads, migrations, checks)
│   ├── Dockerfile
│   ├── requirements.txt
│   └── .env.example
├── docker-compose.dev.yml     # Development stack
├── docker-compose.yml         # Production stack
└── README.md                  # Project documentation

Data Flow

1. Backend connects to PostgreSQL (both instances) and Redis on startup
2. Checks mtg_refresh_log in mtgdata database for existing data
3. If no data exists, downloads MTGJSON files from https://mtgjson.com/api/v5/
4. Upserts data into mtgdata tables (mtg_sets, mtg_cards, etc.)
5. Data available via REST endpoints

Key Database Connections

• Primary DB: postgresql+asyncpg://mtgonline_user:mtgonline_password@postgres:5432/mtgonline
• MTG DB: postgresql+asyncpg://mtgonline_user:mtgonline_password@mtgdata:5432/mtgdata
• Redis: redis://redis:6379

Environment Configuration

Docker Compose Dev Environment Variables

environment:
  DATABASE_URL: "postgresql+asyncpg://mtgonline_user:mtgonline_password@postgres:5432/mtgonline"
  MTG_DATABASE_URL: "postgresql+asyncpg://mtgonline_user:mtgonline_password@mtgdata:5432/mtgdata"
  REDIS_URL: "redis://redis:6379"

Service Ports (Host to Container)

• PostgreSQL: 5432:5432
• MTG Data PostgreSQL: 5433:5432
• Redis: 6379:6379
• Backend: 5555:8000

API Endpoints

Health & Status

• GET /health - Health check with MTGJSON status
• GET / - API info

Authentication

• POST /auth/login - User login
• POST /auth/register - User registration
• POST /auth/refresh - Refresh JWT
• GET /auth/me - Current user

Users

• GET /users/{user_id} - Get user
• PATCH /users/{user_id} - Update user
• POST /users/{user_id}/ban - Ban user (admin)

Decks

• GET /decks/ - List decks
• POST /decks/ - Create deck
• GET /decks/{deck_id} - Get deck
• PATCH /decks/{deck_id} - Update deck
• DELETE /decks/{deck_id} - Delete deck

MTG Cards

• GET /api/cards/ - Search cards
• GET /api/cards/{card_id} - Get card
• GET /api/sets/ - List sets

Admin

• GET /admin/users - List all users
• GET /admin/bans - List bans
• POST /admin/bans - Create ban

Data Management

• POST /refresh - Trigger MTGJSON refresh

WebSocket

• WS /ws/{room_id} - Real-time game communication

MTGJSON Data Pipeline

Downloaded Files

• AllPrintings.psql - Main card data (PostgreSQL format)
• AllIdentifiers.json - Card identifiers
• Keywords.json - Card keywords
• CardTypes.json - Card type definitions
• AllDeckFiles.zip - Deck files (unzipped on load)

Refresh Logic

1. On Startup: Checks mtg_refresh_log for existing data
2. If No Data: Downloads and loads all MTGJSON files (may take minutes)
3. Manual Refresh: POST /refresh triggers immediate reload
4. Logging: All refreshes logged to mtg_refresh_log with status, timing, and counts

Git Repository

Gitea Repository: https://git.optimex.systems/admin/mtgonline.git Credentials: Located at /home/wall-o/projects/gitea_credentials.txt Current Branch: main Last Commit: 46abfe5 - "Add comprehensive documentation for MTG Online Backend"

Commit History

46abfe5 - Add comprehensive documentation for MTG Online Backend
6f01e2d - Initial project setup

Running the Project

Start Services

cd /home/wall-o/projects/mtgonline
docker compose -f docker-compose.dev.yml up -d

Check Services

docker compose -f docker-compose.dev.yml ps
docker compose -f docker-compose.dev.yml logs -f backend

Stop Services

docker compose -f docker-compose.dev.yml down

Full Cleanup

# Stop and remove all containers
docker compose -f docker-compose.dev.yml down

# Remove images
docker images rm mtgonline-backend:latest postgres:14-alpine redis:7-alpine

# Clear build cache and prune
docker builder prune -af
docker system prune -af --volumes

Testing

Run Tests

cd /home/wall-o/projects/mtgonline/backend
docker exec -it <backend_container_id> pytest
# OR
cd /home/wall-o/projects/mtgonline/backend
pytest

Database Verification

# Check MTG data tables
docker exec <mtgdata_container_id> psql -U mtgonline_user mtgdata -c "\dt"

# Check refresh log
docker exec <mtgdata_container_id> psql -U mtgonline_user mtgdata -c "SELECT * FROM mtg_refresh_log ORDER BY refresh_time DESC LIMIT 5;"

Troubleshooting

Backend can't connect to databases

• Verify all services are running: docker compose -f docker-compose.dev.yml ps
• Check logs: docker compose -f docker-compose.dev.yml logs backend
• Ensure environment variables match docker-compose.dev.yml

MTGJSON download fails

• Check network connectivity to mtgjson.com
• Verify DATA_DIR has write permissions
• Check disk space: df -h
• Manual download: python scripts/download_mtgjson.py

Database tables missing

• Run initialization: docker exec -i <mtgdata_container_id> psql -U mtgonline_user mtgdata < /path/to/scripts/init-mtgdata.sql
• Check tables: docker exec <mtgdata_container_id> psql -U mtgonline_user mtgdata -c "\dt"

CORS errors

• Check CORS_ORIGINS setting in app/core/settings.py
• Ensure frontend URL matches allowed origins

Next Phase: Backend Expansion for Frontend Support

Primary Focus: Expand the PostgreSQL database schema and API endpoints to support frontend deckbuilding and gameplay features on a per-user basis.

Database Schema Expansion

• Per-user deck storage (decks, folders, custom card sets)
• Game state persistence (saved games, match history, game logs)
• User card collection tracking (owned cards, favorites)
• Game room state management (active games, waiting lists)
• Tournament and custom rule support

API Endpoints to Implement

• Deck CRUD with user ownership and sharing
• Game room creation, joining, and state management
• Real-time WebSocket endpoints for multiplayer gameplay
• Card collection APIs (search, filter, organize)
• Game history and replay APIs
• Admin tools for game monitoring and moderation

Frontend Features to Support (from ROADMAP.md Phase 2)

• Deck Builder: Card search with filters (name, color, type, set), drag-and-drop editor, import/export formats
• Game Interface: Game board visualization, player zones (hand, library, graveyard, exile, command), real-time updates
• Chat System: Room chat, game chat, player list, moderator tools
• Admin Dashboard: User management, ban/unban controls, game logs, system statistics

Integration Requirements (from ROADMAP.md Phase 3)

• WebSocket client with reconnection logic
• Card database caching and search functionality
• Game logic implementation (turn-based state, mana tracking, stack resolution)
• Performance optimization (virtual scrolling, memoization, code splitting)

Deployment & Production (from ROADMAP.md Phase 4)

• Docker Compose for development and production
• CI/CD pipeline with GitHub Actions
• Security hardening (rate limiting, input validation, HTTPS)
• Monitoring and alerting (structured logging, error tracking)

See ROADMAP.md for complete feature specifications and timeline.

State File

Current state saved at: /home/wall-o/projects/mtgonline/state.json

{
  "task_description": "Complete documentation and cleanup of MTG Online Backend project",
  "current_step": "All tasks completed: documentation, commit/push to Gitea, Docker cleanup",
  "files_created": [
    "/home/wall-o/projects/mtgonline/README.md",
    "/home/wall-o/projects/mtgonline/backend/README.md"
  ],
  "files_modified": [
    "/home/wall-o/projects/mtgonline/README.md",
    "/home/wall-o/projects/mtgonline/state.json"
  ],
  "decisions": [
    "Updated root README with current architecture (dual PostgreSQL, Redis, MTGJSON pipeline)",
    "Created comprehensive backend README with architecture, database setup, and troubleshooting",
    "Hardcoded environment variables in docker-compose.dev.yml to prevent connection issues",
    "Backend successfully connects to mtgdata:5432/mtgdata (not localhost)",
    "All Docker resources cleaned up: containers, images, volumes, networks"
  ],
  "next_steps": [],
  "blockers": [],
  "commit_hash": "46abfe5",
  "timestamp": "2026-07-21T03:56:00Z"
}

Access Information

• Backend API: http://localhost:5555
• Swagger Docs: http://localhost:5555/docs
• Health Check: http://localhost:5555/health
• Gitea: https://git.optimex.systems/admin/mtgonline

Key Files to Review

Current State

1. /home/wall-o/projects/mtgonline/state.json - Current project state
2. /home/wall-o/projects/mtgonline/README.md - Project documentation
3. /home/wall-o/projects/mtgonline/backend/README.md - Backend documentation

Architecture & Configuration

4. /home/wall-o/projects/mtgonline/docker-compose.dev.yml - Docker configuration
5. /home/wall-o/projects/mtgonline/backend/app/core/settings.py - Application settings
6. /home/wall-o/projects/mtgonline/backend/app/main.py - FastAPI entry point

MTGJSON Integration

7. /home/wall-o/projects/mtgonline/backend/app/services/mtgjson_manager.py - MTGJSON pipeline

Strategic Documents

8. /home/wall-o/projects/mtgonline/ROADMAP.md - Complete feature roadmap and timeline
9. /home/wall-o/projects/mtgonline/STATEMENT_OF_INTENT.md - Project vision and objectives

Database & Models

10. /home/wall-o/projects/mtgonline/backend/app/models/models.py - SQLAlchemy ORM models
11. /home/wall-o/projects/mtgonline/backend/app/models/mtg_models.py - MTG-specific models

APIs

12. /home/wall-o/projects/mtgonline/backend/app/routers/ - All API route modules
13. /home/wall-o/projects/mtgonline/backend/app/schemas/ - Pydantic request/response schemas

Environment

• OS: Linux 6.8.0-136-generic (x86_64)
• Docker: Available
• Python: 3.12.3
• Working Directory: /home/wall-o/projects/mtgonline

---

Last Updated: 2026-07-21T03:56:00Z Status: Phase 1 Complete. Ready for Phase 2: Backend Expansion for Frontend Support.

Next Action: Begin backend database schema expansion and API development for deckbuilding and gameplay features.

Timeline:

Phase	Duration	Status
Phase 1: Backend Foundation	2 weeks	✅ Complete
Phase 2: Frontend Development	4 weeks	Not Started
Phase 3: Integration & Polish	2 weeks	Not Started
Phase 4: Deployment & Production	1 week	Not Started
Phase 5: Advanced Features	Ongoing	Future