Add comprehensive documentation for MTG Online Backend
- Updated root README with current architecture (dual PostgreSQL, Redis, MTGJSON pipeline) - Created backend/README.md with detailed architecture, database setup, and troubleshooting - Updated state.json to reflect completed documentation phase
This commit is contained in:
@@ -1,291 +1,104 @@
|
|||||||
# MTG Online Web Application
|
# MTG Online Backend
|
||||||
|
|
||||||
A modern web-based implementation of the MTG Online multiplayer Magic: The Gathering platform.
|
A Python FastAPI application that processes Magic: The Gathering card data from [MTGJSON](https://mtgjson.com/) and stores it in PostgreSQL, with Redis for caching.
|
||||||
|
|
||||||
## Features
|
## Overview
|
||||||
|
|
||||||
- **User Authentication**: Secure JWT-based authentication with bcrypt password hashing
|
This project provides a backend API for a Magic: The Gathering Online platform. It downloads and processes MTGJSON v5 dataset dumps, loads them into a PostgreSQL database, and exposes REST endpoints for card data, user authentication, deck management, and game state.
|
||||||
- **Deck Building**: Full-featured deck editor with import/export in multiple formats
|
|
||||||
- **Real-Time Multiplayer**: WebSocket-based game server for live gameplay
|
|
||||||
- **Protocol Compatibility**: Compatible with MTG Online protocol buffer messages
|
|
||||||
- **Card Database**: Integration with MTJSON for comprehensive card data
|
|
||||||
- **Admin Tools**: Comprehensive moderation and administration dashboard
|
|
||||||
|
|
||||||
## Tech Stack
|
### Key Features
|
||||||
|
|
||||||
- **Backend**: Python 3.12, FastAPI, SQLAlchemy (async), PostgreSQL
|
- **MTGJSON Data Pipeline** — Downloads `AllPrintings.psql`, `AllIdentifiers.json`, `Keywords.json`, `CardTypes.json`, and `AllDeckFiles.zip` from MTGJSON v5 on startup or via a `POST /refresh` endpoint.
|
||||||
- **Frontend**: React, TypeScript, Zustand (coming soon)
|
- **Dual PostgreSQL** — Two databases: `mtgonline` for the application (users, decks, auth) and `mtgdata` for MTG card data.
|
||||||
- **Game Server**: WebSocket with real-time state synchronization
|
- **Redis Caching** — Used for card lookup caching and interaction pipeline state.
|
||||||
- **Authentication**: JWT tokens with bcrypt password hashing
|
- **REST API** — `/docs` (Swagger) available at runtime.
|
||||||
- **Database**: PostgreSQL with async driver (asyncpg)
|
|
||||||
- **Protocol**: Protocol buffer message compatibility
|
|
||||||
|
|
||||||
## Getting Started
|
## Architecture
|
||||||
|
|
||||||
|
```
|
||||||
|
mtgonline/
|
||||||
|
├── backend/ # FastAPI application
|
||||||
|
│ ├── app/
|
||||||
|
│ │ ├── core/ # Settings, database engines, Redis client
|
||||||
|
│ │ ├── models/ # SQLAlchemy ORM models (app + MTG)
|
||||||
|
│ │ ├── routers/ # API route modules
|
||||||
|
│ │ ├── schemas/ # Pydantic request/response schemas
|
||||||
|
│ │ ├── services/ # Business logic (MTGJSON manager, card DB, game server)
|
||||||
|
│ │ └── main.py # FastAPI app entry point
|
||||||
|
│ ├── scripts/ # Utility scripts (downloads, migrations, checks)
|
||||||
|
│ ├── Dockerfile
|
||||||
|
│ └── requirements.txt
|
||||||
|
├── docker-compose.dev.yml # Development stack (Postgres x2, Redis, Backend)
|
||||||
|
├── docker-compose.yml # Production stack
|
||||||
|
├── scripts/ # Shared utility scripts
|
||||||
|
└── README.md
|
||||||
|
```
|
||||||
|
|
||||||
|
### Data Flow
|
||||||
|
|
||||||
|
1. **On startup**, the backend connects to PostgreSQL (both instances) and Redis.
|
||||||
|
2. It checks `mtg_refresh_log` in the `mtgdata` database for existing data.
|
||||||
|
3. If no data exists, it downloads MTGJSON files from `https://mtgjson.com/api/v5/`, unzips if needed, and upserts them into `mtgdata` tables (`mtg_sets`, `mtg_cards`, etc.).
|
||||||
|
4. The data is then available via REST endpoints.
|
||||||
|
|
||||||
|
## Quick Start
|
||||||
|
|
||||||
### Prerequisites
|
### Prerequisites
|
||||||
|
|
||||||
- Docker and Docker Compose
|
- Docker and Docker Compose
|
||||||
- Python 3.12+ (for development)
|
|
||||||
- PostgreSQL 14+ (if running without Docker)
|
|
||||||
- Redis (optional, for caching)
|
|
||||||
- **MTGJSON data files** (see below)
|
|
||||||
|
|
||||||
### Installation
|
### Run the Stack
|
||||||
|
|
||||||
1. **Clone the repository**
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
git clone https://github.com/yourusername/mtgonline-web.git
|
cd /home/wall-o/projects/mtgonline
|
||||||
cd mtgonline-web
|
|
||||||
|
# Start all services (Postgres x2, Redis, Backend)
|
||||||
|
docker compose -f docker-compose.dev.yml up -d
|
||||||
|
|
||||||
|
# View logs
|
||||||
|
docker compose -f docker-compose.dev.yml logs -f backend
|
||||||
```
|
```
|
||||||
|
|
||||||
2. **Configure environment**
|
The backend will automatically download MTGJSON data on first startup (this may take several minutes).
|
||||||
|
|
||||||
|
### Access
|
||||||
|
|
||||||
|
| Service | Address |
|
||||||
|
|---------------|---------------------|
|
||||||
|
| Backend API | `http://localhost:5555` |
|
||||||
|
| Swagger Docs | `http://localhost:5555/docs` |
|
||||||
|
| Health Check | `http://localhost:5555/health` |
|
||||||
|
| PostgreSQL (app) | `localhost:5432` |
|
||||||
|
| PostgreSQL (MTG) | `localhost:5433` |
|
||||||
|
| Redis | `localhost:6379` |
|
||||||
|
|
||||||
|
### Manual Data Refresh
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
cp .env.example .env
|
# Trigger a manual MTGJSON refresh
|
||||||
# Edit .env with your configuration
|
curl -X POST http://localhost:5555/refresh
|
||||||
```
|
```
|
||||||
|
|
||||||
3. **Prepare MTGJSON data files**
|
## Environment Variables
|
||||||
|
|
||||||
The application requires MTGJSON data files. You have two options:
|
| Variable | Default | Description |
|
||||||
|
|----------|---------|-------------|
|
||||||
|
| `DATABASE_URL` | `postgresql+asyncpg://mtgonline_user:mtgonline_password@postgres:5432/mtgonline` | Primary database connection |
|
||||||
|
| `MTG_DATABASE_URL` | `postgresql+asyncpg://mtgonline_user:mtgonline_password@mtgdata:5432/mtgdata` | MTG data database connection |
|
||||||
|
| `REDIS_URL` | `redis://redis:6379` | Redis connection |
|
||||||
|
| `DATA_DIR` | `/app/data` | Directory for MTGJSON files |
|
||||||
|
| `DEBUG` | `False` | Enable debug logging |
|
||||||
|
|
||||||
**Option A: Download from MTGJSON (recommended)**
|
## Docker Cleanup
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Create data directory
|
# Stop and remove all containers
|
||||||
mkdir -p data
|
docker compose -f docker-compose.dev.yml down
|
||||||
|
|
||||||
# Download required files from MTGJSON
|
# Remove images and prune
|
||||||
# See scripts/download-mtgjson.sh for automated download
|
docker system prune -a --volumes
|
||||||
bash scripts/download-mtgjson.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
**Option B: Provide your own JSON files**
|
|
||||||
|
|
||||||
Place these files in the `data/` directory:
|
|
||||||
- `AllSetFiles/` (directory containing set JSON files)
|
|
||||||
- `AllPrintings.json`
|
|
||||||
- `AllIdentifiers.json`
|
|
||||||
- `CardTypes.json`
|
|
||||||
- `DeckList.json`
|
|
||||||
- `Keywords.json`
|
|
||||||
- `SetList.json`
|
|
||||||
|
|
||||||
### Running with Docker (recommended)
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Copy the data directory to a volume mount path
|
|
||||||
mkdir -p ~/mtg-data
|
|
||||||
cp -r data/* ~/mtg-data/
|
|
||||||
|
|
||||||
# Start the application
|
|
||||||
sudo docker compose up -d
|
|
||||||
```
|
|
||||||
|
|
||||||
The backend container will automatically process the JSON files from the mounted volume and populate the database.
|
|
||||||
|
|
||||||
## Project Structure
|
|
||||||
|
|
||||||
```
|
|
||||||
mtgonline-web/
|
|
||||||
├── backend/
|
|
||||||
│ ├── app/
|
|
||||||
│ │ ├── core/ # Core configuration and utilities
|
|
||||||
│ │ │ ├── settings.py # Application settings
|
|
||||||
│ │ │ ├── database.py # Database engine and sessions
|
|
||||||
│ │ │ └── security.py # Authentication and password hashing
|
|
||||||
│ │ ├── models/ # SQLAlchemy ORM models
|
|
||||||
│ │ │ └── models.py
|
|
||||||
│ │ ├── schemas/ # Pydantic schemas
|
|
||||||
│ │ │ ├── schemas.py
|
|
||||||
│ │ │ ├── proto_messages.py
|
|
||||||
│ │ │ └── protocol_constants.py
|
|
||||||
│ │ ├── routers/ # API route handlers
|
|
||||||
│ │ │ ├── auth.py
|
|
||||||
│ │ │ ├── users.py
|
|
||||||
│ │ │ ├── decks.py
|
|
||||||
│ │ │ ├── rooms.py
|
|
||||||
│ │ │ ├── games.py
|
|
||||||
│ │ │ ├── admin.py
|
|
||||||
│ │ │ └── ws.py
|
|
||||||
│ │ ├── services/ # Business logic services
|
|
||||||
│ │ │ ├── game_server.py
|
|
||||||
│ │ │ ├── card_database.py
|
|
||||||
│ │ │ └── deck_parser.py
|
|
||||||
│ │ └── main.py # FastAPI application
|
|
||||||
│ ├── tests/ # Test suite
|
|
||||||
│ ├── requirements.txt # Python dependencies
|
|
||||||
│ ├── pyproject.toml # Ruff configuration
|
|
||||||
│ └── .env.example # Environment template
|
|
||||||
├── frontend/ # React/TypeScript frontend (coming soon)
|
|
||||||
├── shared/ # Shared protocol definitions
|
|
||||||
│ └── proto/ # Protocol buffer definitions
|
|
||||||
└── README.md
|
|
||||||
```
|
|
||||||
|
|
||||||
|
|
||||||
## Docker Deployment
|
|
||||||
|
|
||||||
### 1. Prepare the Data Directory
|
|
||||||
|
|
||||||
Create a directory on your host machine to store the MTGJSON files:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
mkdir -p ~/mtg-data
|
|
||||||
cp -r data/* ~/mtg-data/
|
|
||||||
```
|
|
||||||
|
|
||||||
### 2. Configure Environment
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cp .env.example .env
|
|
||||||
# Edit .env with your configuration
|
|
||||||
```
|
|
||||||
|
|
||||||
### 3. Mount and Start
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Start with data directory mounted (Linux/macOS)
|
|
||||||
sudo docker compose up -d
|
|
||||||
|
|
||||||
# The data directory is automatically mounted to /app/data in the backend container
|
|
||||||
# Alternatively, specify a custom path:
|
|
||||||
sudo docker compose up -d --build
|
|
||||||
```
|
|
||||||
|
|
||||||
The backend will:
|
|
||||||
- Detect the JSON files in the mounted volume
|
|
||||||
- Upsert all data into the PostgreSQL database
|
|
||||||
- Start the FastAPI application
|
|
||||||
|
|
||||||
### 4. Verify
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Check backend logs
|
|
||||||
docker compose logs -f backend
|
|
||||||
|
|
||||||
# Test health endpoint
|
|
||||||
curl http://localhost:8000/health
|
|
||||||
```
|
|
||||||
|
|
||||||
### Updating Data
|
|
||||||
|
|
||||||
When MTGJSON releases updates:
|
|
||||||
1. Copy new `.json` files to `~/mtg-data/`
|
|
||||||
2. The backend will automatically process new files on next startup
|
|
||||||
3. Or trigger a manual refresh via `/refresh` endpoint
|
|
||||||
|
|
||||||
## Development
|
|
||||||
|
|
||||||
### Run Locally
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd backend
|
|
||||||
pip install -r requirements.txt
|
|
||||||
python -m uvicorn app.main:app --reload --host 0.0.0.0 --port 8000
|
|
||||||
```
|
|
||||||
|
|
||||||
### Run Tests
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd backend
|
|
||||||
pytest
|
|
||||||
```
|
|
||||||
|
|
||||||
## Project Structure
|
|
||||||
|
|
||||||
```
|
|
||||||
mtgonline-web/
|
|
||||||
├── backend/
|
|
||||||
│ ├── app/
|
|
||||||
│ │ ├── core/ # Core configuration and utilities
|
|
||||||
│ │ │ ├── settings.py # Application settings
|
|
||||||
│ │ │ ├── database.py # Database engine and sessions
|
|
||||||
│ │ │ └── security.py # Authentication and password hashing
|
|
||||||
│ │ ├── models/ # SQLAlchemy ORM models
|
|
||||||
│ │ │ └── models.py
|
|
||||||
│ │ ├── schemas/ # Pydantic schemas
|
|
||||||
│ │ │ ├── schemas.py
|
|
||||||
│ │ │ ├── proto_messages.py
|
|
||||||
│ │ │ └── protocol_constants.py
|
|
||||||
│ │ ├── routers/ # API route handlers
|
|
||||||
│ │ │ ├── auth.py
|
|
||||||
│ │ │ ├── users.py
|
|
||||||
│ │ │ ├── decks.py
|
|
||||||
│ │ │ ├── rooms.py
|
|
||||||
│ │ │ ├── games.py
|
|
||||||
│ │ │ ├── admin.py
|
|
||||||
│ │ │ └── ws.py
|
|
||||||
│ │ ├── services/ # Business logic services
|
|
||||||
│ │ │ ├── game_server.py
|
|
||||||
│ │ │ ├── card_database.py
|
|
||||||
│ │ │ └── deck_parser.py
|
|
||||||
│ │ └── main.py # FastAPI application
|
|
||||||
│ ├── tests/ # Test suite
|
|
||||||
│ ├── requirements.txt # Python dependencies
|
|
||||||
│ ├── pyproject.toml # Ruff configuration
|
|
||||||
│ └── .env.example # Environment template
|
|
||||||
├── frontend/ # React/TypeScript frontend (coming soon)
|
|
||||||
├── shared/ # Shared protocol definitions
|
|
||||||
│ └── proto/ # Protocol buffer definitions
|
|
||||||
├── data/ # MTGJSON data files (not in git)
|
|
||||||
├── scripts/ # Utility scripts
|
|
||||||
└── README.md
|
|
||||||
```
|
|
||||||
|
|
||||||
## API Endpoints
|
|
||||||
|
|
||||||
### Authentication
|
|
||||||
|
|
||||||
- `POST /api/v1/auth/login` - User login
|
|
||||||
- `POST /api/v1/auth/register` - User registration
|
|
||||||
- `POST /api/v1/auth/refresh` - Refresh access token
|
|
||||||
- `GET /api/v1/auth/me` - Get current user
|
|
||||||
|
|
||||||
### Users
|
|
||||||
|
|
||||||
- `GET /api/v1/users/{user_id}` - Get user by ID
|
|
||||||
- `PATCH /api/v1/users/{user_id}` - Update user profile
|
|
||||||
- `POST /api/v1/users/{user_id}/ban` - Ban user (admin)
|
|
||||||
- `POST /api/v1/users/{user_id}/unban` - Unban user (admin)
|
|
||||||
|
|
||||||
### Decks
|
|
||||||
|
|
||||||
- `GET /api/v1/decks/` - List decks
|
|
||||||
- `POST /api/v1/decks/` - Create deck
|
|
||||||
- `GET /api/v1/decks/{deck_id}` - Get deck
|
|
||||||
- `PATCH /api/v1/decks/{deck_id}` - Update deck
|
|
||||||
- `DELETE /api/v1/decks/{deck_id}` - Delete deck
|
|
||||||
- `GET /api/v1/decks/folders` - List folders
|
|
||||||
- `POST /api/v1/decks/folders` - Create folder
|
|
||||||
- `DELETE /api/v1/decks/folders/{folder_id}` - Delete folder
|
|
||||||
|
|
||||||
### Admin
|
|
||||||
|
|
||||||
- `GET /api/v1/admin/users` - List all users (admin)
|
|
||||||
- `GET /api/v1/admin/bans` - List all bans (admin)
|
|
||||||
- `POST /api/v1/admin/bans` - Create ban (admin)
|
|
||||||
- `POST /api/v1/admin/bans/{ban_id}/unban` - Unban user (admin)
|
|
||||||
- `GET /api/v1/admin/logs` - List game logs (admin)
|
|
||||||
|
|
||||||
## Testing
|
|
||||||
|
|
||||||
Run the test suite:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd backend
|
|
||||||
pytest
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## License
|
## License
|
||||||
|
|
||||||
MIT License
|
MIT
|
||||||
|
|
||||||
## Contributing
|
|
||||||
|
|
||||||
Contributions are welcome! Please open an issue or submit a pull request.
|
|
||||||
|
|
||||||
## Support
|
|
||||||
|
|
||||||
For questions or issues, please open a GitHub issue.
|
|
||||||
|
|||||||
@@ -0,0 +1,271 @@
|
|||||||
|
# MTG Online Backend
|
||||||
|
|
||||||
|
Python FastAPI application for processing MTGJSON card data and managing the MTG Online platform backend.
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
### Core Components
|
||||||
|
|
||||||
|
The backend consists of several key layers:
|
||||||
|
|
||||||
|
**Core Layer** (`app/core/`)
|
||||||
|
- `settings.py` — Application configuration using pydantic-settings with environment variable overrides
|
||||||
|
- `database.py` — Dual PostgreSQL engine setup (mtgonline app DB + mtgdata MTG cards DB)
|
||||||
|
- `redis_client.py` — Redis connection and caching utilities
|
||||||
|
|
||||||
|
**Models** (`app/models/`)
|
||||||
|
- `models.py` — SQLAlchemy ORM models for application data (users, decks, auth)
|
||||||
|
- `mtg_models.py` — ORM models for MTG card data
|
||||||
|
|
||||||
|
**Services** (`app/services/`)
|
||||||
|
- `mtgjson_manager.py` — Primary MTGJSON data pipeline (download, unzip, upsert)
|
||||||
|
- `mtgjson_downloader.py` — HTTP client for MTGJSON API
|
||||||
|
- `mtgjson_loader.py` — Data loading and transformation
|
||||||
|
- `mtgjson_uploader.py` — Database upsert operations
|
||||||
|
- `card_database.py` — Card data access layer
|
||||||
|
- `game_server.py` — Game state management
|
||||||
|
- `deck_parser.py` — Deck list parsing and validation
|
||||||
|
|
||||||
|
**Routers** (`app/routers/`)
|
||||||
|
- `auth.py` — JWT authentication endpoints
|
||||||
|
- `users.py` — User management
|
||||||
|
- `decks.py` — Deck CRUD operations
|
||||||
|
- `rooms.py` — Game room management
|
||||||
|
- `games.py` — Game state endpoints
|
||||||
|
- `admin.py` — Admin tools
|
||||||
|
- `card_router.py` — MTG card data API
|
||||||
|
- `interactions.py` — Card interaction engine
|
||||||
|
- `refresh.py` — MTGJSON data refresh endpoint
|
||||||
|
- `ws.py` — WebSocket support
|
||||||
|
|
||||||
|
**Schemas** (`app/schemas/`)
|
||||||
|
- `schemas.py` — Pydantic models for request/response validation
|
||||||
|
- `proto_messages.py` — Protocol buffer message definitions
|
||||||
|
- `protocol_constants.py` — MTG protocol constants
|
||||||
|
|
||||||
|
### Data Flow
|
||||||
|
|
||||||
|
```
|
||||||
|
MTGJSON API (https://mtgjson.com/api/v5/)
|
||||||
|
↓ download
|
||||||
|
MTGJSON files (AllPrintings.psql, AllIdentifiers.json, etc.)
|
||||||
|
↓ load/transform
|
||||||
|
PostgreSQL (mtgdata database)
|
||||||
|
↓ query
|
||||||
|
REST API / Swagger Docs
|
||||||
|
```
|
||||||
|
|
||||||
|
## Database Setup
|
||||||
|
|
||||||
|
### Primary Database (mtgonline)
|
||||||
|
- **Purpose**: Application data (users, decks, auth tokens)
|
||||||
|
- **Connection**: `postgresql+asyncpg://mtgonline_user:mtgonline_password@postgres:5432/mtgonline`
|
||||||
|
- **Port**: 5432 (internal), 5432:5432 (host)
|
||||||
|
|
||||||
|
### MTG Data Database (mtgdata)
|
||||||
|
- **Purpose**: MTG card data, sets, refresh logs
|
||||||
|
- **Connection**: `postgresql+asyncpg://mtgonline_user:mtgonline_password@mtgdata:5432/mtgdata`
|
||||||
|
- **Port**: 5432 (internal), 5433:5432 (host)
|
||||||
|
- **Tables**:
|
||||||
|
- `mtg_sets` — Card sets metadata
|
||||||
|
- `mtg_cards` — Individual card data
|
||||||
|
- `mtg_refresh_log` — Refresh history and status
|
||||||
|
|
||||||
|
### Redis
|
||||||
|
- **Purpose**: Caching, session management
|
||||||
|
- **Connection**: `redis://redis:6379`
|
||||||
|
- **Port**: 6379 (internal), 6379:6379 (host)
|
||||||
|
|
||||||
|
## MTGJSON Data Pipeline
|
||||||
|
|
||||||
|
### Downloaded Files
|
||||||
|
|
||||||
|
The backend downloads these files from MTGJSON v5:
|
||||||
|
- `AllPrintings.psql` — Main card data (PostgreSQL format)
|
||||||
|
- `AllIdentifiers.json` — Card identifiers (Multiverse, Scryfall, etc.)
|
||||||
|
- `Keywords.json` — Card keywords
|
||||||
|
- `CardTypes.json` — Card type definitions
|
||||||
|
- `AllDeckFiles.zip` — Deck files (must be unzipped)
|
||||||
|
|
||||||
|
### Refresh Logic
|
||||||
|
|
||||||
|
1. **On Startup**: Checks `mtg_refresh_log` for existing data
|
||||||
|
2. **If No Data**: Downloads and loads all MTGJSON files (may take minutes)
|
||||||
|
3. **Manual Refresh**: `POST /refresh` endpoint triggers immediate reload
|
||||||
|
4. **Logging**: All refreshes logged to `mtg_refresh_log` with status, timing, and counts
|
||||||
|
|
||||||
|
## Environment Variables
|
||||||
|
|
||||||
|
| Variable | Default | Description |
|
||||||
|
|----------|---------|-------------|
|
||||||
|
| `DATABASE_URL` | `postgresql+asyncpg://mtgonline_user:mtgonline_password@postgres:5432/mtgonline` | Primary database |
|
||||||
|
| `MTG_DATABASE_URL` | `postgresql+asyncpg://mtgonline_user:mtgonline_password@mtgdata:5432/mtgdata` | MTG data database |
|
||||||
|
| `REDIS_URL` | `redis://redis:6379` | Redis connection |
|
||||||
|
| `DATA_DIR` | `/app/data` | MTGJSON files directory |
|
||||||
|
| `UPLOAD_DIR` | `/app/uploads` | User uploads directory |
|
||||||
|
| `DEBUG` | `False` | Enable debug logging |
|
||||||
|
| `LOG_LEVEL` | `INFO` | Logging level |
|
||||||
|
| `SECRET_KEY` | `change-me-in-production` | JWT secret |
|
||||||
|
| `JWT_SECRET_KEY` | `change-me-in-production` | JWT signing key |
|
||||||
|
|
||||||
|
## Running the Backend
|
||||||
|
|
||||||
|
### Docker (Recommended)
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Build image
|
||||||
|
cd backend
|
||||||
|
docker build -t mtgonline-backend:latest .
|
||||||
|
|
||||||
|
# Run with dependencies
|
||||||
|
docker compose -f ../docker-compose.dev.yml up -d backend
|
||||||
|
```
|
||||||
|
|
||||||
|
### Local Development
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd backend
|
||||||
|
|
||||||
|
# Create venv
|
||||||
|
python -m venv venv
|
||||||
|
source venv/bin/activate
|
||||||
|
|
||||||
|
# Install dependencies
|
||||||
|
pip install -r requirements.txt
|
||||||
|
|
||||||
|
# Set environment variables
|
||||||
|
export DATABASE_URL="postgresql+asyncpg://mtgonline_user:mtgonline_password@localhost:5432/mtgonline"
|
||||||
|
export MTG_DATABASE_URL="postgresql+asyncpg://mtgonline_user:mtgonline_password@localhost:5433/mtgdata"
|
||||||
|
export REDIS_URL="redis://localhost:6379"
|
||||||
|
|
||||||
|
# Run server
|
||||||
|
python -m uvicorn app.main:app --reload --host 0.0.0.0 --port 8000
|
||||||
|
```
|
||||||
|
|
||||||
|
## API Endpoints
|
||||||
|
|
||||||
|
### Health & Status
|
||||||
|
- `GET /health` — Health check with MTGJSON status
|
||||||
|
- `GET /` — API info
|
||||||
|
|
||||||
|
### Authentication
|
||||||
|
- `POST /auth/login` — User login
|
||||||
|
- `POST /auth/register` — User registration
|
||||||
|
- `POST /auth/refresh` — Refresh JWT
|
||||||
|
- `GET /auth/me` — Current user
|
||||||
|
|
||||||
|
### Users
|
||||||
|
- `GET /users/{user_id}` — Get user
|
||||||
|
- `PATCH /users/{user_id}` — Update user
|
||||||
|
- `POST /users/{user_id}/ban` — Ban user (admin)
|
||||||
|
|
||||||
|
### Decks
|
||||||
|
- `GET /decks/` — List decks
|
||||||
|
- `POST /decks/` — Create deck
|
||||||
|
- `GET /decks/{deck_id}` — Get deck
|
||||||
|
- `PATCH /decks/{deck_id}` — Update deck
|
||||||
|
- `DELETE /decks/{deck_id}` — Delete deck
|
||||||
|
|
||||||
|
### MTG Cards
|
||||||
|
- `GET /api/cards/` — Search cards
|
||||||
|
- `GET /api/cards/{card_id}` — Get card
|
||||||
|
- `GET /api/sets/` — List sets
|
||||||
|
|
||||||
|
### Admin
|
||||||
|
- `GET /admin/users` — List all users
|
||||||
|
- `GET /admin/bans` — List bans
|
||||||
|
- `POST /admin/bans` — Create ban
|
||||||
|
|
||||||
|
### Data Management
|
||||||
|
- `POST /refresh` — Trigger MTGJSON refresh
|
||||||
|
|
||||||
|
### WebSocket
|
||||||
|
- `WS /ws/{room_id}` — Real-time game communication
|
||||||
|
|
||||||
|
## Utility Scripts
|
||||||
|
|
||||||
|
Located in `scripts/`:
|
||||||
|
- `download_mtgjson.py` — Manual MTGJSON download
|
||||||
|
- `check_mtgjson_status.py` — Verify data freshness
|
||||||
|
- `verify_mtgjson_data.py` — Data validation
|
||||||
|
- `inspect_db.py` — Database inspection
|
||||||
|
- `load_mtgjson_data.py` — Data loading
|
||||||
|
|
||||||
|
## Testing
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd backend
|
||||||
|
|
||||||
|
# Run tests
|
||||||
|
pytest
|
||||||
|
|
||||||
|
# Run with coverage
|
||||||
|
pytest --cov=app --cov-report=html
|
||||||
|
```
|
||||||
|
|
||||||
|
## Project Structure
|
||||||
|
|
||||||
|
```
|
||||||
|
backend/
|
||||||
|
├── app/
|
||||||
|
│ ├── __init__.py
|
||||||
|
│ ├── main.py # FastAPI app entry
|
||||||
|
│ ├── core/
|
||||||
|
│ │ ├── settings.py # Configuration
|
||||||
|
│ │ ├── database.py # Database engines
|
||||||
|
│ │ └── redis_client.py # Redis utilities
|
||||||
|
│ ├── models/
|
||||||
|
│ │ ├── models.py # App models
|
||||||
|
│ │ └── mtg_models.py # MTG models
|
||||||
|
│ ├── routers/
|
||||||
|
│ │ ├── auth.py # Auth endpoints
|
||||||
|
│ │ ├── users.py # User endpoints
|
||||||
|
│ │ ├── decks.py # Deck endpoints
|
||||||
|
│ │ ├── rooms.py # Room endpoints
|
||||||
|
│ │ ├── games.py # Game endpoints
|
||||||
|
│ │ ├── admin.py # Admin endpoints
|
||||||
|
│ │ ├── card_router.py # Card API
|
||||||
|
│ │ ├── interactions.py # Card interactions
|
||||||
|
│ │ ├── refresh.py # Data refresh
|
||||||
|
│ │ └── ws.py # WebSocket
|
||||||
|
│ ├── schemas/
|
||||||
|
│ │ ├── schemas.py # Pydantic models
|
||||||
|
│ │ ├── proto_messages.py # Protocol messages
|
||||||
|
│ │ └── protocol_constants.py
|
||||||
|
│ └── services/
|
||||||
|
│ ├── mtgjson_manager.py # MTGJSON pipeline
|
||||||
|
│ ├── mtgjson_downloader.py
|
||||||
|
│ ├── mtgjson_loader.py
|
||||||
|
│ ├── mtgjson_uploader.py
|
||||||
|
│ ├── card_database.py # Card data access
|
||||||
|
│ ├── game_server.py # Game logic
|
||||||
|
│ └── deck_parser.py # Deck parsing
|
||||||
|
├── scripts/ # Utility scripts
|
||||||
|
├── tests/ # Test suite
|
||||||
|
├── Dockerfile # Container build
|
||||||
|
├── requirements.txt # Python dependencies
|
||||||
|
├── pyproject.toml # Ruff config
|
||||||
|
├── .env.example # Environment template
|
||||||
|
└── setup_db.py # Database setup script
|
||||||
|
```
|
||||||
|
|
||||||
|
## Troubleshooting
|
||||||
|
|
||||||
|
### Backend can't connect to databases
|
||||||
|
- Verify all services are running: `docker compose -f ../docker-compose.dev.yml ps`
|
||||||
|
- Check logs: `docker compose -f ../docker-compose.dev.yml logs backend`
|
||||||
|
- Ensure environment variables match docker-compose.dev.yml
|
||||||
|
|
||||||
|
### MTGJSON download fails
|
||||||
|
- Check network connectivity to mtgjson.com
|
||||||
|
- Verify DATA_DIR has write permissions
|
||||||
|
- Check disk space: `df -h`
|
||||||
|
- Manual download: `python scripts/download_mtgjson.py`
|
||||||
|
|
||||||
|
### Database tables missing
|
||||||
|
- Run initialization: `docker exec -i mtgdata psql -U mtgonline_user mtgdata < /path/to/scripts/init-mtgdata.sql`
|
||||||
|
- Check tables: `docker exec mtgdata psql -U mtgonline_user mtgdata -c "\dt"`
|
||||||
|
|
||||||
|
## License
|
||||||
|
|
||||||
|
MIT
|
||||||
+26
-37
@@ -1,37 +1,26 @@
|
|||||||
# MTG Online Backend - Project State
|
{
|
||||||
|
"task_description": "Complete documentation and cleanup of MTG Online Backend project",
|
||||||
## Project Overview
|
"current_step": "Documentation complete, committing and pushing to Gitea, then cleaning up Docker",
|
||||||
A Python FastAPI backend service for MTG Online (Magic: The Gathering) that integrates with MTGJSON data and provides APIs for deck management, game rooms, and card data.
|
"files_created": [
|
||||||
|
"/home/wall-o/projects/mtgonline/README.md",
|
||||||
## Current Status: Ready for Deployment
|
"/home/wall-o/projects/mtgonline/backend/README.md"
|
||||||
|
],
|
||||||
### Completed Steps
|
"files_modified": [
|
||||||
1. ✅ Set up Docker Compose stack (PostgreSQL, Redis, Backend, Refresh services)
|
"/home/wall-o/projects/mtgonline/README.md"
|
||||||
2. ✅ Clean up corrupted MTGJSON data files
|
],
|
||||||
3. ✅ Downloaded fresh MTGJSON data from API
|
"decisions": [
|
||||||
4. ✅ Simplified MTGJSON data management service
|
"Updated root README to reflect current architecture (dual PostgreSQL, Redis, MTGJSON pipeline)",
|
||||||
5. ✅ Updated database schema to support all JSON data
|
"Created comprehensive backend README with architecture, database setup, MTGJSON pipeline, and troubleshooting",
|
||||||
6. ✅ Fixed directory naming issues
|
"Hardcoded environment variables in docker-compose.dev.yml to prevent connection issues",
|
||||||
7. ✅ Rebuilt and redeployed backend
|
"Backend successfully connects to mtgdata:5432/mtgdata (not localhost)"
|
||||||
|
],
|
||||||
### Next Steps
|
"next_steps": [
|
||||||
- Verify backend health and database connectivity
|
"Commit and push changes to Gitea repository",
|
||||||
- Test the refresh cycle
|
"Stop and destroy all Docker containers",
|
||||||
- Confirm all MTGJSON data is properly stored
|
"Clear Docker build cache",
|
||||||
|
"Run docker system prune to remove all traces"
|
||||||
## Key Files
|
],
|
||||||
- Dockerfile: `/home/wall-o/projects/mtgonline/backend/Dockerfile`
|
"blockers": [],
|
||||||
- docker-compose.yml: `/home/wall-o/projects/mtgonline/docker-compose.yml`
|
"commit_hash": "",
|
||||||
- .env.local: `/home/wall-o/projects/mtgonline/.env.local`
|
"timestamp": "2026-07-21T03:47:00Z"
|
||||||
- Database schema: `/home/wall-o/projects/mtgonline/backend/scripts/init-mtgdata.sql`
|
}
|
||||||
- Main application: `/home/wall-o/projects/mtgonline/backend/app/main.py`
|
|
||||||
- Database connection: `/home/wall-o/projects/mtgonline/backend/app/core/database.py`
|
|
||||||
- MTGJSON manager: `/home/wall-o/projects/mtgonline/backend/app/services/mtgjson_manager.py`
|
|
||||||
|
|
||||||
## Technical Details
|
|
||||||
- Python 3.12 with FastAPI
|
|
||||||
- PostgreSQL 16 with asyncpg
|
|
||||||
- Redis 7 for caching
|
|
||||||
- MTGJSON data integration for card database
|
|
||||||
- Docker Compose for orchestration
|
|
||||||
- Volume-based persistence for databases and MTG data
|
|
||||||
|
|||||||
Reference in New Issue
Block a user