# MTG Online Backend

Python FastAPI application for processing MTGJSON card data and managing the MTG Online platform backend.

## Architecture

### Core Components

The backend consists of several key layers:

**Core Layer** (`app/core/`)
- `settings.py` — Application configuration using pydantic-settings with environment variable overrides
- `database.py` — Dual PostgreSQL engine setup (mtgonline app DB + mtgdata MTG cards DB)
- `redis_client.py` — Redis connection and caching utilities

**Models** (`app/models/`)
- `models.py` — SQLAlchemy ORM models for application data (users, decks, auth)
- `mtg_models.py` — ORM models for MTG card data

**Services** (`app/services/`)
- `mtgjson_manager.py` — Primary MTGJSON data pipeline (download, unzip, upsert)
- `mtgjson_downloader.py` — HTTP client for MTGJSON API
- `mtgjson_loader.py` — Data loading and transformation
- `mtgjson_uploader.py` — Database upsert operations
- `card_database.py` — Card data access layer
- `game_server.py` — Game state management
- `deck_parser.py` — Deck list parsing and validation

**Routers** (`app/routers/`)
- `auth.py` — JWT authentication endpoints
- `users.py` — User management
- `decks.py` — Deck CRUD operations
- `rooms.py` — Game room management
- `games.py` — Game state endpoints
- `admin.py` — Admin tools
- `card_router.py` — MTG card data API
- `interactions.py` — Card interaction engine
- `refresh.py` — MTGJSON data refresh endpoint
- `ws.py` — WebSocket support

**Schemas** (`app/schemas/`)
- `schemas.py` — Pydantic models for request/response validation
- `proto_messages.py` — Protocol buffer message definitions
- `protocol_constants.py` — MTG protocol constants

### Data Flow

```
MTGJSON API (https://mtgjson.com/api/v5/)
    ↓ download
MTGJSON files (AllPrintings.psql, AllIdentifiers.json, etc.)
    ↓ load/transform
PostgreSQL (mtgdata database)
    ↓ query
REST API / Swagger Docs
```

## Database Setup

### Primary Database (mtgonline)
- **Purpose**: Application data (users, decks, auth tokens)
- **Connection**: `postgresql+asyncpg://mtgonline_user:mtgonline_password@postgres:5432/mtgonline`
- **Port**: 5432 (internal), 5432:5432 (host)

### MTG Data Database (mtgdata)
- **Purpose**: MTG card data, sets, refresh logs
- **Connection**: `postgresql+asyncpg://mtgonline_user:mtgonline_password@mtgdata:5432/mtgdata`
- **Port**: 5432 (internal), 5433:5432 (host)
- **Tables**:
  - `mtg_sets` — Card sets metadata
  - `mtg_cards` — Individual card data
  - `mtg_refresh_log` — Refresh history and status

### Redis
- **Purpose**: Caching, session management
- **Connection**: `redis://redis:6379`
- **Port**: 6379 (internal), 6379:6379 (host)

## MTGJSON Data Pipeline

### Downloaded Files

The backend downloads these files from MTGJSON v5:
- `AllPrintings.psql` — Main card data (PostgreSQL format)
- `AllIdentifiers.json` — Card identifiers (Multiverse, Scryfall, etc.)
- `Keywords.json` — Card keywords
- `CardTypes.json` — Card type definitions
- `AllDeckFiles.zip` — Deck files (must be unzipped)

### Refresh Logic

1. **On Startup**: Checks `mtg_refresh_log` for existing data
2. **If No Data**: Downloads and loads all MTGJSON files (may take minutes)
3. **Manual Refresh**: `POST /refresh` endpoint triggers immediate reload
4. **Logging**: All refreshes logged to `mtg_refresh_log` with status, timing, and counts

## Environment Variables

| Variable | Default | Description |
|----------|---------|-------------|
| `DATABASE_URL` | `postgresql+asyncpg://mtgonline_user:mtgonline_password@postgres:5432/mtgonline` | Primary database |
| `MTG_DATABASE_URL` | `postgresql+asyncpg://mtgonline_user:mtgonline_password@mtgdata:5432/mtgdata` | MTG data database |
| `REDIS_URL` | `redis://redis:6379` | Redis connection |
| `DATA_DIR` | `/app/data` | MTGJSON files directory |
| `UPLOAD_DIR` | `/app/uploads` | User uploads directory |
| `DEBUG` | `False` | Enable debug logging |
| `LOG_LEVEL` | `INFO` | Logging level |
| `SECRET_KEY` | `change-me-in-production` | JWT secret |
| `JWT_SECRET_KEY` | `change-me-in-production` | JWT signing key |

## Running the Backend

### Docker (Recommended)

```bash
# Build image
cd backend
docker build -t mtgonline-backend:latest .

# Run with dependencies
docker compose -f ../docker-compose.dev.yml up -d backend
```

### Local Development

```bash
cd backend

# Create venv
python -m venv venv
source venv/bin/activate

# Install dependencies
pip install -r requirements.txt

# Set environment variables
export DATABASE_URL="postgresql+asyncpg://mtgonline_user:mtgonline_password@localhost:5432/mtgonline"
export MTG_DATABASE_URL="postgresql+asyncpg://mtgonline_user:mtgonline_password@localhost:5433/mtgdata"
export REDIS_URL="redis://localhost:6379"

# Run server
python -m uvicorn app.main:app --reload --host 0.0.0.0 --port 8000
```

## API Endpoints

### Health & Status
- `GET /health` — Health check with MTGJSON status
- `GET /` — API info

### Authentication
- `POST /auth/login` — User login
- `POST /auth/register` — User registration
- `POST /auth/refresh` — Refresh JWT
- `GET /auth/me` — Current user

### Users
- `GET /users/{user_id}` — Get user
- `PATCH /users/{user_id}` — Update user
- `POST /users/{user_id}/ban` — Ban user (admin)

### Decks
- `GET /decks/` — List decks
- `POST /decks/` — Create deck
- `GET /decks/{deck_id}` — Get deck
- `PATCH /decks/{deck_id}` — Update deck
- `DELETE /decks/{deck_id}` — Delete deck

### MTG Cards
- `GET /api/cards/` — Search cards
- `GET /api/cards/{card_id}` — Get card
- `GET /api/sets/` — List sets

### Admin
- `GET /admin/users` — List all users
- `GET /admin/bans` — List bans
- `POST /admin/bans` — Create ban

### Data Management
- `POST /refresh` — Trigger MTGJSON refresh

### WebSocket
- `WS /ws/{room_id}` — Real-time game communication

## Utility Scripts

Located in `scripts/`:
- `download_mtgjson.py` — Manual MTGJSON download
- `check_mtgjson_status.py` — Verify data freshness
- `verify_mtgjson_data.py` — Data validation
- `inspect_db.py` — Database inspection
- `load_mtgjson_data.py` — Data loading

## Testing

```bash
cd backend

# Run tests
pytest

# Run with coverage
pytest --cov=app --cov-report=html
```

## Project Structure

```
backend/
├── app/
│   ├── __init__.py
│   ├── main.py                 # FastAPI app entry
│   ├── core/
│   │   ├── settings.py         # Configuration
│   │   ├── database.py         # Database engines
│   │   └── redis_client.py     # Redis utilities
│   ├── models/
│   │   ├── models.py           # App models
│   │   └── mtg_models.py       # MTG models
│   ├── routers/
│   │   ├── auth.py             # Auth endpoints
│   │   ├── users.py            # User endpoints
│   │   ├── decks.py            # Deck endpoints
│   │   ├── rooms.py            # Room endpoints
│   │   ├── games.py            # Game endpoints
│   │   ├── admin.py            # Admin endpoints
│   │   ├── card_router.py      # Card API
│   │   ├── interactions.py     # Card interactions
│   │   ├── refresh.py          # Data refresh
│   │   └── ws.py               # WebSocket
│   ├── schemas/
│   │   ├── schemas.py          # Pydantic models
│   │   ├── proto_messages.py   # Protocol messages
│   │   └── protocol_constants.py
│   └── services/
│       ├── mtgjson_manager.py  # MTGJSON pipeline
│       ├── mtgjson_downloader.py
│       ├── mtgjson_loader.py
│       ├── mtgjson_uploader.py
│       ├── card_database.py    # Card data access
│       ├── game_server.py      # Game logic
│       └── deck_parser.py      # Deck parsing
├── scripts/                    # Utility scripts
├── tests/                      # Test suite
├── Dockerfile                  # Container build
├── requirements.txt            # Python dependencies
├── pyproject.toml              # Ruff config
├── .env.example                # Environment template
└── setup_db.py                 # Database setup script
```

## Troubleshooting

### Backend can't connect to databases
- Verify all services are running: `docker compose -f ../docker-compose.dev.yml ps`
- Check logs: `docker compose -f ../docker-compose.dev.yml logs backend`
- Ensure environment variables match docker-compose.dev.yml

### MTGJSON download fails
- Check network connectivity to mtgjson.com
- Verify DATA_DIR has write permissions
- Check disk space: `df -h`
- Manual download: `python scripts/download_mtgjson.py`

### Database tables missing
- Run initialization: `docker exec -i mtgdata psql -U mtgonline_user mtgdata < /path/to/scripts/init-mtgdata.sql`
- Check tables: `docker exec mtgdata psql -U mtgonline_user mtgdata -c "\dt"`

## License

MIT