akadmin
a01e33eb5e
Removed obsolete scripts that were not imported or used: - card_interaction_rule_engine.py - card_profile_extractor.py - create_card_interaction_graph.py - interaction_determinator.py - interaction_pipeline.py - interaction_recommender.py - interaction_schema.py - recommendation_engine.py - migrate_complete.py - migrate_schema.py - test_interaction_determinator.py - check_mtgjson_full.py - check_mtgjson_status.py - verify_integration.py - verify_mtgjson_data.py - sanity_check_mtgjson.py - investigate_sets.py - inspect_db.py - code_review.md - monitor/mtg_monitor.py Removed test artifacts: - test.db - test_download.py - test_system.py - setup_db.py - BACKEND_TESTING_SUMMARY.md - CHAT_PROMPT_TEST.md - CONTINUATION_PROMPT.md - PORTED_STATE.md - SPEC_synergy-mapping-engine.md - STATE.md - SUPPORTED_FILE_TYPES.md Removed sensitive/environment files: - .env.local - state.json (backend) Cleaned up: - __pycache__ directories - venv directory Backend scripts/ directory now contains only essential data loading and maintenance scripts.
105 lines
3.8 KiB
Markdown
105 lines
3.8 KiB
Markdown
# MTG Online Backend
|
|
|
|
A Python FastAPI application that processes Magic: The Gathering card data from [MTGJSON](https://mtgjson.com/) and stores it in PostgreSQL, with Redis for caching.
|
|
|
|
## Overview
|
|
|
|
This project provides a backend API for a Magic: The Gathering Online platform. It downloads and processes MTGJSON v5 dataset dumps, loads them into a PostgreSQL database, and exposes REST endpoints for card data, user authentication, deck management, and game state.
|
|
|
|
### Key Features
|
|
|
|
- **MTGJSON Data Pipeline** — Downloads `AllPrintings.psql`, `AllIdentifiers.json`, `Keywords.json`, `CardTypes.json`, and `AllDeckFiles.zip` from MTGJSON v5 on startup or via a `POST /refresh` endpoint.
|
|
- **Dual PostgreSQL** — Two databases: `mtgonline` for the application (users, decks, auth) and `mtgdata` for MTG card data.
|
|
- **Redis Caching** — Used for card lookup caching and interaction pipeline state.
|
|
- **REST API** — `/docs` (Swagger) available at runtime.
|
|
|
|
## Architecture
|
|
|
|
```
|
|
mtgonline/
|
|
├── backend/ # FastAPI application
|
|
│ ├── app/
|
|
│ │ ├── core/ # Settings, database engines, Redis client
|
|
│ │ ├── models/ # SQLAlchemy ORM models (app + MTG)
|
|
│ │ ├── routers/ # API route modules
|
|
│ │ ├── schemas/ # Pydantic request/response schemas
|
|
│ │ ├── services/ # Business logic (MTGJSON manager, card DB, game server)
|
|
│ │ └── main.py # FastAPI app entry point
|
|
│ ├── scripts/ # Utility scripts (downloads, migrations, checks)
|
|
│ ├── Dockerfile
|
|
│ └── requirements.txt
|
|
├── docker-compose.dev.yml # Development stack (Postgres x2, Redis, Backend)
|
|
├── docker-compose.yml # Production stack
|
|
├── scripts/ # Shared utility scripts
|
|
└── README.md
|
|
```
|
|
|
|
### Data Flow
|
|
|
|
1. **On startup**, the backend connects to PostgreSQL (both instances) and Redis.
|
|
2. It checks `mtg_refresh_log` in the `mtgdata` database for existing data.
|
|
3. If no data exists, it downloads MTGJSON files from `https://mtgjson.com/api/v5/`, unzips if needed, and upserts them into `mtgdata` tables (`mtg_sets`, `mtg_cards`, etc.).
|
|
4. The data is then available via REST endpoints.
|
|
|
|
## Quick Start
|
|
|
|
### Prerequisites
|
|
|
|
- Docker and Docker Compose
|
|
|
|
### Run the Stack
|
|
|
|
```bash
|
|
cd /home/wall-o/projects/mtgonline
|
|
|
|
# Start all services (Postgres x2, Redis, Backend)
|
|
docker compose -f docker-compose.dev.yml up -d
|
|
|
|
# View logs
|
|
docker compose -f docker-compose.dev.yml logs -f backend
|
|
```
|
|
|
|
The backend will automatically download MTGJSON data on first startup (this may take several minutes).
|
|
|
|
### Access
|
|
|
|
| Service | Address |
|
|
|---------------|---------------------|
|
|
| Backend API | `http://localhost:5555` |
|
|
| Swagger Docs | `http://localhost:5555/docs` |
|
|
| Health Check | `http://localhost:5555/health` |
|
|
| PostgreSQL (app) | `localhost:5432` |
|
|
| PostgreSQL (MTG) | `localhost:5433` |
|
|
| Redis | `localhost:6379` |
|
|
|
|
### Manual Data Refresh
|
|
|
|
```bash
|
|
# Trigger a manual MTGJSON refresh
|
|
curl -X POST http://localhost:5555/refresh
|
|
```
|
|
|
|
## Environment Variables
|
|
|
|
| Variable | Default | Description |
|
|
|----------|---------|-------------|
|
|
| `DATABASE_URL` | `postgresql+asyncpg://mtgonline_user:mtgonline_password@postgres:5432/mtgonline` | Primary database connection |
|
|
| `MTG_DATABASE_URL` | `postgresql+asyncpg://mtgonline_user:mtgonline_password@mtgdata:5432/mtgdata` | MTG data database connection |
|
|
| `REDIS_URL` | `redis://redis:6379` | Redis connection |
|
|
| `DATA_DIR` | `/app/data` | Directory for MTGJSON files |
|
|
| `DEBUG` | `False` | Enable debug logging |
|
|
|
|
## Docker Cleanup
|
|
|
|
```bash
|
|
# Stop and remove all containers
|
|
docker compose -f docker-compose.dev.yml down
|
|
|
|
# Remove images and prune
|
|
docker system prune -a --volumes
|
|
```
|
|
|
|
## License
|
|
|
|
MIT
|