Files

T

akadmin a01e33eb5e Clean up backend folder - remove obsolete scripts and test files

Removed obsolete scripts that were not imported or used:
- card_interaction_rule_engine.py
- card_profile_extractor.py
- create_card_interaction_graph.py
- interaction_determinator.py
- interaction_pipeline.py
- interaction_recommender.py
- interaction_schema.py
- recommendation_engine.py
- migrate_complete.py
- migrate_schema.py
- test_interaction_determinator.py
- check_mtgjson_full.py
- check_mtgjson_status.py
- verify_integration.py
- verify_mtgjson_data.py
- sanity_check_mtgjson.py
- investigate_sets.py
- inspect_db.py
- code_review.md
- monitor/mtg_monitor.py

Removed test artifacts:
- test.db
- test_download.py
- test_system.py
- setup_db.py
- BACKEND_TESTING_SUMMARY.md
- CHAT_PROMPT_TEST.md
- CONTINUATION_PROMPT.md
- PORTED_STATE.md
- SPEC_synergy-mapping-engine.md
- STATE.md
- SUPPORTED_FILE_TYPES.md

Removed sensitive/environment files:
- .env.local
- state.json (backend)

Cleaned up:
- __pycache__ directories
- venv directory

Backend scripts/ directory now contains only essential data loading and maintenance scripts.

2026-07-22 03:00:04 +00:00

3.8 KiB

Raw Blame History

MTG Online Backend

A Python FastAPI application that processes Magic: The Gathering card data from MTGJSON and stores it in PostgreSQL, with Redis for caching.

Overview

This project provides a backend API for a Magic: The Gathering Online platform. It downloads and processes MTGJSON v5 dataset dumps, loads them into a PostgreSQL database, and exposes REST endpoints for card data, user authentication, deck management, and game state.

Key Features

MTGJSON Data Pipeline — Downloads AllPrintings.psql, AllIdentifiers.json, Keywords.json, CardTypes.json, and AllDeckFiles.zip from MTGJSON v5 on startup or via a POST /refresh endpoint.
Dual PostgreSQL — Two databases: mtgonline for the application (users, decks, auth) and mtgdata for MTG card data.
Redis Caching — Used for card lookup caching and interaction pipeline state.
REST API — /docs (Swagger) available at runtime.

Architecture

mtgonline/
├── backend/                  # FastAPI application
│   ├── app/
│   │   ├── core/             # Settings, database engines, Redis client
│   │   ├── models/           # SQLAlchemy ORM models (app + MTG)
│   │   ├── routers/          # API route modules
│   │   ├── schemas/          # Pydantic request/response schemas
│   │   ├── services/         # Business logic (MTGJSON manager, card DB, game server)
│   │   └── main.py           # FastAPI app entry point
│   ├── scripts/              # Utility scripts (downloads, migrations, checks)
│   ├── Dockerfile
│   └── requirements.txt
├── docker-compose.dev.yml    # Development stack (Postgres x2, Redis, Backend)
├── docker-compose.yml        # Production stack
├── scripts/                  # Shared utility scripts
└── README.md

Data Flow

On startup, the backend connects to PostgreSQL (both instances) and Redis.
It checks mtg_refresh_log in the mtgdata database for existing data.
If no data exists, it downloads MTGJSON files from https://mtgjson.com/api/v5/, unzips if needed, and upserts them into mtgdata tables (mtg_sets, mtg_cards, etc.).
The data is then available via REST endpoints.

Quick Start

Prerequisites

Docker and Docker Compose

Run the Stack

cd /home/wall-o/projects/mtgonline

# Start all services (Postgres x2, Redis, Backend)
docker compose -f docker-compose.dev.yml up -d

# View logs
docker compose -f docker-compose.dev.yml logs -f backend

The backend will automatically download MTGJSON data on first startup (this may take several minutes).

Access

Service	Address
Backend API	`http://localhost:5555`
Swagger Docs	`http://localhost:5555/docs`
Health Check	`http://localhost:5555/health`
PostgreSQL (app)	`localhost:5432`
PostgreSQL (MTG)	`localhost:5433`
Redis	`localhost:6379`

Manual Data Refresh

# Trigger a manual MTGJSON refresh
curl -X POST http://localhost:5555/refresh

Environment Variables

Variable	Default	Description
`DATABASE_URL`	`postgresql+asyncpg://mtgonline_user:mtgonline_password@postgres:5432/mtgonline`	Primary database connection
`MTG_DATABASE_URL`	`postgresql+asyncpg://mtgonline_user:mtgonline_password@mtgdata:5432/mtgdata`	MTG data database connection
`REDIS_URL`	`redis://redis:6379`	Redis connection
`DATA_DIR`	`/app/data`	Directory for MTGJSON files
`DEBUG`	`False`	Enable debug logging

Docker Cleanup

# Stop and remove all containers
docker compose -f docker-compose.dev.yml down

# Remove images and prune
docker system prune -a --volumes

License

MIT

3.8 KiB Raw Blame History