akadmin
915242b330
- Added MTG card ORM models (mtg_cards, mtg_sets tables) - Created card_database service with search, get_by_name, get_by_set - Added Redis client with caching layer (3600s TTL default) - Created card router with caching on all endpoints: - Search cards (5min cache) - Get card by name (10min cache) - Get cards by set (15min cache) - Get card types/rarities (30min cache) - Get sets (1hr cache) - Get statistics (1hr cache) - Updated settings.py: - Added JWT_SECRET_KEY field - Added DB_CONFIG and REDIS_CONFIG dictionaries - Updated security.py to use JWT_SECRET_KEY with fallback - Updated auth.py to use timezone-aware datetimes - Updated refresh_mtg.py to use settings instead of os.environ - Updated mtg_monitor.py to use settings for connections - Added services package with __init__.py All 20 tests passing.
3.6 KiB
3.6 KiB
MTG Online Backend - Docker Migration Plan
Overview
Migrate from internal database to Dockerized PostgreSQL with mtgjson.com "All Printings" dataset.
Architecture
┌─────────────────────────────────────────────────────────────┐
│ Docker Compose │
├─────────────────────────────────────────────────────────────┤
│ ┌─────────────────────┐ ┌──────────────────────────┐ │
│ │ Backend Container │───▶│ PostgreSQL Container │ │
│ │ (FastAPI app) │ │ (MTG Data + App DB) │ │
│ │ │ │ │ │
│ │ - API endpoints │ │ - cockatrice_db (app) │ │
│ │ - Auth system │ │ - mtgdata_db (mtgjson) │ │
│ │ - Weekly refresh │ │ │ │
│ └─────────────────────┘ └──────────────────────────┘ │
└─────────────────────────────────────────────────────────────┘
Tasks
1. mtgjson Dataset Analysis
- Download and examine All Printings dataset
- Map mtgjson schema to SQLAlchemy models
- Identify card image files (PNG/JPG)
- Plan database schema for card data
2. Docker Infrastructure
- Create Dockerfile for backend
- Create docker-compose.yml with PostgreSQL
- Configure environment variables
- Set up volume mounts for data persistence
- Configure health checks
3. Database Migration
- Create migration script for mtgjson schema
- Update models.py with MTG card models
- Add multi-database support (cockatrice + mtgdata)
- Create schema for weekly refresh
4. Weekly Refresh Logic
- Create refresh script to download latest mtgjson
- Implement incremental update logic
- Add database cleanup for unused cards
- Schedule via cron in Docker container
5. Card Image Support
- Verify image files in dataset (setJSON)
- Add image storage/CDN support
- Create API endpoints for card images
Files to Create/Modify
Docker Files
backend/Dockerfiledocker-compose.ymlbackend/.dockerignore
Database Migration
backend/app/core/database_multi.py(multi-database support)backend/scripts/refresh_mtg.py(weekly refresh)backend/scripts/init_mtg_db.py(initial mtgjson import)
Configuration
backend/.env.example(updated with Docker settings)backend/app/core/settings.py(add MTG settings)
Models
backend/app/models/mtg_models.py(mtgjson card models)backend/app/models/models.py(update for multi-DB)
Commands
Build and Run
cd /home/wall-o/projects/mtgonline
docker-compose up -d
docker-compose logs -f backend
Database Refresh
docker-compose exec backend python -m app.scripts.refresh_mtg
Check Status
docker-compose ps
docker-compose logs backend
Status: IN PROGRESS
- Current step: Creating Docker infrastructure
- Next: Download mtgjson dataset and analyze schema