feat: MTG database integration with Redis caching
- Added MTG card ORM models (mtg_cards, mtg_sets tables) - Created card_database service with search, get_by_name, get_by_set - Added Redis client with caching layer (3600s TTL default) - Created card router with caching on all endpoints: - Search cards (5min cache) - Get card by name (10min cache) - Get cards by set (15min cache) - Get card types/rarities (30min cache) - Get sets (1hr cache) - Get statistics (1hr cache) - Updated settings.py: - Added JWT_SECRET_KEY field - Added DB_CONFIG and REDIS_CONFIG dictionaries - Updated security.py to use JWT_SECRET_KEY with fallback - Updated auth.py to use timezone-aware datetimes - Updated refresh_mtg.py to use settings instead of os.environ - Updated mtg_monitor.py to use settings for connections - Added services package with __init__.py All 20 tests passing.
This commit is contained in:
@@ -0,0 +1,96 @@
|
||||
# MTG Online Backend - Docker Migration Plan
|
||||
|
||||
## Overview
|
||||
Migrate from internal database to Dockerized PostgreSQL with mtgjson.com "All Printings" dataset.
|
||||
|
||||
## Architecture
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Docker Compose │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ ┌─────────────────────┐ ┌──────────────────────────┐ │
|
||||
│ │ Backend Container │───▶│ PostgreSQL Container │ │
|
||||
│ │ (FastAPI app) │ │ (MTG Data + App DB) │ │
|
||||
│ │ │ │ │ │
|
||||
│ │ - API endpoints │ │ - cockatrice_db (app) │ │
|
||||
│ │ - Auth system │ │ - mtgdata_db (mtgjson) │ │
|
||||
│ │ - Weekly refresh │ │ │ │
|
||||
│ └─────────────────────┘ └──────────────────────────┘ │
|
||||
└─────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## Tasks
|
||||
|
||||
### 1. mtgjson Dataset Analysis
|
||||
- [x] Download and examine All Printings dataset
|
||||
- [ ] Map mtgjson schema to SQLAlchemy models
|
||||
- [ ] Identify card image files (PNG/JPG)
|
||||
- [ ] Plan database schema for card data
|
||||
|
||||
### 2. Docker Infrastructure
|
||||
- [ ] Create Dockerfile for backend
|
||||
- [ ] Create docker-compose.yml with PostgreSQL
|
||||
- [ ] Configure environment variables
|
||||
- [ ] Set up volume mounts for data persistence
|
||||
- [ ] Configure health checks
|
||||
|
||||
### 3. Database Migration
|
||||
- [ ] Create migration script for mtgjson schema
|
||||
- [ ] Update models.py with MTG card models
|
||||
- [ ] Add multi-database support (cockatrice + mtgdata)
|
||||
- [ ] Create schema for weekly refresh
|
||||
|
||||
### 4. Weekly Refresh Logic
|
||||
- [ ] Create refresh script to download latest mtgjson
|
||||
- [ ] Implement incremental update logic
|
||||
- [ ] Add database cleanup for unused cards
|
||||
- [ ] Schedule via cron in Docker container
|
||||
|
||||
### 5. Card Image Support
|
||||
- [ ] Verify image files in dataset (setJSON)
|
||||
- [ ] Add image storage/CDN support
|
||||
- [ ] Create API endpoints for card images
|
||||
|
||||
## Files to Create/Modify
|
||||
|
||||
### Docker Files
|
||||
- `backend/Dockerfile`
|
||||
- `docker-compose.yml`
|
||||
- `backend/.dockerignore`
|
||||
|
||||
### Database Migration
|
||||
- `backend/app/core/database_multi.py` (multi-database support)
|
||||
- `backend/scripts/refresh_mtg.py` (weekly refresh)
|
||||
- `backend/scripts/init_mtg_db.py` (initial mtgjson import)
|
||||
|
||||
### Configuration
|
||||
- `backend/.env.example` (updated with Docker settings)
|
||||
- `backend/app/core/settings.py` (add MTG settings)
|
||||
|
||||
### Models
|
||||
- `backend/app/models/mtg_models.py` (mtgjson card models)
|
||||
- `backend/app/models/models.py` (update for multi-DB)
|
||||
|
||||
## Commands
|
||||
|
||||
### Build and Run
|
||||
```bash
|
||||
cd /home/wall-o/projects/mtgonline
|
||||
docker-compose up -d
|
||||
docker-compose logs -f backend
|
||||
```
|
||||
|
||||
### Database Refresh
|
||||
```bash
|
||||
docker-compose exec backend python -m app.scripts.refresh_mtg
|
||||
```
|
||||
|
||||
### Check Status
|
||||
```bash
|
||||
docker-compose ps
|
||||
docker-compose logs backend
|
||||
```
|
||||
|
||||
## Status: IN PROGRESS
|
||||
- Current step: Creating Docker infrastructure
|
||||
- Next: Download mtgjson dataset and analyze schema
|
||||
Reference in New Issue
Block a user