Commit Graph
26 Commits
Author SHA1 Message Date
akadmin 46abfe5fbc Add comprehensive documentation for MTG Online Backend
- Updated root README with current architecture (dual PostgreSQL, Redis, MTGJSON pipeline)
- Created backend/README.md with detailed architecture, database setup, and troubleshooting
- Updated state.json to reflect completed documentation phase
2026-07-21 03:50:44 +00:00
akadmin 6f01e2d1b4 Add docker-compose.yml for multi-container deployment 2026-07-21 03:00:09 +00:00
akadmin 90822d48c4 Fix MTGJSON download functionality
- Add refresh router with admin-only endpoints
- Fix _decompress_file to handle .psql files correctly
- Fix duplicate imports in mtgjson_manager.py
- Mount refresh router in main.py
- Add download_mtgjson.py standalone script
- Add SUPPORTED_FILE_TYPES.md documentation
2026-07-21 02:58:37 +00:00
akadmin 0eacaba28b docs: update README with volume mount instructions and Docker setup 2026-07-20 22:56:14 +00:00
akadmin ea3c1d9691 Refactor MTGJSON manager to use local files only 2026-07-20 22:20:26 +00:00
akadmin 6a9d90c3b5 Update state.json for full deployment review 2026-07-20 21:48:44 +00:00
akadmin 3fae593a22 Fix MTGJSON download: use .json files, remove gzip unpacking
- Changed REQUIRED_FILES to REQUIRED_JSON_FILES and REQUIRED_ZIP_FILES
- AllSetFiles.zip kept as zip (only available compressed on MTGJSON)
- All other files downloaded as .json (no compression)
- Removed gzip import and unpacking logic for .gz files
- Updated EXPECTED_MIN_SIZES to reflect actual .json file sizes
- Removed validation for non-existent files
2026-07-20 13:59:45 +00:00
akadmin 2369c2cc8c Add comprehensive handoff documentation for new thread 2026-07-20 13:23:40 +00:00
akadmin ad2742c1cf Fix MTGJSON download: use .gz URLs and handle pre-uncompressed files 2026-07-20 04:46:15 +00:00
akadmin 65a1f95b7f Save state and update project documentation 2026-07-20 04:39:46 +00:00
akadmin 20fb57258b Increase download timeout to 60 minutes for large MTGJSON files 2026-07-20 04:38:53 +00:00
akadmin d347e0e586 Double all timeout values to prevent download/upsert timeouts 2026-07-20 04:37:31 +00:00
akadmin 6b3fa63501 Fix: Use download_with_sanity_check() and fail container on download failure 2026-07-20 04:06:43 +00:00
akadmin 0643544327 Update state.json for sanity check implementation 2026-07-20 03:56:32 +00:00
akadmin dedc9a35b3 Add MTGJSON data sanity check with file size validation and retry logic 2026-07-20 03:54:34 +00:00
akadmin f722d7ab62 chore: update state.json with deployment information 2026-07-20 03:20:24 +00:00
akadmin 99c7d08bb1 feat: MTGJSON data manager service with download, unpack, and upsert
- Created MTGJSONManager service for complete data lifecycle
- Handles download, unpack (gzip/zip), and PostgreSQL upsert
- ON CONFLICT DO UPDATE preserves existing data
- Startup triggers initial download on first container init
- Health check verifies MTG data exists in database
- Weekly refresh via MTG_REFRESH_INTERVAL_DAYS setting
- Updated docker-compose start_period to 600s for download time
2026-07-20 03:17:23 +00:00
akadmin bb231a5f5d feat: Add MTGJSON data loading and download scripts
- Fix MtgSet model to match database schema (removed created_at, added image column)
- Create load_mtgjson_data.py script to load AllSetFiles, AllPrintings.psql, and other MTGJSON data
- Create download_mtgjson_data.py script to download MTGJSON API data files
- Add SPEC_synergy-mapping-engine.md documentation

API endpoints are now working (200 OK) but database needs data loading via download_mtgjson_data.py
then load_mtgjson_data.py
2026-07-20 02:08:30 +00:00
akadmin baf13ce294 feat: add MTGJSON to PostgreSQL data loader with upsert logic
- Converts MTGJSON v5 AllPrintings.json to PostgreSQL format
- Upserts data to mtgdata database using psycopg2
- Handles sets and cards with proper foreign key relationships
- Batch processing of 100 cards at a time
- Proper transaction management with commit/rollback
- Verified: 14,866 sets and 14,826 cards loaded successfully
2026-07-19 15:14:54 +00:00
akadmin 02ef8e36bc Complete backend verification: fix syntax errors, create __init__.py files, add setup_db.py, update docker-compose for two PostgreSQL containers 2026-07-18 23:02:44 +00:00
akadmin 3a70feaba5 Add system test script and scripts directory
- Added test_system.py for comprehensive backend testing
- Added scripts/ directory with .gitkeep placeholder
- Updates state.json for latest progress tracking
2026-07-18 19:13:30 +00:00
akadmin 963f130e5f Fix route registration: Add card_router to main.py 2026-07-18 18:49:44 +00:00
akadmin 915242b330 feat: MTG database integration with Redis caching
- Added MTG card ORM models (mtg_cards, mtg_sets tables)
- Created card_database service with search, get_by_name, get_by_set
- Added Redis client with caching layer (3600s TTL default)
- Created card router with caching on all endpoints:
  - Search cards (5min cache)
  - Get card by name (10min cache)
  - Get cards by set (15min cache)
  - Get card types/rarities (30min cache)
  - Get sets (1hr cache)
  - Get statistics (1hr cache)
- Updated settings.py:
  - Added JWT_SECRET_KEY field
  - Added DB_CONFIG and REDIS_CONFIG dictionaries
- Updated security.py to use JWT_SECRET_KEY with fallback
- Updated auth.py to use timezone-aware datetimes
- Updated refresh_mtg.py to use settings instead of os.environ
- Updated mtg_monitor.py to use settings for connections
- Added services package with __init__.py

All 20 tests passing.
2026-07-18 18:41:05 +00:00
akadmin 167a352d44 Fix backend test suite - all 20 tests passing
- Updated JWT tokens to include privlevel for authorization
- Fixed test fixtures to properly hash passwords with bcrypt
- Fixed client fixture to share database session with test fixtures
- Reordered deck router routes to prevent conflicts
- Removed relationship fields from FolderResponse schema
- Updated conftest.py with proper async session management
2026-07-18 16:58:39 +00:00
akadmin 312ac27f88 Fix backend bugs: password hash column rename, dynamic import, AuditLog response_model, websocket reference, missing exports, bcrypt version pin 2026-07-18 05:14:37 +00:00
akadmin 86c12376f8 Initial commit of mtgonline project 2026-07-18 04:57:40 +00:00