Commit Graph
7 Commits
Author SHA1 Message Date
akadmin 90822d48c4 Fix MTGJSON download functionality
- Add refresh router with admin-only endpoints
- Fix _decompress_file to handle .psql files correctly
- Fix duplicate imports in mtgjson_manager.py
- Mount refresh router in main.py
- Add download_mtgjson.py standalone script
- Add SUPPORTED_FILE_TYPES.md documentation
2026-07-21 02:58:37 +00:00
akadmin d347e0e586 Double all timeout values to prevent download/upsert timeouts 2026-07-20 04:37:31 +00:00
akadmin bb231a5f5d feat: Add MTGJSON data loading and download scripts
- Fix MtgSet model to match database schema (removed created_at, added image column)
- Create load_mtgjson_data.py script to load AllSetFiles, AllPrintings.psql, and other MTGJSON data
- Create download_mtgjson_data.py script to download MTGJSON API data files
- Add SPEC_synergy-mapping-engine.md documentation

API endpoints are now working (200 OK) but database needs data loading via download_mtgjson_data.py
then load_mtgjson_data.py
2026-07-20 02:08:30 +00:00
wall-o db01e29a54 Final project commit: MTGJSON data integration and backend API 2026-07-20 01:09:45 +00:00
wall-o b170dfd577 feat: Add MTGJSON data integration
- Added MTGJSON data downloader (downloads all MTGJSON API v5 files)
- Added MTGJSON data loader (imports data into PostgreSQL)
- Added MTGJSON data uploader (alternative upsert logic)
- Fixed route ordering in card_router.py (/sets before /{card_name})
- Added load_mtgjson_data.py entry point script

MTGJSON data sources:
- AllPrintings.psql.gz (main cards database)
- AllSetFiles.zip (set and card data)
- AllDeckFiles.zip (deck data)
- AllIdentifiers.json.gz (card identifiers)
- CardTypes.json.gz (card types)
- DeckList.json.gz (deck list metadata)
- Keywords.json.gz (card keywords)
- SetList.json.gz (set list metadata)

Note: MTGJSON set.json does NOT contain image URLs. Only cards have image_uris.
Sets have iconSvgUrl (SVG icons) but no raster image URLs.
2026-07-20 00:15:25 +00:00
akadmin baf13ce294 feat: add MTGJSON to PostgreSQL data loader with upsert logic
- Converts MTGJSON v5 AllPrintings.json to PostgreSQL format
- Upserts data to mtgdata database using psycopg2
- Handles sets and cards with proper foreign key relationships
- Batch processing of 100 cards at a time
- Proper transaction management with commit/rollback
- Verified: 14,866 sets and 14,826 cards loaded successfully
2026-07-19 15:14:54 +00:00
akadmin 3a70feaba5 Add system test script and scripts directory
- Added test_system.py for comprehensive backend testing
- Added scripts/ directory with .gitkeep placeholder
- Updates state.json for latest progress tracking
2026-07-18 19:13:30 +00:00