Fix MTGJSON download: use .json files, remove gzip unpacking
- Changed REQUIRED_FILES to REQUIRED_JSON_FILES and REQUIRED_ZIP_FILES - AllSetFiles.zip kept as zip (only available compressed on MTGJSON) - All other files downloaded as .json (no compression) - Removed gzip import and unpacking logic for .gz files - Updated EXPECTED_MIN_SIZES to reflect actual .json file sizes - Removed validation for non-existent files
This commit is contained in:
+16
-32
@@ -1,43 +1,27 @@
|
||||
{
|
||||
"task_description": "MTG Online Backend - MTGJSON Data Integration & Docker Deployment",
|
||||
"current_step": "Testing full container lifecycle",
|
||||
"task_description": "Update MTGJSON data manager to download .json files instead of .json.gz files, remove unnecessary unpacking logic",
|
||||
"current_step": "Docker image rebuilt with updated code",
|
||||
"files_created": [
|
||||
"backend/app/services/mtgjson_manager.py",
|
||||
"backend/scripts/sanity_check_mtgjson.py",
|
||||
"backend/scripts/verify_mtgjson_data.py",
|
||||
"backend/scripts/maintain_mtgjson.sh"
|
||||
"backend/app/services/mtgjson_manager.py"
|
||||
],
|
||||
"files_modified": [
|
||||
"backend/app/services/mtgjson_manager.py",
|
||||
"backend/app/scripts/refresh_mtg.py",
|
||||
"backend/app/main.py",
|
||||
"docker-compose.yml",
|
||||
"docker-compose.dev.yml",
|
||||
"backend/Dockerfile"
|
||||
"backend/app/services/mtgjson_manager.py"
|
||||
],
|
||||
"decisions": [
|
||||
"Created comprehensive MTGJSONManager service in app/services/",
|
||||
"Manager handles download, unpack, upsert with ON CONFLICT DO UPDATE",
|
||||
"Startup triggers initial download on first container init",
|
||||
"Health check verifies MTG data exists in database",
|
||||
"Refresh interval set to 7 days (weekly)",
|
||||
"Docker compose start_period set to 600s for download time",
|
||||
"Added file size validation against expected minimums",
|
||||
"Implemented retry logic with exponential backoff (up to 3 attempts)",
|
||||
"Cleanup deletes corrupted data before retry",
|
||||
"Container marked unhealthy if validation fails after all retries",
|
||||
"Download timeout increased to 60 minutes (3600s) for large files"
|
||||
"Changed REQUIRED_FILES to REQUIRED_JSON_FILES and REQUIRED_ZIP_FILES",
|
||||
"AllSetFiles.zip kept as zip (only available compressed on MTGJSON API)",
|
||||
"All other files downloaded as .json (no compression)",
|
||||
"Removed gzip import and unpacking logic for .gz files",
|
||||
"Updated EXPECTED_MIN_SIZES to reflect actual .json file sizes (500MB+ for large files)",
|
||||
"Removed validation for non-existent files (MagicSets.json, MagicRoots.json, SetTranslations.json)"
|
||||
],
|
||||
"next_steps": [
|
||||
"Save state to state.json",
|
||||
"Commit and push to Gitea",
|
||||
"Stop and destroy all Docker containers",
|
||||
"Build backend Docker container",
|
||||
"Deploy stack and monitor",
|
||||
"Check container health status",
|
||||
"Verify logs show successful initialization"
|
||||
"Restart Docker container with new image",
|
||||
"Run data refresh to verify download and upsert work correctly",
|
||||
"Monitor logs for any issues",
|
||||
"Verify database has correct data"
|
||||
],
|
||||
"blockers": [],
|
||||
"commit_hash": "0643544",
|
||||
"timestamp": 1784598400
|
||||
"commit_hash": "",
|
||||
"timestamp": "2026-07-20T13:52:00Z"
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user