mtgonline/backend/SPEC_synergy-mapping-engine.md

Technical Specification: MTG Synergy Mapping Engine

1. Project Overview

The goal is to create a Python-based data pipeline that processes MTG card data from MTGJSON, identifies synergistic relationships between cards, and stores these relationships in a PostgreSQL database. This "Data Map" will power a deck-building assistant that suggests cards based on mechanical and strategic complementarity.

2. Tech Stack

• Language: Python 3.12+
• Libraries: pandas (data manipulation), SQLAlchemy (ORM), psycopg2 (DB driver), re (regex for text processing).
• Database: PostgreSQL.
• Data Source: MTGJSON (AllPrintings.json, AllSets.json).

3. Phase 1: Data Ingestion & Normalization

The script must flatten the nested MTGJSON structure into a relational format.

3.1 Extraction

Extract the following fields from AllPrintings.json:

• name, manaCost, types, text (oracle text), colorIdentity, set.

3.2 Text Processing (TextProcessor Class)

Implement a class to convert raw oracle text into "Functional Tokens."

• Regex Mapping: Use a dictionary of regex patterns to identify key actions.
  • Example: "draw a card" \rightarrow TOKEN_DRAW_1
  • Example: "destroy all creatures" \rightarrow TOKEN_BOARD_WIPE_CREATURE
• Tagging: Extract subtypes (Tribes) from the types field (e.g., "Elf", "Zombie").

4. Phase 2: The Synergy Engine (Logic)

The engine must evaluate every card pair and assign a weighted connection based on three tiers of synergy.

Tier A: Hard Synergies (Weight: 1.0)

Logic: Direct mechanical triggers.

• Tribal Link: If Card_A.tags (Tribe) \cap Card_B.text (contains Tribe name) \neq \emptyset.
• Trigger-Response: Identify "Providers" (e.g., "Whenever you gain life") and "Payoffs" (e.g., "When you gain life, [Effect]"). Link Provider \rightarrow Payoff.

Tier B: Functional Similarity (Weight: 0.6)

Logic: Substitution/Role mapping.

• Role Dictionary: Define roles (e.g., RAMP, CARD_DRAW, REMOVAL).
• Mapping: If both cards share the same Role_ID based on their Functional Tokens, create a link.

Tier C: Strategic Archetypes (Weight: 0.3)

Logic: Thematic co-occurrence.

• Archetype Buckets: Define keyword groups (e.g., GRAVEYARD_STRAT = ["mill", "graveyard", "reanimate"]).
• Density Check: If both cards have a high overlap of keywords from the same bucket, create a link.

5. Phase 3: Database Schema (PSQL)

Implement the following schema:

Table: cards

• card_id: UUID (Primary Key)
• name: VARCHAR
• oracle_text: TEXT
• mana_cost: VARCHAR
• color_identity: ARRAY[VARCHAR]
• tags: ARRAY[VARCHAR] (Stored functional tokens and tribes)

Table: synergy_types

• type_id: INT (Primary Key)
• label: VARCHAR (e.g., 'Tribal', 'Mechanical', 'Substitute')

Table: card_connections

• card_id_a: UUID (FK \rightarrow cards)
• card_id_b: UUID (FK \rightarrow cards)
• type_id: INT (FK \rightarrow synergy_types)
• weight: FLOAT
• Constraint: CHECK (card_id_a < card_id_b) to prevent bidirectional duplicates.

6. Phase 4: Execution Pipeline

The script must execute in the following order:

1. Ingest: Parse JSON \rightarrow Bulk load into cards table.
2. Analyze: Run TextProcessor \rightarrow Update cards.tags.
3. Map:
   • Iterate through card pairs.
   • Evaluate Tiers A, B, and C.
   • Insert identified synergies into card_connections.
4. Index: Create B-Tree indices on card_id_a and card_id_b.

7. Phase 5: Recommendation Logic (API Level)

The resulting database must support the following query logic for the API:

1. Input: A list of card_ids currently in a deck.
2. Query: Find all card_id_b linked to any of the input IDs in card_connections.
3. Aggregate: Sum the weight for each suggested card.
4. Filter: Remove suggestions that do not match the color_identity of the deck.
5. Output: Return the top N cards sorted by aggregate weight.