zxdb/CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Overview

This is a Python script that downloads ZX Spectrum game files from the ZXDB open database. It queries a local MySQL instance containing ZXDB data, then downloads and organizes game files (tapes, disks, snapshots, pokes) into a configurable folder structure with caching support.

Setup

Virtual environment:

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

MySQL database: Runs in a Docker container. Connect with:

mysql -h 172.18.0.2 -P 3306 -u root -p
# Password: unJEPimbJddHP8
# Then: use zxdb
# To import: source /path/to/ZXDB.sql

Configuration: Edit config.py directly (tracked in git; credentials are intentionally non-sensitive). Key settings:

• DB_* — MySQL connection params
• DESTINATION_PATH / CACHE_PATH / TEMP_FILE — local paths
• SHOULD_CLEAR_DESTINATION_PATH — wipes destination before each run
• SHOULD_SPLIT_MODERN_AND_CLASSIC / SHOULD_SORT_BY_YEAR / SHOULD_SORT_BY_LETTER / SHOULD_SORT_BY_DEVELOPER — folder organization modes
• WAIT / MIN_WAIT / MAX_WAIT — rate limiting between downloads
• LAST_CLASSIC_YEAR — year cutoff for classic/modern split
• ZXDB_SQL_PATH — where to place the extracted ZXDB_mysql.sql (used by zxdb/setup/database_import.py)
• ZXDB_STATE_FILE — JSON file tracking the last successful import (github_commit_date + last_import)

Running

python main.py

Architecture

The code is organized as a Python package (zxdb/) orchestrated by main.py. There is no global state: data flows as a list of dictionaries between functions.

Entry point (main.py):

elements = database.connect(query_index=3)
elements = organizer.process_elements(elements)
filesystem.print_elements(elements, mode=1)
filesystem.clear_destination_folder()
downloader.get_files(elements)
filesystem.remove_empty_directories(config.DESTINATION_PATH)

Module structure:

• zxdb/database.py — load_queries(), select(), connect(): connects to MySQL and returns elements list
• zxdb/organizer.py — normalize_path(), update_url(), url_filename(), process_elements(), get_final_destination_folder()
• zxdb/downloader.py — download_file(), unzip_file(), process_cache_file(), get_files(elements)
• zxdb/filesystem.py — clear_destination_folder(), remove_empty_directories(), print_elements(elements, mode), print_status()
• zxdb/queries.sql — Four SQL queries (index 0–3), loaded via Path(__file__).parent / "queries.sql"
• zxdb/setup/database_import.py — download_zxdb(destination): streaming download of ZXDB_mysql.sql.zip from GitHub + ZIP extraction; runnable via python -m zxdb.setup.database_import
• zxdb/setup/docker_manager.py — ensure_container(), wait_for_mysql(container, timeout), db_exists(container), import_sql(container, sql_path): Docker container management and SQL import
• zxdb/setup/__main__.py — Full setup orchestrator: Docker → MySQL ready → GitHub change detection → download → import → save state. Run via python -m zxdb.setup

SQL queries (zxdb/queries.sql): Four queries (index 0–3). Query 3 (current default) filters for ZX-Spectrum machines only (m.text like 'ZX-%') and returns only tape/disk/snapshot/poke file types.

Folder organization (get_final_destination_folder()): Builds a path from up to four nested levels based on the sort flags in config.py:

• folder1: classics/modern, by_developer, by_year, or empty
• folder2: developer letter + developer name (e.g., D/Dinamic) or empty
• folder3: release year or unknown
• folder4: first letter of game title (e.g., A, 0-9) or empty

URL resolution (update_url()): URLs from the DB are relative paths. They get prefixed based on their start: /zxdb → spectrumcomputing.co.uk, /pub → WOS mirror, /nvg → NVG mirror.

Cache: Files are stored in CACHE_PATH/<game_folder_name>/<subfolder>/. The cache key is the original game_folder_name (title + year + developer), independent of the current sort configuration, so changing sort options reuses cached files.

File types stored at game root (no subfolder): Tape image, Disk image, Snapshot image, POK pokes file. All other file types go into a named subfolder.

ZIP extraction (unzip_file()): Only extracts files matching known emulator formats: .tap, .tzx, .stl, .sna, .z80, .dsk, .fdi, .trd, .img, .mgt, .szx, .dck.