RnD · Линия доказательств Data notes
Каталог и оглавление
DATA.IDX RnD/research/data/google-trends/README.md raw.md ->

Google Trends evidence bundle: AI-first D&D/TTRPG platform

This directory contains a reproducible capture from the official Google Trends Explore web endpoints. It intentionally does not substitute SEO search-volume estimates for Google Trends data.

Re-run#

From the project root on 2026-07-21:

rtk uv run --python 3.12 python research/data/google-trends/fetch_google_trends.py

The script creates an immutable timestamped folder under runs/ and records the latest run ID in latest_run.txt. Every run includes:

  • manifest.json: retrieval timestamps (UTC), full request log/statuses, parameters, source URLs, SHA-256 hashes and failures;
  • raw/: unmodified JSON payloads decoded from the official endpoints;
  • derived/: tidy CSVs for interest over time, summary statistics, geography and related queries;
  • errors.jsonl: exact endpoint errors, including rate limits.

SerpApi fallback for arbitrary non-RU specs#

When the official Explore web endpoints return HTTP 429, the stdlib-only fetch_serpapi_trends.py collector can retrieve the same UI-normalized data through SerpApi. It is a third-party intermediary, not the invite-only official Google Trends API alpha.

Pass both paths explicitly so a global run cannot overwrite the RU bundle:

SERPAPI_API_KEY=... rtk uv run python \
  research/data/google-trends/fetch_serpapi_trends.py \
  --spec research/global-validation/data/google-trends/query_spec.json \
  --output-root research/global-validation/data/google-trends

The key is read only from the process environment and is removed from archived parameters, provider payloads and errors. latest_run.txt changes only after every required job reaches a terminal provider result. A confirmed provider no results response is archived as ok_no_data, not silently converted into a network failure or a demand conclusion. SerpApi does not expose Google's original hasData field: explicit values, including "0", are recorded as available; empty values are marked unknown.

period-comparison.csv is a transparent secondary calculation over the latest five-year CSV: the arithmetic mean of the first 52 and last 52 complete weekly buckets, with zero/insufficient-data buckets retained exactly as Google returned them. It is useful only for direction within the same request; sparse series are flagged and not used as market-size evidence.

The query groups and explicit date ranges live in query_spec.json. Each group has no more than five comparison items. The runs use Web Search, all categories, tz=0, explicit periods ending on the last complete UTC date before capture, and separate Russian Federation (geo=RU) from worldwide (geo=) requests.

Interpretation constraints#

Google Trends values are sampled and normalized, not absolute search volume. Within one request, 100 is the peak relative interest among the included terms and dates; 50 is half that relative level; 0 can mean insufficient data. Values from separate groups or time windows are independently rescaled and must not be compared as if they shared one denominator. The repeated Dungeons & Dragons search term is a bridge anchor across several groups, but it does not turn the index into a search-count estimate.

Search terms are broad-match strings in Trends. Topics use verified Google Knowledge Graph IDs returned by the official autocomplete endpoint. Topics can aggregate language variants, while search terms preserve the lexical question. The worldwide Topic group intentionally mixes entity scopes (category, games, website/software) and is only a directional benchmark, never market share.

The Knave is explicitly isolated in a disambiguation group: Google autocomplete returns non-RPG entities for that wording. Knave RPG has a more specific Topic suggestion. Related queries must be inspected before treating raw The Knave interest as RPG demand.

Official methodology references#