# Google Trends evidence bundle: AI-first D&D/TTRPG platform

This directory contains a reproducible capture from the official Google Trends
Explore web endpoints. It intentionally does not substitute SEO search-volume
estimates for Google Trends data.

## Re-run

From the project root on 2026-07-21:

```bash
rtk uv run --python 3.12 python research/data/google-trends/fetch_google_trends.py
```

The script creates an immutable timestamped folder under `runs/` and records
the latest run ID in `latest_run.txt`. Every run includes:

- `manifest.json`: retrieval timestamps (UTC), full request log/statuses,
  parameters, source URLs, SHA-256 hashes and failures;
- `raw/`: unmodified JSON payloads decoded from the official endpoints;
- `derived/`: tidy CSVs for interest over time, summary statistics, geography
  and related queries;
- `errors.jsonl`: exact endpoint errors, including rate limits.

## SerpApi fallback for arbitrary non-RU specs

When the official Explore web endpoints return HTTP 429, the stdlib-only
`fetch_serpapi_trends.py` collector can retrieve the same UI-normalized data
through SerpApi. It is a third-party intermediary, not the invite-only official
Google Trends API alpha.

Pass both paths explicitly so a global run cannot overwrite the RU bundle:

```bash
SERPAPI_API_KEY=... rtk uv run python \
  research/data/google-trends/fetch_serpapi_trends.py \
  --spec research/global-validation/data/google-trends/query_spec.json \
  --output-root research/global-validation/data/google-trends
```

The key is read only from the process environment and is removed from archived
parameters, provider payloads and errors. `latest_run.txt` changes only after
every required job reaches a terminal provider result. A confirmed provider
`no results` response is archived as `ok_no_data`, not silently converted into
a network failure or a demand conclusion. SerpApi does not expose Google's
original `hasData` field: explicit values, including `"0"`, are recorded as
available; empty values are marked `unknown`.

`period-comparison.csv` is a transparent secondary calculation over the latest
five-year CSV: the arithmetic mean of the first 52 and last 52 complete weekly
buckets, with zero/insufficient-data buckets retained exactly as Google
returned them. It is useful only for direction within the same request; sparse
series are flagged and not used as market-size evidence.

The query groups and explicit date ranges live in `query_spec.json`. Each group
has no more than five comparison items. The runs use Web Search, all categories,
`tz=0`, explicit periods ending on the last complete UTC date before capture,
and separate Russian Federation (`geo=RU`) from worldwide (`geo=`) requests.

## Interpretation constraints

Google Trends values are sampled and normalized, not absolute search volume.
Within one request, 100 is the peak relative interest among the included terms
and dates; 50 is half that relative level; 0 can mean insufficient data. Values
from separate groups or time windows are independently rescaled and must not be
compared as if they shared one denominator. The repeated
`Dungeons & Dragons` search term is a bridge anchor across several groups, but
it does not turn the index into a search-count estimate.

Search terms are broad-match strings in Trends. Topics use verified Google
Knowledge Graph IDs returned by the official autocomplete endpoint. Topics can
aggregate language variants, while search terms preserve the lexical question.
The worldwide Topic group intentionally mixes entity scopes (category, games,
website/software) and is only a directional benchmark, never market share.

`The Knave` is explicitly isolated in a disambiguation group: Google
autocomplete returns non-RPG entities for that wording. `Knave RPG` has a more
specific Topic suggestion. Related queries must be inspected before treating
raw `The Knave` interest as RPG demand.

## Official methodology references

- Google Trends data FAQ: https://support.google.com/trends/answer/4365533
- Google News Initiative, understanding Trends data:
  https://newsinitiative.withgoogle.com/resources/trainings/fundamentals/google-trends-understanding-the-data/
- Google Trends API alpha status:
  https://developers.google.com/search/apis/trends
- Official BigQuery Trends dataset scope (Top 25 / Rising only, therefore not a
  replacement for arbitrary D&D queries):
  https://support.google.com/trends/answer/12764470