# Brand Demand Index — Methodology

Method version: 1.0.0

## What this index is

The Brand Demand Index measures daily public **attention** to consumer brands,
using Wikipedia pageview counts for brand articles mapped to public-company
tickers and aligned to fiscal quarters. It measures attention, not demand and
not purchases. It is not a forecast, not a recommendation, and not investment
advice.

Attention of this kind is expected to be informative for researched purchases —
categories such as cars, phones, video games, footwear and films, where buyers
read before they buy — and to work poorly for low-involvement goods that people
buy without researching.

Past relationships between attention and subsequently reported revenue do not
guarantee future results.

## Sources

- Wikimedia pageview analytics (per-article daily counts and project
  aggregates). Wikimedia's analytics datasets are released under the CC0
  dedication; we store counts and identifiers only, never article text.
- Wikidata structured data (CC0) for entity identity, ownership and exclusion
  checks.
- SEC EDGAR (US government work) for tickers, fiscal calendars, filing times
  and reported revenue.

## Collection lag (D-2)

Wikimedia publishes a day's pageview data hours to more than a day after the
day ends. The daily run at 09:00 UTC therefore collects valid_date = D-2, the
most recent day reliably present. Each observation records both the day it
measures and the moment we fetched it.

## The two timestamps, and asof

Every row is bitemporal and carries two timestamps:

- **valid_date** — the day the views occurred (valid time);
- **fetched_at / recorded_at** — when we actually obtained or recorded the data
  (transaction time).

Nothing is ever overwritten; corrections are appended events with a later
transaction time. Every query takes an `asof` parameter (default: now) that
governs every input — the entity map, the fiscal calendar, the control basket,
the method version, and the observation rows — so a query for what we knew on a
given date returns exactly what we had published by that date.

## Backfill

Rows for dates before go-live (the valid_date of the genesis manifest) are
backfilled: they were fetched after the fact and could not have been known on
their valid_date. They are flagged `fetch_kind: backfill`, and their asof is
their actual fetched_at. Backfill exists so the first year of the product has a
prior-year denominator.

## Normalization basket

Daily counts are normalized against a fixed control basket to remove
Wikipedia-wide traffic swings:

norm(p, d) = views_user(p, d) / median over basket pages b of
( views_user(b, d) / mean of views_user(b) over 2015-07 through 2016-06 )

A median of ratios is used so that a spike on any single basket page barely
moves the normalizer. Basket selection rule: at least 30 English Wikipedia
pages whose Wikidata item has P31 (instance of) in a fixed class list (such as
chemical element, mathematical concept, historical era), that are not an
instance of human (Q5), not a company and not a product, whose page was created
before 2015, and that have not been moved since 2015. Basket pages are never
removed; a new basket is a new method version published as a separate shadow
series.

## Redirects, renames and merges

Per-article pageview data is keyed by title, not page ID, so views stay
attached to whichever title was requested. On every fetch we resolve the page
ID to its current title and all redirect titles, fetch each title, and sum all
titles' views per page ID when rolling up. A page rename is recorded as an
event and both the old and new titles continue to be collected. When two page
IDs are discovered to cover the same brand, a page_merged event (with its
effective date) is appended: rollups queried with an asof before the event
return the old numbers, and rollups queried after return the merged numbers,
summing both page IDs for the valid_dates on which both existed, with per-page
contributions preserved. Nothing already published changes.

## Agent-class variants and regime breaks

Wikimedia classifies traffic into agent classes: user, spider, and (from
2020-04-29) automated. We store all classes and publish three index variants:
user, user+automated, and all. Dates where the source's definitions changed are
listed in SOURCE_REGIMES.md; a quarter comparison whose two windows straddle a
regime boundary is reported as null with reason regime_break unless all
variants agree in sign.

## Integrity: what is proven and what is not

Every day's raw responses, sidecars, table files and entity-log snapshot are
listed in a manifest. Manifests are hash-chained, signed with Ed25519, and
externally timestamped by two independent RFC 3161 timestamp authorities, so
any alteration after sealing — by anyone, including us — is detectable by an
external verifier holding only our public key and any prior chain head.

This proves integrity after sealing. It does not prove the counts were
collected correctly. Collection correctness is addressed separately by a daily
public reproducibility job that rebuilds past dates from raw bytes, and by
clients re-fetching Wikimedia directly for spot checks.

## Personal-data exclusion

Any page whose Wikidata item has any P31 (instance of) statement with value Q5
(human) is excluded, even when the person is the brand: daily attention to a
named person is treated as personal data and is not collected or published.
Disambiguation pages are also excluded. Brand pages that are companies or
products are retained.

## Scorecard

The accuracy scorecard compares the index against subsequently reported
revenue. It includes all tickers and all quarters since go-live with no
exclusions, including quarters where the index was null or wrong. Per company,
the earnings time is the earlier of 00:00 UTC on the 8-K report date and the
filing's acceptance time, and the index value used is the one sealed in the
last manifest before that moment; both timestamps are recorded. Consensus
estimates are out of scope for this product. Reported metrics are quarterly
cross-ticker Spearman rank correlations, the same for revenue acceleration, and
a sign hit rate with a Wilson 95% interval; no per-ticker p-values are
reported.

## Trademarks

Brand names are used only as identifiers. Trademarks belong to their owners,
and no affiliation or endorsement is implied.