Product Truth Drift Index Methodology
For this report, StoreSteady evaluated a frozen corpus of 500 Shopify storefronts. This page documents selection, denominator, exclusion, redaction, source, and version rules.
Corpus And Window
Selection rule: Frozen external retail Shopify storefront corpus selected before analysis; no merchant-identifying examples are published.
Freeze timestamp: 2026-05-21T22:00:01.570Z
Evaluation window: 2026-05-23T16:50:03.820Z to 2026-05-23T23:24:45.338Z
Fields collected: public PDP product evidence, public Shopify products JSON where available, AI shopping answer product claim, price, currency, availability, product existence, and material attributes, calibration and reproducibility labels for confirmed product-truth mismatch.
Denominator Rules
Every published rate has its own evaluable N and Wilson 95% confidence interval.
Access friction, detector non-evaluability, deterministic conflicts, tentative findings, and confirmed findings are kept separate.
No blended denominator is used in this report.
Metric Quality Table
| Metric | Tier | N | Count | CI | Caveat |
|---|---|---|---|---|---|
| product_truth_mismatch | Headline | 273 | 12 | 2.53% - 7.52% | Publication-grade supporting evidence from the Breakage product_truth_mismatch row; not a connected-merchant-data compliance claim. |
| not_product_truth_evaluable | Limitation | 500 | 227 | 41.1% - 49.8% | Counted outside the product-truth denominator; missing or blocked evidence is not treated as a finding. |
Redaction Policy
- No store names, raw domains, raw URLs, raw crawl text, or raw AI responses are published.
- No exact promo codes, private feed/admin evidence, or merchant-identifying product examples are published.
- Public examples fail closed unless Caleb explicitly approves the specific exposure.
Related Public Benchmarks
This report is not a placement dashboard, market forecast, AEO forecast, or financial impact estimate. It measures evidence quality and evaluability under the StoreSteady public research protocol.
Sources
- StoreSteady published aggregate report JSON - Published aggregate artifact used as the source of calibrated public-report counts.
- OpenAI crawler documentation - Context for AI crawler and robots controls; not used as a ranking or reach claim.
- Google robots.txt guidance - Crawler-access context for robots restrictions and non-circumventing evaluation.
- Google merchant listing structured data - Public product-structured-data context for merchant listing evidence.
- OpenAI product feed specification - Product-data freshness context; not used as a connected-merchant-data audit.
- Shopify product taxonomy - Shopify product-data context for product categorization and storefront facts.
Version History
- 2026-summer.public-v1 (2026-05-25): Initial public report from the published Summer 2026 Breakage product-truth aggregate and scanner detector primitives.