{"schemaVersion":1,"qualityContract":{"title":"Research Report Quality Contract & Publish Template v1","version":"2026-summer.public-v1","path":"tasks/specs/research-report-quality-contract-v1.md"},"reportName":"Product Truth Drift Index","slug":"product-truth-drift-index","releaseLabel":"Summer 2026","methodologyVersion":"2026-summer.public-v1","generatedAt":"2026-05-25T16:01:25.453Z","paths":{"canonical":"/resources/research/product-truth-drift-index","methodology":"/resources/research/product-truth-drift-index/methodology","aggregateCsv":"/resources/research/product-truth-drift-index/product-truth-drift-index-summer-2026.csv","reportJson":"/resources/research/product-truth-drift-index/report.json","reportJsonDownload":"/resources/research/product-truth-drift-index/report-json","ogImage":"/api/og/research-report/product-truth-drift-index"},"headline":{"metric":"product_truth_mismatch","label":"Breakage-confirmed product-truth mismatch","tier":"headline","denominatorLabel":"Product-truth-evaluable stores","evaluableN":273,"findingN":12,"alpha":0.9233,"accuracy":0.9899,"flaggedN":61,"nonFindingN":139,"caveat":"Publication-grade supporting evidence from the Breakage product_truth_mismatch row; not a connected-merchant-data compliance claim.","rate":0.044,"ciLow":0.0253,"ciHigh":0.0752,"statement":"12 of 273 product-truth-evaluable stores had a confirmed public product-truth mismatch.","interpretation":"The publication-grade rate is supporting Breakage evidence. The v1 report does not yet publish a full PDP-vs-schema-vs-sitemap deterministic drift census."},"corpus":{"publicDescription":"For this report, StoreSteady evaluated a frozen corpus of 500 Shopify storefronts.","selectionRule":"Frozen external retail Shopify storefront corpus selected before analysis; no merchant-identifying examples are published.","freezeTimestamp":"2026-05-21T22:00:01.570Z","evaluationWindow":"2026-05-23T16:50:03.820Z to 2026-05-23T23:24:45.338Z","scannedStoreN":500,"fieldsCollected":["public PDP product evidence","public Shopify products JSON where available","AI shopping answer product claim","price, currency, availability, product existence, and material attributes","calibration and reproducibility labels for confirmed product-truth mismatch"]},"evidenceFunnel":[{"label":"Frozen corpus","value":"500","detail":"Shopify storefronts entered the public research frame."},{"label":"Product-truth evaluable","value":"273","detail":"Stores with enough public product evidence to evaluate the Breakage product-truth metric."},{"label":"Confirmed mismatch","value":"12","detail":"Stores with a calibrated, reproducible product-truth mismatch."},{"label":"Access friction separate","value":"210","detail":"Stores with access friction are not counted inside the product-truth mismatch rate."}],"metricQuality":[{"metric":"product_truth_mismatch","label":"Breakage-confirmed product-truth mismatch","tier":"headline","denominatorLabel":"Product-truth-evaluable stores","evaluableN":273,"findingN":12,"alpha":0.9233,"accuracy":0.9899,"flaggedN":61,"nonFindingN":139,"caveat":"Publication-grade supporting evidence from the Breakage product_truth_mismatch row; not a connected-merchant-data compliance claim.","rate":0.044,"ciLow":0.0253,"ciHigh":0.0752},{"metric":"not_product_truth_evaluable","label":"Not in product-truth denominator","tier":"limitation","denominatorLabel":"Frozen scanned storefront corpus","evaluableN":500,"findingN":227,"alpha":null,"accuracy":null,"flaggedN":null,"nonFindingN":null,"caveat":"Counted outside the product-truth denominator; missing or blocked evidence is not treated as a finding.","rate":0.454,"ciLow":0.4109,"ciHigh":0.4978}],"detectorPrimitives":[{"signalId":"scanner_variant_group_markup_missing","label":"ProductGroup or variant markup missing","status":"smoke validated; no frozen-corpus rate","caveat":"Detects visible variants without ProductGroup, hasVariant, productGroupID, or isVariantOf markup."},{"signalId":"scanner_offer_url_canonical_mismatch","label":"Offer URL and canonical product URL disagree","status":"smoke validated; no frozen-corpus rate","caveat":"Compares canonical, sitemap, Offer.url, and selected-variant URL paths."},{"signalId":"scanner_image_url_inconsistent","label":"Product image signals disagree","status":"smoke validated; no frozen-corpus rate","caveat":"Compares PDP primary image, og:image, and Product.image identity."},{"signalId":"scanner_price_ambiguity","label":"Visible and structured prices are ambiguous","status":"smoke validated; no frozen-corpus rate","caveat":"Detects multiple prices without an obvious sale, member, subscription, or active-price context."}],"attributeReadiness":null,"sections":[{"id":"what-measured","heading":"What StoreSteady Measured","paragraphs":["The publication-grade row measures product facts that could be compared against public product evidence and that survived calibration and reproducibility gates.","StoreSteady treats this as product-truth drift evidence because the public answerable product fact and observed public product evidence disagree. It is not a connected-feed audit."],"bullets":["Product-truth mismatch rate: 12 / 273 evaluable stores, Wilson 95% CI 2.53% to 7.52%.","Judgment quality: alpha 0.9233 and calibration accuracy 98.99%.","Deterministic PDP/schema drift primitives exist but are not published as corpus-wide rates in v1."]},{"id":"deterministic-scope","heading":"Deterministic Drift Scope","paragraphs":["The upstream public-drift detector family covers PDP copy, JSON-LD/Product and ProductGroup markup, Offer.url, canonical URL, sitemap URL, image signals, price text, and availability context.","Those detector primitives were not persisted as a frozen-corpus aggregate in the published Breakage artifact. The report therefore lists them as detector-ready primitives instead of manufacturing a denominator."]},{"id":"not-measured","heading":"What StoreSteady Did Not Measure","paragraphs":["The report does not audit private merchant-feed state, does not use private admin evidence, and does not infer hidden model behavior.","It also does not publish named-store examples, raw URLs, exact product names, raw AI responses, source quotes, or private feed/admin evidence."]},{"id":"quality-gates","heading":"Quality Gates","paragraphs":["The product_truth_mismatch row uses Breakage-level calibration gates: two independent judges, human tiebreak policy, alpha, accuracy, false-positive and false-negative review, and 2-of-3 reproducibility for confirmed candidates.","The deterministic public-surface detector rows require a future frozen-corpus denominator audit before any rate can be published."]}],"limitations":["The headline is a Breakage-confirmed supporting product-truth row, not a full PDP-vs-schema-vs-sitemap deterministic drift census.","No private Merchant Center feed, Shopify admin, or connected-store data is used in public examples or claims.","Dynamic pricing, sale timing, locale, and member-only price contexts can create false positives; future deterministic rates require timestamped crawl windows and tolerance bands."],"sources":[{"label":"StoreSteady published aggregate report JSON","url":"https://www.storesteady.com/resources/research/agentic-commerce-breakage-index/report.json","note":"Published aggregate artifact used as the source of calibrated public-report counts."},{"label":"OpenAI crawler documentation","url":"https://developers.openai.com/api/docs/bots","note":"Context for AI crawler and robots controls; not used as a ranking or reach claim."},{"label":"Google robots.txt guidance","url":"https://developers.google.com/search/docs/crawling-indexing/robots/intro","note":"Crawler-access context for robots restrictions and non-circumventing evaluation."},{"label":"Google merchant listing structured data","url":"https://developers.google.com/search/docs/appearance/structured-data/merchant-listing","note":"Public product-structured-data context for merchant listing evidence."},{"label":"OpenAI product feed specification","url":"https://developers.openai.com/commerce/specs/file-upload/products","note":"Product-data freshness context; not used as a connected-merchant-data audit."},{"label":"Shopify product taxonomy","url":"https://help.shopify.com/en/manual/products/details/product-category","note":"Shopify product-data context for product categorization and storefront facts."}],"versionHistory":[{"version":"2026-summer.public-v1","date":"2026-05-25","change":"Initial public report from the published Summer 2026 Breakage product-truth aggregate and scanner detector primitives."}],"redactionPolicy":{"aggregateOnly":true,"rowLevelPublicFieldsAreHashed":null,"excludedPublicFields":["store names","raw domains","raw URLs","raw crawl text","raw AI responses","exact promo codes","private feed/admin evidence","merchant-identifying product examples"]}}