Connecting to the live Observatory…
MEASUREMENT CONTRACT 1.5 Awaiting current snapshot

LIVE SEARCH & AI CRAWLER EVIDENCE

See Which Search & AI Crawlers Are Reading Your Website

Watch machine retrieval as it happens. This live Observatory measures which search and AI-related crawlers reach our website, whether their identities are corroborated, and whether they successfully retrieve useful public information.

Not a demonstration dataset. These are actual requests to Sydney Business Web, measured over a rolling 24-hour window.

01 / CORROBORATED Qualifying crawler systems — Autonomous crawlers matching published operator IP ranges
02 / RETRIEVAL Successful business retrievals — Qualifying business-information GET requests
03 / OUTCOME Business retrieval success rate —% Only when qualifying business attempts exist
04 / OBSERVED SEPARATELY Other machine requests observed — Diagnostic traffic, not counted as verified retrievals
The broader pictureNot everything observed meets the corroboration threshold — and that's useful to know.
—Corroborated accesses
—Systems shown separately

And this is not just about Sydney Business Web.

This Observatory can be installed on suitable business websites to demonstrate what real machines request, which claims can be corroborated and where retrieval succeeds or fails. The diagnostic population remains visible separately, without inflating the evidence.

Latest evidence snapshot—
Next automatic refresh—
Measurement windowRolling 24 hours

Contract 1.5: a claimed crawler User-Agent must match a published operator IP range to qualify. This is source-network corroboration, not cryptographic proof of a particular crawler or evidence of endorsement.

LIVE / ROLLING 24-HOUR EVIDENCE

Who Has Actually Been Here?

A live record of search and AI-related machine requests reaching Sydney Business Web. Contract 1.5 distinguishes IP-corroborated autonomous crawlers from other recognisable but separately classified machine activity.

CRAWLER SYSTEMS — Qualifying systems observed

Distinct autonomous crawlers whose claimed identities match published operator IP ranges. This is an observed count, not a fixed list.

BUSINESS INFORMATION — Successful business retrievals

Qualifying GET requests for current registered business resources that returned a successful 2xx response.

BUSINESS OUTCOMES —% Retrieval success rate

The proportion of scored business-information attempts that succeeded. Shows N/A when there are no qualifying attempts.

MACHINE DISCOVERY — Successful discovery retrievals

Qualifying successful requests for approved discovery resources, including robots.txt and sitemaps.

Below are the systems behind those figures. Every crawler in the first table contributes to the corroborated evidence above. The second table shows additional observed activity without adding it to those totals.

CORROBORATED IDENTITY + AUTONOMOUS ACTIVITY

Which Crawlers Produced the Evidence?

Only qualifying systems observed during the current rolling 24-hour window appear here. The population expands or contracts with real activity; there is no preset system count.

QUALIFYING THIS 24H — No fixed limit
IP-corroborated autonomous crawler evidence in the last 24 hours
Crawler / provider Role Autonomous
accesses
Business
retrievals
Business
success rate
Discovery
retrievals
Loading corroborated crawler evidence…

A green marker indicates source-IP corroboration against an operator-published range. It is not cryptographic verification of a specific crawler or an endorsement.

VISIBLE, BUT NOT COUNTED AS CORROBORATED EVIDENCE

The Broader Machine Picture

These requests carried recognisable crawler identities but did not qualify for the autonomous, IP-corroborated headline population. Some are unverified identity claims; others may be corroborated but non-autonomous. We show what was observed without treating a User-Agent claim as proof.

—Other observed systems
—Diagnostic accesses
Identity unverified—
Corroborated, non-autonomous—
Additional machine requests excluded from corroborated headline evidence
Claimed crawler / provider Classification Recorded accesses Headline eligibility
Loading other observed crawler activity…

Important: An unverified crawler identity means the public corroboration threshold was not met, not that traffic was absent. These accesses are excluded from all corroborated crawler and retrieval metrics above. Multiple crawler names may belong to one provider; counts represent crawler identities, not necessarily separate companies.

A changing 24-hour population — never a fixed brand list.

Systems appear when activity meets the relevant classification rules and disappear when it leaves the rolling window. The table may show more or fewer names tomorrow. That is measurement doing its job.

What qualifies as public retrieval evidence?

More traffic does not automatically mean better evidence. The Observatory keeps these distinctions explicit.

01 / TEST TRAFFICOur tests are excluded

Synthetic commissioning or diagnostic requests do not enter production evidence.

02 / REQUEST INTENTAutonomy matters

User-requested or owner-triggered fetches are not counted as autonomous crawling.

03 / CLAIMED IDENTITYA User-Agent isn't proof

Claims without published-IP corroboration are visible separately, not promoted into headline numbers.

04 / RESOURCE TYPEBusiness content is distinct

Only approved business resources contribute to business retrieval scores; discovery files are scored separately.

Evidence over appearances. The public dashboard deliberately distinguishes corroborated autonomous retrievals from separately observed machine requests. Neither activity alone establishes AI understanding, citation or recommendation.

03 / WHAT IT MEANS FOR YOUR BUSINESS

What Could This Tell You About Your Website?

The live evidence above happens to come from Sydney Business Web because this is where we built and commissioned the system. The principle is not limited to our website. A suitable business website can be instrumented to observe its own search and AI-related machine retrieval activity.

THE BUSINESS-OWNER VERSION

“Are Google, Bing and AI-related machine systems actually reaching my website — and can they retrieve what I want them to see?”

Instead of assuming the answer is yes, the Observatory provides evidence of what is actually happening. Corroborated crawlers and other observed requests are reported separately.

FOUR PRACTICAL QUESTIONS

From crawler activity to useful answers.

Not another vague visibility score. Four distinct things a business can check.

01

Who is reaching the site?

See which crawler systems meet the published-IP corroboration standard, alongside other crawler-labelled requests that remain explicitly unverified — rather than a theoretical list of bots that might visit someday.

02

Are they retrieving business information?

Separate meaningful retrieval of public business content from requests for images, scripts, fonts, diagnostic resources and other background website traffic.

03

Is retrieval succeeding?

Measure whether qualifying requests for business information actually succeed. A crawler arriving at the door is not the same thing as successfully retrieving the information inside.

04

Has something changed?

Because the evidence is time-based, changes become visible. A machine system can appear, disappear, increase its activity or begin encountering retrieval failures.

AN EXTRA LAYER OF VISIBILITY
TRADITIONAL VIEW

Website analytics usually asks:

“What are my human visitors doing?”

Sessions, page views, conversions, traffic sources and visitor behaviour remain extremely useful business measures.

+
MACHINE RETRIEVAL VIEW

This Observatory asks:

“What are machine systems actually retrieving?”

It adds a different layer of visibility: observable retrieval activity from search and AI-related machine systems.

A SIMPLE EXAMPLE

Suppose an OpenAI-related crawler appears in your corroborated evidence today, successfully retrieves several business pages, and then disappears from tomorrow's rolling window.

That does not mean ChatGPT has suddenly forgotten your business. It means exactly what the instrument says: qualifying activity from that machine system was observed during one measurement window and was not observed during the other.

The dashboard reports evidence — not a story invented around the evidence.

WHY THIS MATTERS

AI visibility should not begin and end with asking an AI system what it thinks about your business.

Those tests can be useful, but they examine the result from the outside. Retrieval evidence gives us another view entirely: whether corroborated machine systems are actually accessing the website and successfully retrieving its information.

Discoverability Access Retrieval Understanding Citation Recommendation
This Observatory measures the access and retrieval portion of that larger AI visibility chain. It does not directly measure understanding, citation or recommendation.
SO HOW DO WE KNOW THE NUMBERS ARE REAL?

That is where the engineering comes in. We distinguish corroborated autonomous evidence from other diagnostic activity before making any public claims.

See the Methodology

04 / HOW WE MEASURE

How Do We Know These Numbers Are Real?

The public dashboard does not count a request simply because it calls itself a crawler. AI Observatory records selected production request and response evidence, then applies explicit identity, activity and resource rules before including it in a public measurement.

CONTRACT 1.5 Current measurement standard 24 HOURS Rolling observation window SEPARATE Corroborated and diagnostic populations
The headline number is deliberately smaller than the activity observed.

When a claimed crawler identity cannot be sufficiently corroborated, or an otherwise corroborated request is not autonomous, that activity may be reported separately as diagnostic evidence. It does not inflate the qualifying crawler or retrieval figures.

THE MEASUREMENT PIPELINE

Five decisions between a request and a public figure.

Each stage narrows what we can responsibly claim, rather than manufacturing a single AI visibility score.

01

Observe production traffic

The collector records selected technical characteristics of requests and responses at the live website edge: claimed crawler, requested resource and returned HTTP status.

REAL REQUESTS, NOT SIMULATED CLAIMS
02

Corroborate the identity

A Googlebot, ClaudeBot or other User-Agent label is not proof of origin. Headline eligibility requires the claimed operator's published source-IP range to match.

USER-AGENT ALONE IS NOT ENOUGH
03

Separate autonomous activity

Owner-triggered diagnostics and user-requested fetches are not independent autonomous crawling. When recorded, they remain outside the autonomous headline population.

AUTONOMOUS MEANS AUTONOMOUS
04

Qualify the resource

Business-content scoring requires a current URL in the explicitly approved resource registry. A separate small registry defines which machine-discovery files count.

ONLY APPROVED RESOURCES ARE SCORED
05

Score the HTTP outcome

Qualifying GET requests returning 2xx count as success; 4xx and 5xx count as failure. Redirects (3xx) remain observed but are neutral in success-rate denominators.

THE DENOMINATOR IS DEFINED
Production request IP corroboration Autonomous activity Approved resource + outcome Counted evidence

A CRITICAL DISTINCTION

Two visible populations. Two different claims.

01 / CORROBORATED EVIDENCE

Eligible for headline measurements

Autonomous crawler activity with a claimed identity matched against the operator's published IP ranges. Qualifying business and discovery requests are scored according to their own resource and response rules.

Included in verified system and retrieval totals.
02 / OBSERVED DIAGNOSTIC ACTIVITY

Visible, but not counted as verified

Requests carrying recognisable crawler labels that do not satisfy the headline criteria. These include unsupported identity claims and, where present, corroborated non-autonomous fetches.

Reported separately, never added to headline totals.

What corroboration means: A matching operator-published IP range supports the claimed provider identity, but is not cryptographic proof of the particular crawler. An unverified label means we could not meet that threshold — not that the request did not happen.

FILTERING MATTERS

What does not inflate the numbers?

Different exclusions apply to different measurements. Not every request is a business-information retrieval, and not every crawler-labelled request qualifies as autonomous.

01 / EXCLUDED ENTIRELY

Our synthetic tests

Commissioning and test requests identified as synthetic are excluded from Contract 1.5 production evidence, including diagnostics.

02 / DIAGNOSTIC ONLY

Triggered or requested fetches

Owner-triggered inspection and user-requested fetches do not qualify as autonomous crawling. When observed, they are classified separately.

03 / DIAGNOSTIC ONLY

Unsupported crawler identities

A familiar User-Agent does not establish origin. Requests without the required IP corroboration stay outside the verified population.

04 / NOT BUSINESS CONTENT

Supporting assets

Images, CSS, JavaScript, fonts and video are not registered business resources; their retrieval cannot inflate the business-content success count.

05 / NOT BUSINESS CONTENT

Security and configuration probes

Requests for secrets, configuration paths and unrelated files do not count as successful retrieval of substantive business information.

06 / NOT SCORED AS RETRIEVAL

HEAD checks and old records

HEAD requests do not retrieve a resource body. Historical Contract 1.4 records are also excluded from Contract 1.5 measurements.

TWO TYPES OF RESOURCE

Business information is not machine discovery.

BUSINESS INFORMATION

Did a qualifying crawler retrieve substantive public content?

Only GET requests for current, approved business-resource URLs enter this score. A 2xx response succeeds, 4xx/5xx fails, and 3xx is neutral.

Examples Service pages, articles and approved public documents
≠
MACHINE DISCOVERY

Did a qualifying crawler successfully reach discovery files?

Only GET requests for the five approved discovery paths are scored: robots.txt, llms.txt, wp-sitemap.xml, sitemap.xml and sitemap_index.xml. The same HTTP outcome rules apply.

Examples robots.txt and sitemap files

READING THE DASHBOARD

Four headline figures. And one separate view.

01 / HEADLINE

Qualifying machine systems

How many distinct, IP-corroborated autonomous crawler systems were observed?

02 / HEADLINE

Business retrievals

How many qualifying GET requests for current business resources returned 2xx?

03 / HEADLINE

Business success rate

Of the scored business attempts (2xx, 4xx and 5xx), what percentage succeeded? No attempts means N/A.

04 / HEADLINE

Discovery retrievals

How many qualifying requests for approved discovery resources returned 2xx?

Separate diagnostic view: other observed crawler systems, their request counts and why their activity was not eligible for the verified population.

FOR THOSE WHO WANT THE ENGINE ROOM

The Technical Architecture

PRODUCTION IMPLEMENTATION

The deployed Observatory collects selected request and response data at the Cloudflare edge. Analytics Engine stores the measurements. A separate reporting Worker applies the Contract 1.5 aggregation rules and serves the public evidence endpoint consumed by this dashboard.

Cloudflare edgeObserve selected requests
Analytics EngineStore measured activity
Reporting WorkerClassify and aggregate
Public dashboardDisplay both populations

The public API reports a rolling 24-hour window. Contract 1.4 historical records are not silently merged into the new Contract 1.5 figures.

Raw telemetry is evidence material. It is not yet a conclusion.

Measurement rules turn the activity we can observe into a smaller set of claims we can explain and defend.
ONE IMPORTANT BOUNDARY REMAINS

Retrieval proves retrieval — not understanding or recommendation.

Next: what the evidence supports, and what it cannot establish about an AI system's internal decisions.

What Does This Prove?

05 / THE EVIDENCE BOUNDARY

What This Proves — and What It Does Not

Good measurement becomes less useful the moment we claim more than the evidence actually shows. This observatory has a deliberately narrow job: to measure observable machine access and retrieval.

Retrieval proves retrieval. It does not automatically prove what happened inside an AI system after the information was retrieved.

✓
CORROBORATED EVIDENCE SUPPORTS

What We Can Say

A qualifying machine system reached the website.

Observable production traffic shows that the website received qualifying machine requests during the measured period.

Its identity met our public corroboration threshold.

The claimed identity was supported by the relevant operator-published source-IP range, not simply a familiar crawler name.

It attempted to retrieve specific resources.

The observatory can distinguish requests for substantive business information from discovery files, support assets and other traffic.

Qualifying retrievals succeeded or failed.

The website response allows us to measure whether retrieval attempts were successfully completed.

Machine-discovery resources were accessible.

Retrieval of resources such as robots.txt and sitemaps can be measured separately from business-content retrieval.

×
THE EVIDENCE DOES NOT ESTABLISH

What We Cannot Say

That an AI system understood the information.

Successful retrieval tells us the information was accessible, not how an AI model interpreted it internally.

That the business was cited in an AI answer.

Retrieval is a necessary precursor to many forms of representation, but it is not itself evidence of citation.

That the business was recommended.

A crawler retrieving a page does not prove that an AI assistant later selected the business for a recommendation.

That Google, OpenAI, Anthropic or another provider endorses us.

Machine access is technical activity. It must never be represented as endorsement or approval by the provider.

That retrieval produced a commercial outcome.

Leads, enquiries, sales and other business outcomes are separate downstream measurements.

WHERE THIS MEASUREMENT SITS

AI Visibility Is a Chain, Not a Single Event

Retrieval is important because a machine cannot reliably work with information it cannot first access. But retrieval is only one part of the larger path from being discoverable to producing a business outcome.

01 Discoverability
→
02 Access
→
03 Retrieval
→
04 Understanding
→
05 Citation / Representation
→
06 Recommendation / Outcome

Highlighted stages: the principal part of the chain measured by this retrieval observatory.

!
The permanent measurement disclaimer

Retrieval evidence shows observable machine access to this website. It does not by itself prove AI understanding, citation, recommendation or endorsement.

WHY DRAW THE LINE SO CLEARLY?

Because a measurement is only useful if you know what it measures.

We could make the story sound more impressive by blending crawler activity, AI answers, rankings, citations and recommendations into one vague “AI visibility score”.

We deliberately do not.

Access is access. Retrieval is retrieval. Representation is representation. Outcomes are outcomes. Each deserves its own evidence.

WANT TO CHECK THE WORK?

The methodology and evidence are published.

The final part of this page links to the supporting technical material, related AI Visibility Verification work and the next step for businesses interested in measuring their own machine retrieval.

See the Evidence & References ↓

06 / SUPPORTING WORK & VERIFICATION

Don’t Just Take the Dashboard at Face Value.

The live display is the visible end of a larger measurement system. We have published the reasoning, measurement boundaries and real-world retrieval evidence behind it so that the method can be examined rather than treated as a black-box marketing claim.

LIVE REPORTING ENDPOINT The dashboard is fed by a separate production reporting service.

The figures displayed above are not hard-coded into this page. The page requests its current measurements from the Sydney Business Web evidence reporting endpoint. It reports corroborated autonomous crawler evidence and separately identifies observed diagnostic activity that does not qualify for the verified headline measurements.

View the Raw Public Evidence Feed ↗

NOW BRING IT BACK TO YOUR BUSINESS

Could We Measure This on Your Website?

Potentially, yes.

This Sydney Business Web implementation is our working production example. The same measurement principles can be applied to suitable business websites where the required access and technical environment allow reliable observation of machine traffic.

The aim is simple: instead of wondering whether search and AI-related systems are reaching your website, give the business a way to observe and measure what is actually happening.

POSSIBLE BUSINESS OUTPUT
✓

Which IP-corroborated autonomous search and AI-related crawler systems were observed

✓

Whether they successfully retrieved substantive business information

✓

Whether machine-discovery resources were successfully accessible

✓

How machine activity changes across a rolling observation period

✓

Which observations qualified for defensible public evidence — and which remain diagnostic

MEASUREMENT CONTRACT 1.5
Two visible populations. One defensible measurement.

Verified autonomous crawlers contribute to the public evidence figures. Other observed crawler-labelled requests appear in a separate diagnostic population, with their verification limits plainly stated. Neither population implies AI understanding, citation or endorsement.

NOT EVERY WEBSITE IS IDENTICAL

We Would Check the Environment First.

This is instrumentation, not a plugin badge. Before promising a dashboard, we need to know that the website and its delivery stack give us the technical access required to measure the right things.

01

Technical Suitability

We examine the website, hosting, DNS/CDN arrangement and the available observation points.

02

Measurement Design

We determine which resources and machine behaviours matter for the business and define what should — and should not — count.

03

Observatory Deployment

The measurement layer is implemented, tested and separated from synthetic commissioning traffic.

04

Baseline & Reporting

Real production observations establish the baseline from which machine retrieval can be monitored over time.

LIVE

AI VISIBILITY SHOULD BE OBSERVABLE

Want to Know What the Machines Are Actually Doing on Your Website?

Talk to Sydney Business Web about whether a machine-retrieval observatory is technically appropriate for your website and how it could fit into a wider AI Visibility measurement program.

No claims of guaranteed AI citation, recommendation or ranking. We measure what the available evidence allows us to measure.

Engineer → Corroborate → Verify Retrieval → Test Representation

Sydney Business Web — AI Visibility Verification

PRACTICAL QUESTIONS

Live AI Retrieval Evidence FAQ

The measurement is deliberately narrow. These are the practical questions we would expect a business owner to ask after seeing the live evidence above.

01 Can this system be installed on any website? +

Not necessarily. The website, hosting, DNS/CDN arrangement and available technical access need to be assessed first. The measurement system requires a reliable observation point where relevant machine requests and website responses can be recorded without interfering with normal website operation.

02 Does seeing an AI crawler mean my business is being recommended by AI? +

No. It can provide evidence that a qualifying machine system reached the website and successfully retrieved information. It does not by itself prove that an AI system understood, cited, represented, recommended or endorsed the business.

03 Why do the crawler names change from day to day? +

Because the public dashboard uses a rolling 24-hour observation window. A crawler system appears when qualifying activity from it is observed inside that window and can disappear again when that activity falls outside it.

The displayed population is therefore evidence of what was actually observed during the current period — not a fixed list of crawler brands.

04 Why are some machine requests excluded from the public figures? +

Because counting everything would produce a larger number, but a less useful one.

Synthetic tests, owner-triggered diagnostics, insufficiently corroborated crawler identities, support assets, security and configuration probes, HEAD-only requests and other non-qualifying activity are prevented from inflating the headline retrieval evidence.

05 Can the dashboard show more than five crawler systems? +

Yes. There is no five-system limit and no fixed crawler list. The dashboard is generated from the qualifying systems actually observed during the rolling measurement window.

It may therefore display three systems, five systems, eight systems or more as real machine activity changes.

06 Can this tell me whether ChatGPT, Claude or another AI system has actually read my website? +

It can show qualifying retrieval activity attributable to relevant machine systems when that activity is actually observed and meets the required corroboration and measurement rules.

That is strong evidence of machine access and retrieval. It does not reveal what an AI model subsequently understood, retained, cited or did with the retrieved information.

THE SHORT VERSION

We measure what happened at the website. We do not pretend that this reveals everything that happened afterwards inside somebody else's AI system.

EVIDENCE & TECHNICAL SOURCES

Internal Research & External References

The live retrieval evidence above sits on top of published Sydney Business Web research and first-party technical documentation from the companies operating the infrastructure, search engines and machine systems being measured.

FIRST-PARTY DOCUMENTATION

External Technical Sources

Wherever practical, we reference the organisations actually operating the infrastructure and crawler systems rather than secondary marketing or SEO commentary.

CF Cloudflare

Workers Analytics Engine

Cloudflare documentation for the analytics platform used to record and aggregate measurements from the production retrieval observatory.

Official Documentation ↗
CF Cloudflare

Analytics Engine SQL API

Documentation for querying Analytics Engine datasets through the SQL API used by the separate reporting layer behind the public evidence dashboard.

Official Documentation ↗
CF Cloudflare

Verified Bots

Cloudflare's documentation covering verified automated systems and the distinction between an asserted crawler identity and independently verified bot traffic.

Official Documentation ↗
G Google Search Central

Googlebot & Crawler Verification

Google's first-party documentation describing Googlebot, crawler identification and methods for verifying whether a request genuinely originates from Google.

Official Documentation ↗
B Microsoft Bing

Verify Bingbot

Microsoft's guidance for determining whether traffic claiming to be Bingbot actually originates from Bing infrastructure.

Official Documentation ↗
AI OpenAI

OpenAI Publishers & Developers

OpenAI guidance covering web discovery, crawler controls and OAI-SearchBot access for content that may be surfaced through ChatGPT search.

Official Documentation ↗
A Anthropic / Claude

Anthropic Web Crawlers & Site Controls

Anthropic's first-party documentation describing its web crawler activity and the controls available to website owners.

Official Documentation ↗
AP Apple

About Applebot

Apple's documentation describing Applebot, its uses, crawler identification, verification methods and website access controls.

Official Documentation ↗
✓
Primary sources wherever possible.

We deliberately favour documentation published by the organisations operating the infrastructure and machine systems being discussed.

LIVE RETRIEVAL MEASUREMENT

NOW PUT THE INSTRUMENT ON YOUR WEBSITE

Want to Know Which Search & AI Crawlers Are Actually Reaching Your Business?

Sydney Business Web can assess whether your website is suitable for machine-retrieval monitoring and, where technically appropriate, build an observatory around your own production traffic.

✓

Stop relying entirely on crawler lists, assumptions and occasional AI searches. Measure what is actually happening at your website.

01 Observe Real machine traffic
→
02 Corroborate Machine identity
→
03 Measure Successful retrieval
→
04 Track Changes over time

We assess technical suitability first. Retrieval monitoring measures observable machine access and retrieval — it does not guarantee AI citation, recommendation, ranking or endorsement.

Engineer → Corroborate → Verify Retrieval → Test Representation Sydney Business Web — AI Visibility Engineering
Engineering Implementation Note How raw machine traffic becomes defensible retrieval evidence Implementation detail for developers, infrastructure teams and technical reviewers

Production telemetry separated from public reporting

The AI Observatory measures machine activity reaching the production website at the Cloudflare edge. A production Cloudflare Worker records selected request and response characteristics into Cloudflare Analytics Engine.

Public reporting is handled separately. A read-only reporting Worker queries the telemetry dataset, applies the Observatory's evidence rules, aggregates the qualifying observations and supplies the live dashboard through a dedicated evidence endpoint.

01 Production Edge Observe live machine requests and responses
02 Analytics Engine Persist selected telemetry measurements
03 Reporting Worker Qualify, classify and aggregate observations
04 Evidence Endpoint Supply the rolling public evidence display

Evidence is qualified, not merely counted

Raw automated traffic is not treated as retrieval evidence. A crawler-labelled User-Agent alone is insufficient. Observations must satisfy the applicable identity, autonomy, resource and retrieval-state rules before they can contribute to the public measurements.

Production Traffic  →  Corroborated Identity  →  Autonomous Activity  →  Resource Classification  →  Successful Retrieval  →  Public Evidence

Synthetic commissioning requests, owner-triggered diagnostics, insufficiently corroborated crawler identities, support assets, security probes and HEAD-only requests are prevented from inflating substantive business-information retrieval figures.

Cloudflare Workers Edge Telemetry Analytics Engine Request Classification Response Status Identity Corroboration Autonomous Activity Resource Filtering Rolling 24-Hour Evidence

The evidence boundary is deliberate

The resulting telemetry establishes observable evidence of machine access and retrieval. It does not claim to expose the internal reasoning, model weights or downstream decision processes of Google, OpenAI, Anthropic, Microsoft or other AI providers.

Successful retrieval therefore proves that qualifying machine activity reached and retrieved information from the website. It does not by itself prove subsequent understanding, citation, representation, recommendation or endorsement.