AI Agent Readiness — An Engineer's Field Notes, Investigation 01. Sydney Business Web tests Cloudflare Markdown for Agents on a live production website, examining content conversion, JSON-LD preservation and practical engineering implications.
AI Agent Readiness / An Engineer’s Field Notes / Investigation 01
The website speaks Markdown. Does that mean the AI understands it?
Cloudflare can turn a live HTML page into Markdown when a machine asks for it. We enabled the feature on just one production page, inspected the response and checked whether our connected business-identity schema survived the journey.
As it turns out, the switch works. That is a useful result, but an engineer who stops testing when the indicator lamp comes on deserves whatever happens next.
Field test: 28 October 2026 · Test platform: Sydney Business Web production homepage · Configuration: Cloudflare Pro, single-page rule · Result: successful conversion and verified JSON-LD structural fidelity
01 / Why this matters
Not every machine needs your website dressed up as a browser page.
Traditional websites deliver HTML, styling, scripts and interface elements so browsers can display a complete experience. An automated reader may only need the underlying content, headings, links and facts.
Markdown is a comparatively simple text format with recognisable headings, lists and links. Cloudflare’s Markdown for Agents feature uses HTTP content negotiation: a requesting system asks for text/markdown and Cloudflare can return a converted representation of the same page.
That is the mechanism. Whether a particular AI crawler asks for it, benefits from it or uses it in a subsequent answer is a separate question.
The engineering distinction
Access means a system successfully receives a response.
Integrity means that useful information survives the change in representation.
Interpretation means a system makes appropriate sense of that information.
Commercial benefit means something genuinely useful happens for a business.
We tested the first two. We are not claiming the last two.
02 / Our test setup
A controlled trial, not an enthusiastic flick of every switch.
We used the Sydney Business Web homepage: a real production WordPress page with Thrive-designed content, navigation, FAQs and multiple manually maintained JSON-LD graphs. It is a more demanding test case than a plain demonstration page.
We upgraded our Cloudflare plan to Pro to access the feature and left the zone-wide toggle disabled. Instead, we deployed a Configuration Rule matching only the canonical homepage:
(http.host eq "sydneybusinessweb.com.au" and http.request.uri.path eq "/")That rule enables Markdown for Agents for matching requests, while avoiding a site-wide rollout before the behaviour is understood. The rule matches the path independently of query parameters; it is not a crawler-specific rule.
Homepage only. Existing site-wide setting remains off.
Use Accept: text/markdown in an HTTP request.
Record response headers, content and preserved JSON-LD.
Cloudflare documents this selective deployment method in its Markdown for Agents documentation.
03 / Reproducible request
One HTTP request. No mystical prompting required.
Our Windows PowerShell request was deliberately straightforward:
curl.exe -sS -D md-headers.txt -o homepage.md -H "Accept: text/markdown" "https://sydneybusinessweb.com.au/"We then inspected the returned headers and the start of homepage.md. The recorded response included:
HTTP/1.1 200 OK
Content-Type: text/markdown; charset=utf-8
CF-Cache-Status: HIT
Age: 2557
Vary: Accept-Encoding,User-Agent, accept
Content-Signal: ai-train=yes, search=yes, ai-input=yesThese are observations from one successful production response, not a statistical reliability survey. The HIT and Age values show that a cached response was involved; they do not on their own prove every possible HTML/Markdown cache variation is safe.
The response began with YAML metadata containing a description, title and image, followed by the converted body. In the captured output, substantial navigation also remained.
| Evidence | Observed result | Interpretation |
|---|---|---|
| HTTP status | 200 OK | Successful retrieval |
| Representation | text/markdown; charset=utf-8 | Conversion activated |
| Negotiation | Vary: ... accept | Accept included as a response variant |
| Cache | CF-Cache-Status: HIT | Cached response observed |
| Metadata | Title, description, image | YAML frontmatter present |
| Structured data | 6 original objects present | Complete object-level comparison performed |
A useful caution: the response did not establish that all recognised AI crawlers request Markdown. Clients must actually send the appropriate Accept header for this negotiation path to matter.
04 / The important integrity test
Did the Intelligent Entity Skeleton make it through?
For Sydney Business Web, this was the central test. Our homepage is not merely descriptive text: it includes a connected schema model defining the website, business, people, services, FAQ relationships and external identity references.
Cloudflare documents preservation of JSON-LD — and only JSON-LD among scripts — during conversion. All other script and style content is removed. Our live production test confirmed this behaviour for a homepage containing six connected JSON-LD objects. Cloudflare places the page’s JSON-LD scripts at the end of the converted document, inside a single fenced JSON code block. In our case, the fence contains six successive standalone JSON objects. That is a documented output convention, not one combined JSON document; a parser attempting json.loads() on the entire fence will report extra data.
We parsed the six Markdown JSON objects individually with Python’s JSONDecoder.raw_decode(), parsed the six application/ld+json scripts extracted from the original homepage HTML, then compared each pair of Python objects using structural equality (original == converted). That checks nested dictionaries, arrays and values; it ignores insignificant whitespace and JSON object-key formatting. All six comparisons returned True. This is a semantic-structure comparison, not a claim that original text bytes are identical.
Original JSON-LD scripts
Converted JSON-LD objects
Equal parsed structures
| Source script (HFCM ID) | Graph subject | Comparison |
|---|---|---|
| #32 | WebPage and FAQPage | Identical |
| #131 | Person identities and credentials | Identical |
| #85 | WebSite identity | Identical |
| #144 | LocalBusiness identity and relationships | Identical |
| #167 | Master services catalogue | Identical |
| #230 | GitHub and profile identity references | Identical |
Result: no changed values, omitted keys, altered @id values or lost nested relationships were found in this comparison. A separate UTF-8 file inspection confirmed that punctuation displayed incorrectly in one PowerShell session remained correct in the saved Markdown.
Machine-reader caveat: Cloudflare’s documented single-fence output is not itself a valid single JSON document when multiple JSON-LD scripts are concatenated. Software must parse each successive JSON object, rather than pass the entire fence to one json.loads() call. We observed precisely that failure and recovered all six objects with a streaming decoder.
This is evidence of structural fidelity in this test. It is not evidence that a particular model has interpreted the graph, trusted its claims, or recommended our business.
05 / What gave us pause
A successful conversion is not the end of the investigation.
Navigation still takes up space
Our converted Markdown retained a substantial menu and other interface text before the main content. On a large WordPress site, less HTML is not automatically the same as clean, concise information. We have not yet measured page-size or token savings against HTML.
Content Signals deserve a deliberate policy
Our converted response contained ai-train=yes, search=yes, ai-input=yes. Cloudflare documents this as the default when the origin does not supply its own Content-Signal header. We have not separately tested the origin policy here; deciding the intended permissions belongs in a dedicated investigation.
Monitoring may need review
Our AI Observatory distinguishes meaningful document retrievals using response types and other evidence. A text/markdown representation introduces a classification case that needs explicit review before we assume all successful requests are counted consistently. No Observatory code was modified for this test.
One response cannot settle reliability
We have separately observed intermittent HTTP failures during origin/security troubleshooting. This experiment documents a successful conversion, but does not establish failure rates or attribute unrelated intermittent responses to this feature.
Cache qualification: the returned Vary: Accept-Encoding,User-Agent, accept header included User-Agent. We have not established which layer added each entry or how every cache key is configured; the extra variation deserves inspection rather than an unsupported attribution to the origin. We have also not yet run the independent browser-format and non-homepage isolation checks.
Cloudflare’s October 9, 2026 update describes a streaming converter, a 6 MiB decompressed-HTML limit and removal of its earlier token-count response headers. Do not interpret their absence in our response as a failed test. See Cloudflare's October 2026 changelog.
06 / Commercial assessment
Which business should actually bother?
Worth evaluating
- Content-heavy sites with important public documentation or reference material.
- Organisations developing agents that intentionally request Markdown.
- Teams prepared to test content fidelity, caching, security and ongoing maintenance.
- Businesses with well-defined machine-readable content worth exposing accurately.
Not an automatic priority
- Sites whose main information is unclear, inaccessible or inconsistent in HTML already.
- Businesses expecting a switch to cause AI recommendations or citations.
- Sites with complex forms, authenticated workflows or transactions that require more than a text representation.
- Owners who have no evidence that their intended consumers prefer Markdown.
Cloudflare documents Markdown for Agents as available on Pro, Business and Enterprise plans and eligible SSL for SaaS customers, with no additional feature charge on supported plans. That is not the same as saying adoption has no cost: plan eligibility, testing, maintenance and operational monitoring all matter. See the current Cloudflare feature documentation for supported plans and limitations.
07 / The engineer's verdict
It works. That's the beginning of the evidence, not the conclusion of the sales pitch.
Our limited production deployment returned genuine Markdown through HTTP content negotiation. The response retained its metadata and headers, and all six JSON-LD objects were structurally identical to their originals. For a site built around a connected business identity, that is a significant and reassuring result.
Our decision is to keep the experiment tightly scoped while we assess practical value and operational effects, not assume that every page should be converted merely because the option exists.
At sea, you don't certify a ship because the engine starts once. You take her out and see what happens when the weather becomes unreasonable. Software is much the same, although usually with fewer wet socks.
What remains open: controlled HTML-versus-Markdown byte and token comparison; default HTML behaviour; non-homepage scope verification; content extraction quality; caching variance; Content Signals policy; and Observatory retrieval classification when a recognised machine requests Markdown.
Sources and test record
Documentation and evidence
- Cloudflare — Markdown for Agents: reference, output format, configuration and limitations.
- Cloudflare — October 9, 2026: streaming conversion update.
- Cloudflare — July 13, 2026: origin header and Content Signals preservation.
- Related SBW investigation: Agentic Browsing Readiness.
- Related SBW investigation: AI Crawler Monitoring at the Cloudflare Edge.
- SBW test artifacts: HTTP response header capture, homepage Markdown response and original HFCM JSON-LD export, collected October 28, 2026. The six JSON objects were compared locally as parsed Python structures. These files are not represented here as public third-party evidence.
Independent editorial note: This field report describes observed behaviour under the tested configuration and date. Cloudflare is not a sponsor of or affiliated with this investigation. Behaviour may change with subsequent platform releases. This page is not a claim of guaranteed AI visibility, inclusion or recommendation.