Media and publishing: how AI-ready are the leaders?
News and publishing sites whose content AI assistants summarise and cite. We ran the LLMready crawler against each homepage and scored it the same way as a free audit.
64/100
Median (all industries: 77)
0%
Publish an llms.txt
17%
Allow all 4 core AI crawlers
67%
Use Schema.org JSON-LD
Key findings
- Substack leads with 68/100 (D); The Guardian trails at 49.
- The media and publishing median is 13 points below the median of all 102 sites we measured.
- 5 of 6 block at least one core AI crawler (GPTBot, PerplexityBot, ClaudeBot or Google-Extended) in robots.txt.
- 0% publish an llms.txt and 0% an llms-full.txt.
- Weakest category on average: bot access at 38/100.
Bot access
38
Content extractability
69
Semantic metadata
76
Citation readiness
70
Scores
Leaderboard
| # | Site | Score | Bot access | Extractability | Metadata | Citation | Core bots | llms.txt | Biggest gap |
|---|---|---|---|---|---|---|---|---|---|
| 1 | 68 Substacksubstack.com | 68 D | 76 | 72 | 62 | 58 | llms.txt is present and valid | ||
| 2 | 67 TechCrunchtechcrunch.com | 67 D | 34 | 66 | 100 | 83 | 3 | llms.txt is present and valid | |
| 3 | 64 WIREDwired.com | 64 D | 34 | 71 | 91 | 71 | 3 | llms.txt is present and valid | |
| 4 | 63 The Vergetheverge.com | 63 D | 31 | 71 | 86 | 76 | 3 | llms.txt is present and valid | |
| 5 | 54 BBCbbc.com | 54 F | 19 | 63 | 76 | 69 | 4 | llms.txt is present and valid | |
| 6 | 49 The Guardiantheguardian.com | 49 F | 31 | 69 | 38 | 60 | 2 | llms.txt is present and valid |
Not scored because they turned away our crawler: The New York Times (blocked automated requests (HTTP 403)), Medium (blocked automated requests (HTTP 403)), Reuters (blocked automated requests (HTTP 401)), Forbes (blocked automated requests (HTTP 403)), The Economist (blocked automated requests (HTTP 403)), Bloomberg (blocked automated requests (HTTP 403)).
Most common gaps
Where media and publishing sites lose points
Answer-ready formats
100%6 of 6 sites · Citation readiness
Add an FAQ section (with FAQPage schema) and question-style headings that mirror real queries.
Authorship and freshness
100%6 of 6 sites · Citation readiness
Clean heading hierarchy
100%6 of 6 sites · Content extractability
Content density
100%6 of 6 sites · Content extractability
Trim inline scripts, styles and markup bloat so text makes up a larger share of the page.
llms-full.txt is available
100%6 of 6 sites · Bot access
llms.txt is present and valid
100%6 of 6 sites · Bot access
Generate an llms.txt below and upload it to your site root.
Where would your site rank?
Run the same audit on your site and compare it to the media and publishing median of 64.
A high score means AI engines can reach, read and understand a site. It doesn't guarantee citations — those also depend on authority, relevance and the question being asked.