Development

A network lets sites block agent and training crawlers by default

development2026-07-02

Claim

Cloudflare's Content Independence Day lets sites block Agent and Training crawlers by default on ad-supported pages from September, while NotebookLM compresses sources into 60-second vertical videos.

Reporting date: 2026-07-02 (not necessarily the event date)

Sources

Attribution and reporting linked by this record; links do not imply independent verification.

Welcome to July 2, 2026 (theinnermostloop.substack.com)

Knowledge graphopen in browser →

Reported in

Welcome to July 2, 2026

Actor

Evidence for

Supersedes

A streaming service cuts royalties for fully machine-made songs

Referenced by

Welcome to July 2, 2026

Cite or correct this record

Citation formatCitation text

A network lets sites block agent and training crawlers by default
Innermost Loop KB — secondary record
Reporting date: 2026-07-02
KB permalink: https://nicholsn.github.io/innermost-loop-kb/developments/2026-07-02-crawlers-blocked-by-default
Claim: Cloudflare's Content Independence Day lets sites block Agent and Training crawlers by default on ad-supported pages from September, while NotebookLM compresses sources into 60-second vertical videos.
Attributed sources:
- Welcome to July 2, 2026 — https://theinnermostloop.substack.com/p/welcome-to-july-2-2026
This citation does not imply independent verification of linked-source claims.

Suggest a correction — opens a GitHub draft for you to review and submit.