CrawlYield

For publishers on Cloudflare Pay Per Crawl

One flat price for every AI crawl
leaves revenue on the table.

Demand, crawler identity, and content type all move the fair price of a request. CrawlYield sets that price per crawl — inside Cloudflare’s native Pay Per Crawl — and measures the difference against your static rate before anything changes in production.

$200/mo membership 10% of measured incremental yield Shadow mode first
Start membership · $200/mo Estimate your yield gap

How it works

Measure first. Then price each crawl on its merits.

CrawlYield runs on your existing Cloudflare zone and emits Cloudflare’s native Crawler-Price header. Your floor and ceiling stay yours. Settlement stays with Pay Per Crawl.

01

Start membership

Your member portal opens the moment checkout completes: a guided setup path, a yield estimate for your zone, and a direct line to Boson IT engineering.

02

Run in shadow mode

Once Pay Per Crawl is active on your account, CrawlYield prices every request in the background while crawlers keep paying your static rate. You see the gap on real traffic, with no live change.

03

Go live when you are ready

Flip dynamic pricing on. We earn 10% of the incremental yield above your baseline — never a share of the baseline itself.

Estimate

What could a flat rate be costing you?

Enter your monthly AI crawl volume, your static per-request price, and an expected lift. Everything runs in your browser. The result is an estimate to size the opportunity — shadow mode replaces it with your measured number.

Per-request prices of 0.1–0.5¢ are a commonly discussed range for Pay Per Crawl. Lift is your assumption, not a guarantee. CrawlYield earns 10% of the incremental gap only; membership is $200/mo.

Your estimate

Flat baseline $150
Dynamic revenue $195
Yield gap (incremental) $45
CrawlYield take (10%) $4.50
Platform $200

Bar: flat baseline · your share of the gap · CrawlYield 10%

After take, you keep $40.50 of the gap. Subtract the $200 platform and the monthly net versus staying flat is −$159.50.

Evidence

The market context, from Cloudflare’s own data.

Industry figures and worked examples that size the opportunity. They are published averages, not CrawlYield customer results, and not a guarantee of lift.

Cloudflare Radar · 2025

~4%

AI bots averaged about 4.2% of HTML requests in 2025 (range 2.4–6.4%). A small slice of traffic — and a large slice of unpaid extraction.

Source: Cloudflare Radar Year in Review

Typical PPC range

0.1–0.5¢

Pay Per Crawl prices operators discuss in public commonly sit in tenths of a cent. A static rate is easy to set and easy to get wrong across hubs, archives, and stubs.

Commonly discussed market range; not a quote.

Cloudflare Radar · crawl-to-refer

100s : 1

Training-heavy bots often crawl hundreds to thousands of pages for every referral click. Anthropic’s July 2025 ratio was still tens of thousands to one; OpenAI sat around a thousand to one.

Source: Cloudflare: the crawl-to-click gap

Worked example

150k crawls × 0.1¢ = $150 flat. A 30% lift adds $45.

Same rate, same lift, three volumes. The gap scales with volume. Membership stays $200; CrawlYield’s share stays 10% of the gap and never touches the baseline.

Assumes a 0.1¢ static price and ~30% lift. Worked example, not a specific publisher.
AI crawls / mo Flat (0.1¢) Dynamic (~30%) Gap CrawlYield 10%
50,000 $50 $65 $15 $1.50
150,000 $150 $195 $45 $4.50
500,000 $500 $650 $150 $15.00

50,000 crawls / mo

Flat at 0.1¢
$50
Dynamic ~30%
$65
Gap
$15
CrawlYield 10%
$1.50

150,000 crawls / mo

Flat at 0.1¢
$150
Dynamic ~30%
$195
Gap
$45
CrawlYield 10%
$4.50

500,000 crawls / mo

Flat at 0.1¢
$500
Dynamic ~30%
$650
Gap
$150
CrawlYield 10%
$15.00

Shadow mode

See your own number before anything changes.

Shadow mode prices every crawl in the background while your static rate stays in force. You compare the two side by side on your real traffic, then decide when to go live. Nothing changes in production until you say so.

  • Membership opens your portal and setup path today.
  • Shadow mode starts once Pay Per Crawl is active on your Cloudflare account.
  • You control the switch to live pricing.
Start membership · $200/mo Pay Per Crawl setup guide

Pricing

$200 a month. 10% of what CrawlYield adds.

Membership opens your portal immediately. Dynamic pricing on your zone begins when Pay Per Crawl is active on your Cloudflare account; the 10% share applies only to measured yield above your static baseline.

Membership · available now

$200 / month

Everything you need to prepare, measure, and go live.

  • Guided setup path with progress saved to your account
  • Yield estimate for your zone and baseline
  • Shadow-mode measurement and live connect with Boson IT engineering
  • Self-serve billing, invoices, and cancellation

Start membership · $200/mo

Performance share · after measurement

10% of incremental yield

Charged only on revenue above the static baseline you already earn.

  • Your baseline is never included
  • Based on measured live traffic
  • Nothing due at signup

Optional analytics add-on: $99/mo. Crawler payments settle through Cloudflare Pay Per Crawl and are separate from CrawlYield membership.

Stay informed

Not ready yet? Follow CrawlYield.

Membership is open today. If you would rather hear about product updates first, leave a work email and we will keep you posted.

Start membership · $200/mo Sign in