Make Your Website Conversational for People and Agents with NLWeb and AutoRAG
2025-08-28
With NLWeb, an open project by Microsoft, and Cloudflare AutoRAG, conversational search is now a one-click setup for your website....
Nelson Duarte
Posts by Nelson Duarte
Robotcop: enforcing your robots.txt policies and stopping bots before they reach your website
2024-12-10
The AI Crawl Control (formerly AI Audit) now allows you to quickly see which AI services are honoring your robots.txt policies and then automatically enforce the policies against those that aren’t. ...
Billions and billions (of logs): scaling AI Gateway with the Cloudflare Developer Platform
2024-10-24
How we scaled AI Gateway to handle and store billions of requests, using Cloudflare Workers, D1, Durable Objects, and R2....