---
title: Glossary · Cloudflare AI Crawl Control docs
description: Review the definitions for terms used across Cloudflare's AI Crawl
  Control documentation.
lastUpdated: 2025-08-27T16:00:59.000Z
chatbotDeprioritize: false
source_url:
  html: https://developers.cloudflare.com/ai-crawl-control/reference/glossary/
  md: https://developers.cloudflare.com/ai-crawl-control/reference/glossary/index.md
---

Review the definitions for terms used across Cloudflare's AI Crawl Control documentation.

| Term | Definition |
| - | - |
| AI crawler | A bot which scrapes content from websites in support of an AI model, including by scraping content for indexing, retrieval augmented generation, or training. |
| Merchant of Record | The entity who facilitates "buying and selling". For pay per crawl, Cloudflare is the merchant of record. |
| robots.txt | A text file which lists pages in your website that are off-limits for bots. Well-behaved bots respect this file, but some bots may violate it. You can enforce `robots.txt` with Cloudflare WAF custom rules. |
