Frequently Asked Questions (FAQ) about Firecrawl MCP Server

Q: What is an MCP Server? A: MCP stands for Model Context Protocol. An MCP server acts as a bridge, allowing AI models to access and interact with external data sources and tools. The Firecrawl MCP Server specifically adds web scraping capabilities to LLMs.

Q: What are the key features of the Firecrawl MCP Server? A: The key features include web scraping, crawling, search, content extraction, deep research, batch scraping, automatic retries, rate limiting, cloud/self-hosted support, and SSE support.

Q: How do I install the Firecrawl MCP Server? A: You can install it using npx (directly from the command line), manually via npm, through VS Code integration, or via Smithery (legacy method).

Q: How do I configure the Firecrawl MCP Server for Cursor? A: Open Cursor Settings, go to Features > MCP Servers, click “+ Add new global MCP server”, and enter the provided JSON code block with your API key.

Q: What is the FIRECRAWL_API_KEY and where do I get it? A: FIRECRAWL_API_KEY is your unique identifier for using the Firecrawl service. You can create an account and obtain your API key from https://www.firecrawl.dev/app/api-keys.

Q: What are the different tools available in the Firecrawl MCP Server? A: The available tools include scrape, batch_scrape, map, search, crawl, extract, deep_research, and generate_llmstxt.

Q: When should I use the scrape tool vs. the batch_scrape tool? A: Use scrape when you need to extract content from a single, known URL. Use batch_scrape when you have multiple known URLs to scrape efficiently.

Q: What does the map tool do? A: The map tool discovers all indexed URLs on a website.

Q: How does the crawl tool differ from the map tool? A: The map tool only discovers URLs, while the crawl tool extracts content from multiple related pages on a website. Be cautious with crawl as responses can be large and exceed token limits.

Q: What is the purpose of the extract tool? A: The extract tool extracts structured information from web pages using LLM capabilities. You can define a schema for the structured data.

Q: What is the deep_research tool used for? A: The deep_research tool conducts deep web research on a query using intelligent crawling, search, and LLM analysis.

Q: What does the generate_llmstxt tool do? A: The generate_llmstxt tool generates a standardized llms.txt file for a given domain. This file defines how large language models should interact with the site.

Q: How do I configure retry behavior? A: You can configure retry behavior using environment variables such as FIRECRAWL_RETRY_MAX_ATTEMPTS, FIRECRAWL_RETRY_INITIAL_DELAY, FIRECRAWL_RETRY_MAX_DELAY, and FIRECRAWL_RETRY_BACKOFF_FACTOR.

Q: How do I monitor credit usage? A: You can monitor credit usage by setting the FIRECRAWL_CREDIT_WARNING_THRESHOLD and FIRECRAWL_CREDIT_CRITICAL_THRESHOLD environment variables.

Q: What kind of logging does the server provide? A: The server logs operation status, performance metrics, credit usage, rate limit tracking, and error conditions.

Q: What happens if I exceed a rate limit? A: The server provides automatic rate limit handling with exponential backoff. It will retry the request after a delay.

Q: Can I use the Firecrawl MCP Server with a self-hosted Firecrawl instance? A: Yes, you can use the server with a self-hosted instance by setting the FIRECRAWL_API_URL environment variable to your custom API endpoint.

Q: How does Firecrawl MCP Server integrate with UBOS? A: The Firecrawl MCP Server enhances UBOS by allowing AI Agents to access and process real-time information from the web. The different tools provide access to deep research, extraction of structured data, and crawling websites for content.