What is MCP Go Colly?

MCP Go Colly is a web crawling framework that integrates the Model Context Protocol (MCP) with the Colly web scraping library. It provides a flexible and extensible solution for extracting web content for large language model (LLM) applications.

What is MCP?

MCP (Model Context Protocol) is an open protocol that standardizes how applications provide context to LLMs. It enables AI models to access and interact with external data sources and tools.

What are the key features of MCP Go Colly?

Key features include concurrent web crawling, configurable depth and domain restrictions, MCP server integration, graceful shutdown handling, robust error handling, and support for single and batch URL crawling.

How do I install MCP Go Colly?

To install MCP Go Colly, clone the repository, install the dependencies using make deps, and build the binary using make build.

How do I use MCP Go Colly as an MCP tool?

You can use MCP Go Colly as an MCP tool by calling it with the appropriate parameters, such as the URLs to crawl and the maximum crawl depth. Refer to the documentation for the specific JSON format.

What configuration options are available?

Configuration options include max_depth to set the maximum crawl depth and urls to specify the URLs to crawl. Domain restrictions are automatically applied based on the provided URLs.

How can I contribute to the project?

To contribute to the project, fork the repository, create a feature branch, commit your changes, push to the branch, and create a pull request.

Where can I find the claude_desktop_config.json file?

The location of the claude_desktop_config.json file depends on your Claude installation. Refer to the Claude documentation for the specific location on your operating system.

What is UBOS?

UBOS is a full-stack AI Agent Development Platform focused on bringing AI Agents to every business department. It helps you orchestrate AI Agents, connect them with your enterprise data, and build custom AI Agents with your LLM model and Multi-Agent Systems.