Docy: Unlock Documentation for Smarter AI Agents with UBOS

In the rapidly evolving landscape of AI and Large Language Models (LLMs), access to accurate and up-to-date information is paramount. AI agents, especially those designed for coding assistance and technical problem-solving, heavily rely on comprehensive documentation to provide relevant and reliable guidance. However, integrating and managing documentation across diverse tech stacks can be a complex and resource-intensive task.

This is where Docy, an innovative Model Context Protocol (MCP) server, steps in. Docy provides AI agents with seamless and real-time access to the technical documentation they need to function effectively. Integrated with the UBOS AI Agent Development Platform, Docy enhances the capabilities of AI agents, enabling them to deliver more precise, context-aware, and actionable insights.

The Challenge: Documentation Access for AI Agents

AI agents operating in technical domains face several challenges related to documentation:

• Outdated Information: Documentation can quickly become outdated, leading to inaccurate or irrelevant suggestions from AI agents.
• Broken Links: Broken links in documentation can disrupt the information flow and hinder the agent’s ability to gather necessary context.
• Rate Limits: Accessing documentation through traditional web scraping methods can be subject to rate limits, restricting the agent’s ability to retrieve information efficiently.
• Security Concerns: Allowing AI agents to access external documentation sources can raise security concerns, particularly when dealing with sensitive information.

Docy addresses these challenges by providing a secure, efficient, and real-time solution for documentation access.

Docy: Bridging the Gap Between AI and Documentation

Docy acts as a bridge, providing AI models with a standardized way to access and interact with external data sources, specifically documentation websites. By leveraging the Model Context Protocol (MCP), Docy enables AI agents to:

• Search and Retrieve Content: Agents can search for specific information within documentation websites using keywords or queries.
• Navigate Documentation Structures: Agents can navigate through documentation hierarchies, following links and exploring related topics.
• Access Real-Time Information: Docy ensures that agents always have access to the latest version of the documentation, eliminating the risk of outdated information.

Built with FastMCP v2, Docy is designed for performance and scalability, ensuring that AI agents can access documentation quickly and efficiently.

Key Features and Benefits of Docy

Docy offers a range of features and benefits that make it an essential tool for AI agent development:

• Instant Documentation Access: Docy provides direct access to documentation from popular tech stacks like React, Python, and crawl4ai, as well as any other technology for which documentation is available.
• Hot-Reload Support: New documentation sources can be added on-the-fly without requiring a server restart. Simply edit the .docy.urls file to include new documentation URLs.
• Intelligent Caching: Docy employs intelligent caching mechanisms to reduce latency and minimize external requests while ensuring that the content remains fresh.
• Self-Hosted Control: Docy allows you to keep your documentation access within your security perimeter, providing control over data access and security.
• Seamless MCP Integration: Docy integrates seamlessly with Claude, VS Code, and other MCP-enabled AI tools, ensuring a smooth and efficient workflow.
• Crawl4ai Integration: Enables crawling and scraping of documentation websites to keep the information current.

Use Cases: How Docy Empowers AI Agents

Docy can be applied in a wide range of use cases to enhance the capabilities of AI agents:

• Coding Assistance: AI agents can use Docy to verify implementation details against official documentation, explore API usage patterns, and compare implementation options.
• Technical Support: AI-powered chatbots can leverage Docy to provide accurate and up-to-date answers to technical questions, resolving customer issues more efficiently.
• Documentation Generation: AI agents can use Docy to analyze existing documentation and automatically generate new documentation for software projects.
• Knowledge Management: Docy can be used to create a centralized knowledge base for AI agents, providing them with access to all relevant documentation in a single location.

Examples of Docy in Action

Here are a few examples of how Docy can be used to solve common documentation tasks:

• Verifying Implementation:

  • Scenario: An AI agent is assisting a developer with implementing a feature using the Crawl4Ai library.
  • Docy’s Role: The agent can use Docy to access the Crawl4Ai documentation and verify that the implementation is correct.
  • Prompt Example: “Are we implementing Crawl4Ai scrape results correctly? Let’s check the documentation using Docy.”

• Exploring API Usage:

  • Scenario: A developer needs to understand how to use a specific function within the mcp.tool API.
  • Docy’s Role: The agent can use Docy to search the documentation for examples of how to use the mcp.tool function.
  • Prompt Example: “What do the docs say about using mcp.tool? Show me examples from the documentation using Docy.”

• Comparing Implementation Options:

  • Scenario: A developer is deciding how to structure data in a React application.
  • Docy’s Role: The agent can use Docy to access the React documentation and identify the best practices for data structuring.
  • Prompt Example: “How should we structure our data according to the React documentation using Docy? What are the best practices?”

Integration with UBOS: A Powerful Combination

Docy integrates seamlessly with the UBOS AI Agent Development Platform, providing a comprehensive solution for building and deploying AI agents that require access to technical documentation. UBOS offers a range of features that complement Docy’s capabilities, including:

• AI Agent Orchestration: UBOS provides tools for managing and orchestrating AI agents, ensuring that they function efficiently and effectively.
• Enterprise Data Connectivity: UBOS allows AI agents to connect to enterprise data sources, providing them with access to the information they need to make informed decisions.
• Custom AI Agent Development: UBOS provides a platform for building custom AI agents tailored to specific business needs.
• Multi-Agent Systems: UBOS supports the development of multi-agent systems, allowing multiple AI agents to collaborate and solve complex problems.

By combining Docy with UBOS, organizations can create AI agents that are not only intelligent but also well-informed and capable of providing accurate and reliable guidance.

Getting Started with Docy

Integrating Docy into your AI agent development workflow is straightforward. The following steps outline the basic installation and configuration process:

1. Installation: Docy can be installed using various methods, including uv, pip, and Docker.
2. Configuration: Configure Docy by specifying the URLs of the documentation websites you want to include. This can be done using environment variables or a .docy.urls file.
3. Integration: Integrate Docy with your AI agent development platform, such as Claude or VS Code.

Detailed instructions for each of these steps can be found in the Docy documentation.

Installation Options

Docy offers multiple installation methods to suit different development environments:

• Using uv (Recommended): When using uv, no specific installation is required. Use uvx to directly run mcp-server-docy.

• Using PIP: Install mcp-server-docy via pip:

  bash pip install mcp-server-docy

  After installation, run it as a script:

  bash DOCY_DOCUMENTATION_URLS=“https://docs.crawl4ai.com/,https://react.dev/” python -m mcp_server_docy

• Using Docker: Use the Docker image:

  bash docker pull oborchers/mcp-server-docy:latest docker run -i --rm -e DOCY_DOCUMENTATION_URLS=“https://docs.crawl4ai.com/,https://react.dev/” oborchers/mcp-server-docy

Global Server Setup

For team or multi-project development, refer to the server/README.md for instructions on running a persistent SSE server. This setup allows you to maintain a single Docy instance with shared documentation URLs and cache.

Configuration Details

Docy offers flexible configuration options to adapt to various environments and needs.

Configuration Options

The application can be configured using environment variables. Here’s a detailed look at some of the key settings:

• DOCY_DOCUMENTATION_URLS (string): A comma-separated list of URLs pointing to documentation sites. This is crucial for directing Docy to the relevant documentation sources. For example: "https://docs.crawl4ai.com/,https://react.dev/".
• DOCY_DOCUMENTATION_URLS_FILE (string): Specifies the path to a file containing documentation URLs, with one URL per line. Default: .docy.urls.
• DOCY_CACHE_TTL (integer): Sets the cache time-to-live in seconds. This determines how long the cached documentation remains valid before being refreshed. Default: 432000.
• DOCY_CACHE_DIRECTORY (string): Defines the path to the cache directory where documentation content is stored. Default: .docy.cache.
• DOCY_USER_AGENT (string): Allows you to set a custom User-Agent string for HTTP requests, which can be useful for identifying Docy’s requests on the server-side.
• DOCY_DEBUG (boolean): Enables debug logging when set to "true", "1", "yes", or "y". Debug logging provides detailed information about Docy’s operations, helping to diagnose issues during development.
• DOCY_SKIP_CRAWL4AI_SETUP (boolean): When set to "true", "1", "yes", or "y", Docy skips running the crawl4ai-setup command at startup. This can be useful if you have already set up Crawl4ai or do not need it for your use case.
• DOCY_TRANSPORT (string): Determines the transport protocol to use for communication. Options include "sse" (Server-Sent Events) or "stdio" (standard input/output). Default: "stdio".
• DOCY_HOST (string): Defines the host address to bind the server to. Default: "127.0.0.1".
• DOCY_PORT (integer): Specifies the port number to run the server on. Default: 8000.

Environment variables can be set directly in your system or managed via a .env file for convenience.

URL Configuration File

Instead of setting the DOCY_DOCUMENTATION_URLS environment variable, you can create a .docy.urls file in your project directory, listing one URL per line. This approach is particularly beneficial in collaborative environments or when version-controlling documentation sources:

text https://docs.crawl4ai.com/ https://react.dev/

Lines starting with # are treated as comments

https://docs.python.org/3/

The server checks for URLs in the DOCY_DOCUMENTATION_URLS environment variable first. If none are found, it defaults to looking for the .docy.urls file.

Hot Reload for URL File

Docy implements a hot-reload mechanism for the .docy.urls file. The server reads the file on each request rather than caching the URLs. This allows you to:

1. Add, remove, or modify documentation URLs in the .docy.urls file while the server is running.
2. See those changes immediately reflected in subsequent calls to list_documentation_sources_tool or other documentation tools.
3. Avoid restarting the server when modifying your documentation sources.

This feature is especially useful during development, providing immediate feedback and improving workflow efficiency.

Documentation URL Best Practices

When configuring documentation URLs, it’s important to point them to documentation index or introduction pages. These pages should ideally contain:

• Tables of contents
• Navigation structures
• Collections of internal and external links

This allows the LLM to:

1. Start at a high-level documentation page.
2. Discover relevant subpages via links.
3. Navigate to specific documentation as needed.

Using documentation sites with well-structured subpages is highly recommended as it minimizes context usage, improves navigation efficiency, and reduces the need to load entire documentation sets at once.

Caching Behavior

Docy automatically caches documentation content to improve performance. Key aspects of the caching mechanism include:

• At startup, the server pre-fetches and caches all configured documentation URLs from DOCY_DOCUMENTATION_URLS.
• The cache time-to-live (TTL) can be configured via the DOCY_CACHE_TTL environment variable.
• Each new site accessed is automatically loaded into the cache to reduce traffic and improve response times.
• Cached content is stored in a persistent disk-based cache using the diskcache library.
• The cache location can be configured via the DOCY_CACHE_DIRECTORY environment variable (default: .docy.cache).
• The cache persists between server restarts, providing better performance for frequently accessed documentation.

Exceptions to Caching

While most content is cached for performance, there are specific exceptions:

• Documentation URL Lists: When using the .docy.urls file, the list of documentation sources is never cached. Instead, the file is re-read on each request to support hot-reloading of URLs.
• Page Content: The actual content of documentation pages is still cached according to the configured TTL.

This hybrid approach offers both performance benefits for content access and flexibility for documentation source management.

Conclusion: Empowering AI with Documentation

Docy, integrated with the UBOS platform, represents a significant advancement in the way AI agents access and utilize technical documentation. By providing a secure, efficient, and real-time solution for documentation access, Docy empowers AI agents to deliver more accurate, context-aware, and actionable insights. Whether you’re building AI-powered coding assistants, technical support chatbots, or knowledge management systems, Docy can help you unlock the full potential of your AI agents.