Question 1

What is the Puppeteer vision MCP Server?

Accepted Answer

The Puppeteer vision MCP Server is a tool for scraping webpages and converting them to markdown format using Puppeteer, Readability, and Turndown. It features AI-driven interaction capabilities to handle cookies, captchas, and other interactive elements automatically.

Question 2

What is MCP?

Accepted Answer

MCP stands for Model Context Protocol. It is an open protocol that standardizes how applications provide context to Large Language Models (LLMs). The MCP server acts as a bridge, allowing AI models to access and interact with external data sources and tools.

Question 3

How does the AI-driven interaction work?

Accepted Answer

The system uses vision-capable AI models to analyze screenshots of web pages and decide on actions like clicking, typing, or scrolling to bypass overlays and consent forms. This process repeats up to `maxInteractionAttempts`.

Question 4

What environment variables are required to run the server?

Accepted Answer

The required environment variable is `OPENAI_API_KEY`. Optional variables include `VISION_MODEL`, `API_BASE_URL`, `USE_SSE`, `PORT`, and `DISABLE_HEADLESS`.

Question 5

What are the communication modes supported by the server?

Accepted Answer

The server supports two communication modes: stdio (default) and SSE mode. Stdio communicates via standard input/output, while SSE mode communicates via Server-Sent Events over HTTP.

Question 6

How do I use the `scrape-webpage` tool?

Accepted Answer

The `scrape-webpage` tool requires the `url` parameter (the URL of the webpage to scrape). Optional parameters include `autoInteract`, `maxInteractionAttempts`, and `waitForNetworkIdle`.

Question 7

What is the response format of the `scrape-webpage` tool?

Accepted Answer

The tool returns its result in a structured format with `content` (an array containing a single text object with the raw markdown) and `metadata` (status message, success boolean, and content size).

Question 8

Can I run the browser in visible mode?

Accepted Answer

Yes, you can set the `DISABLE_HEADLESS` environment variable to `true` to run the browser in visible mode.

Question 9

How do I install the server for development?

Accepted Answer

Clone the repository, install dependencies using `npm install`, build the project using `npm run build`, set up your environment variables, and run the server using `npm start` or `npm run dev`.

Question 10

What is UBOS, and how does the MCP Server integrate with it?

Accepted Answer

UBOS is a full-stack AI Agent Development Platform focused on bringing AI Agents to every business department. The Puppeteer vision MCP Server can be integrated as an asset within the UBOS platform, allowing AI Agents to access and scrape web content seamlessly.

Question 11

Where can I find the Puppeteer vision MCP Server on UBOS?

Accepted Answer

You can find it on the UBOS Asset Marketplace.

Question 12

Can I customize the behavior of the scraper?

Accepted Answer

Yes, you can modify the behavior of the scraper by editing files such as `src/ai/vision-analyzer.ts`, `src/ai/page-interactions.ts`, `src/scrapers/webpage-scraper.ts`, and `src/utils/markdown-formatters.ts`.

Puppeteer Vision Web Scraper

Resources

Project Details

Recomended MCP Servers

Featured Templates

Your Speaking Avatar

Python Bug Fixer

Pharmacy Admin Panel

Image to text with Claude 3

AI Chatbot Starter Kit v0.1

AI-Powered Product List Manager

Start your free trial