Web Scrapper

HTTPv1.0.0No CredentialJan 22, 2026

Documentation

Guide

Extract content from web pages using CSS selectors.

A Model Context Protocol (MCP) server that exposes web scraping capabilities for extracting structured data from HTML pages.

Overview

The Web Scraper MCP Server provides stateless, multi-user access to web scraping operations:

Web Page Scraping — Extract content from web pages using URLs and optional CSS selectors
Flexible Content Extraction — Target specific elements or retrieve full page content
Structured Data Output — Get parsed HTML, metadata, and extracted data in organized format

Perfect for:

Automated data collection from websites
Building web intelligence and monitoring systems
Integrating web content into AI workflows and pipelines

Tools

Scrape content from a web page given a URL and optional CSS selector. Returns either targeted element data or full page metadata including title, description, and HTML.

Inputs:

url (string, required) — The web page URL to scrape
selector (string, optional) — CSS selector to target specific elements (default: null)

Output:

{
  "result": {
    "url": "https://example.com",
    "data": ["Element 1", "Element 2", "Element 3"]
  }
}

Or without selector:

{
  "result": {
    "url": "https://example.com",
    "title": "Page Title",
    "description": "Page description",
    "html": "<html>...</html>"
  }
}

Usage Example:

POST /mcp/cl-web-scraper/scrape
 
{
  "url": "https://example.com/articles",
  "selector": ".article-title"
}

Or without CSS selector:

POST /mcp/cl-web-scraper/scrape
 
{
  "url": "https://example.com"
}

Reference & Support

CSS Selectors

The selector parameter uses standard CSS selector syntax:

.classname — Select by class
#idname — Select by ID
div > p — Select direct children
div p — Select descendants
[attribute="value"] — Select by attribute
:nth-child(n) — Select by position
.class1.class2 — Select multiple classes

Output Formats

With CSS Selector:

The Web Scraper MCP Server does not require authentication for basic web scraping operations. All tools work without API keys or OAuth tokens.

However, some websites may have:

Rate Limiting — Requests may be throttled if too many are made in rapid succession
Robots.txt Restrictions — Some sites may disallow scraping in their robots.txt file
Terms of Service — Always check the website's ToS before scraping

For websites that require authentication or have specific access controls, you may need to handle authentication separately.

Invalid URL Format

Cause: URL is malformed or inaccessible
Solution:
1. Verify URL starts with http:// or https://
2. Check the URL is correct and the website is accessible
3. Test URL in a web browser first

Empty or No Data Returned

Cause: CSS selector doesn't match any elements on the page
Solution:
1. Verify CSS selector syntax is correct
2. Inspect the web page to find correct selector
3. Try scraping without selector to see full page content
4. Check if page content is dynamically loaded (JavaScript-rendered)

Malformed Request Payload

Cause: JSON payload is invalid or missing required fields
Solution:
1. Validate JSON syntax before sending
2. Ensure url parameter is present (required)
3. Check parameter types match expected values (strings)

Server Not Found

Cause: Incorrect server name in the API endpoint

Connect

Get connection URL

https://gateway.mewcp.com/web-scraper/mcp

Request Builder

import asyncio
from fastmcp import Client

# Real MewCP gateway URL for this server
GATEWAY_URL = "https://gateway.mewcp.com/web-scraper/mcp"
API_KEY = "YOUR_API_KEY"

headers={
    "Authorization": f"Bearer {API_KEY}"
}

async def main():
    
    client = Client(GATEWAY_URL, headers=headers)
    
    async with client:
        # Test the connection by pinging the server
        await client.ping()
        
        # List available operations
        tools = await client.list_tools()
        resources = await client.list_resources()
        prompts = await client.list_prompts()

        # Execute operations
        result = await client.call_tool("example_tool", {"param": "value"})
        print(result)

asyncio.run(main())

Web Scrapper

Documentation

Overview

Tools

Reference & Support

CSS Selectors

Output Formats

Invalid URL Format

Empty or No Data Returned

Malformed Request Payload

Server Not Found

Overview

Tools

Reference & Support

CSS Selectors

Output Formats

Invalid URL Format

Empty or No Data Returned

Malformed Request Payload

Server Not Found

Rate Limit or Access Denied

Insufficient Credits