Nowledge MCP Server

Nowledge MCP Server is a specialized web‑scraping tool that bridges the gap between raw online documentation and AI assistants. By accepting a public URL through the Model Context Protocol, it crawls all relevant pages, cleans and normalizes the HTML, converts the content into clean Markdown, and returns either a single aggregated document or a structured list of page‑level Markdown snippets. This eliminates the need for developers to manually fetch, parse, or clean documentation when building knowledge‑based applications.

The server addresses a common pain point for AI‑powered developers: the difficulty of reliably ingesting web content that is often cluttered with navigation bars, ads, and dynamic scripts. Nowledge removes these distractions automatically, rewrites internal links so they resolve correctly within Markdown, and offers configurable crawling depth and concurrency. The result is a consistent, searchable text corpus that can be fed directly into language models for contextual understanding or summarization.

Key capabilities include:

• Web‑wide fetching of any public URL, enabling rapid onboarding of new libraries or frameworks.
• HTML sanitization that strips non‑essential elements, leaving only the core prose and code examples.
• Link rewriting to preserve navigation within the generated Markdown, making it easier for models to follow context.
• Dual output modes: an aggregate mode that stitches all pages into one document, or a pages mode that returns each page as an individual Markdown block with its path.
• NLP‑friendly metadata such as total pages, byte size, and elapsed time, which help models estimate processing costs.

In real‑world scenarios, Nowledge is invaluable for building AI assistants that answer “how do I use X?” or “what are the best practices for Y?” type queries. For example, a developer can prompt an assistant with , and the assistant receives a clean, structured Markdown summary of the relevant documentation. The tool also supports quick lookups by repository shortform () and can be integrated into CI pipelines to keep internal knowledge bases up‑to‑date automatically.

Integration is straightforward: the server registers a tool that any MCP‑compatible client can invoke. The tool accepts parameters for URL, output mode, and maximum crawl depth, returning a predictable JSON payload that includes status, data, and progress events. This seamless workflow allows developers to embed web‑scraped knowledge directly into conversational agents, search engines, or documentation generators without writing custom crawlers or parsers.