API Overview

Welcome to Lumanu, an AI-first web content extraction utility designed specifically for large language model applications. Most public websites are built for visual layout presentation, introducing excessive noise like cookies, layout grids, advertisement pixels, scripting trackers, and presentation styling. When loaded into prompt workflows, these elements lead to severe token bloat and dilute context accuracy.

Lumanu runs high-performance extraction microservices directly at the edge to fetch web resources, strip out layout noise, cleanly serialize structural tables, and instantly deliver clean plain-text, JSON, YAML, or Markdown content blocks.

Self-Hosting First: Lumanu is built to run easily on a single-file template configuration, allowing direct paste-in integration into a Cloudflare Workers dashboard without a local node environment or terminal build steps.

Quickstart with cURL

Use the global gateway to fetch and parse any remote webpage from your local developer terminal in seconds:

                        bash
                    
$ curl "https://lumanu.mcbcode.com/extract?url=https://example.com"

By default, this endpoint generates a structured JSON response containing the success status, metadata details, and the cleaned output.

GET /extract

Retrieves and strips down text from a single webpage using simple URL query variables.

Query Parameter	Type	Requirement	Description
`url`	String	Required	Target webpage address to extract (e.g., `https://example.com`). Must include valid `http:` or `https:` protocol prefixes.
`format`	String	Optional	Specifies structural representation. Options: `json`, `text`, `markdown`, `yaml`. Defaults to `json`.

                        Example Request
                    
$ curl "https://lumanu.mcbcode.com/extract?url=https://example.com&format=markdown"

POST /extract

Allows bulk operations, letting you extract content from up to 100 URLs in parallel within a single request.

JSON Property	Type	Requirement	Description
`url`	String	Optional	Single target URL. (Takes precedence if `urls` array is not supplied).
`urls`	Array	Optional	Array of web targets (e.g., `["https://site-a.com", "https://site-b.com"]`). Max 100 targets.
`format`	String	Optional	Outputs formatting scheme: `json`, `text`, `markdown`, `yaml`. Defaults to `json`.

                        Example Payload
                    
{
  "urls": [
    "https://example.com/doc-1",
    "https://example.com/doc-2"
  ],
  "format": "markdown"
}

GET /health

A simple check endpoint used to monitor the status and uptime of the deployed Lumanu Worker instance.

                        json
                    
{
  "status": "ok"
}

Format Serializers

Lumanu dynamically changes output formatting based on your requested parameter. Check out the specific schemas below:

JSON

Returns structured metadata alongside the extracted clean text.

                        json
                    
{
  "success": true,
  "count": 1,
  "results": [
    {
      "url": "https://example.com",
      "status": 200,
      "success": true,
      "contentType": "text/html; charset=UTF-8",
      "text": "Example content text block...",
      "length": 1829
    }
  ]
}

Plain Text

Raw text outputs separated by explicit URL boundaries.

                        text
                    
===== https://example.com =====

Example page header content.
Extracted main body layout cleanly.

Markdown

Clean structures organized under Markdown headers (##) for straightforward integration into LLM prompts.

                        markdown
                    
## https://example.com

Example page header content.
Extracted main body layout cleanly.

YAML

Serialized cleanly using a dependency-free nested block scalar parser.

                        yaml
                    
success: true
count: 1
results:
  - url: "https://example.com"
    status: 200
    success: true
    contentType: "text/html; charset=UTF-8"
    text: |
      Example page header content.
      Extracted main body layout cleanly.

Security & SSRF Protection

To safely run an open, self-hosted proxy web parser, Lumanu includes built-in safeguards to prevent Server-Side Request Forgery (SSRF) and private network data leaks.

Blocking Private Networks

Lumanu blocks requests targeting local loopbacks and internal subnets. The engine verifies hostname definitions and blocks them if they match standard local scopes:

Loopbacks: localhost, 127.0.0.1, 0.0.0.0, ::1
IPv4 Private Blocks: 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16
IPv6 Link-Local Blocks: fc00::, fd00::, fe80::

Redirect Policy

To avoid security bypasses via redirect chains, Lumanu uses redirect: "manual". Redirect responses (HTTP 3xx) are treated as explicit errors and are not followed.

The Extraction Engine

The parser runs a sequence of optimizations on fetched web pages to clean up messy HTML before returning the output:

1. Removing Boilerplate Tags

Strips utility markup entirely, including the elements and any nested child contents inside:

script, style, noscript, nav, header, footer, iframe, svg, form

2. Attribute Matching (Ads & Cookie Banners)

Matches class and ID names against common patterns to strip typical boilerplate elements, including:

Cookie/consent banners (e.g., cookie-consent)
Ad wrapper containers (e.g., ad-banner)
Popup modals and sign-up prompts

3. Table-to-Markdown Conversions

Converts HTML table grids into clean, pipe-delimited text blocks that preserve data relationships for AI logic engines:

                        Output Preview
                    
Row Header 1 | Row Header 2
Data Column 1 | Data Column 2

4. Block Element Spacing

Inserts clean break formatting around structural element boundaries (like <p>, <div>, <li>, <h1-h6>, and <blockquote>) to maintain readable vertical spacing.

System Limits & Configuration

The core configuration values hardcoded in the Worker runtime include:

Setting	Value	Purpose
`MAX_URLS`	`100`	Maximum URLs allowed per batch request.
`MAX_DOWNLOAD_BYTES`	`5,242,880` (5 MB)	Max network stream size read per request.
`TIMEOUT_MS`	`30000` (30 seconds)	Connection abort window for slow web targets.
`USER_AGENT`	`Lumanu/1.0 (...)`	Bot header identification sent to origin servers.

Cloudflare Worker Deployment

Deploy Lumanu easily using Wrangler or straight through the Cloudflare web dashboard.

Option A: Quick Edit (No Terminal)

1. Log in to your Cloudflare Dashboard, navigate to Workers & Pages, and click Create Application.

2. Choose Create Worker, name it lumanu, and click Deploy.

3. Click Edit Code, paste your single-file Lumanu script into the dashboard editor, and click Save and Deploy.

Option B: Command Line (Wrangler)

Deploy using Wrangler from your local workspace terminal:

                        bash
                    
# Initialize new wrangler layout context
$ npm install -g wrangler
$ wrangler login

# Deploy the script to your active account edge
$ wrangler deploy app.js --name lumanu

Previous --- Next ---