diff --git a/README.md b/README.md index 2e4166a..5eb5ff9 100644 --- a/README.md +++ b/README.md @@ -1,13 +1,41 @@ -# Stealth-scraper tripwires for Node.js +

+ WebDecoy Node.js SDK: bot detection, AI agent verification and local rules for Express, Fastify, Next.js and Hono. Example tripwire verdict shown in monitor mode. +

-**Turn requests to decoy paths into evidence you can act on.** WebDecoy adds tripwires and local rate limits to Express, Fastify, Next.js and Hono — **no account or API key required for local rules.** +# WebDecoy for Node.js — Bot Detection & AI Agent Verification -Real-browser scrapers can resemble ordinary browsers. A hidden decoy link gives you another signal: a request for a path outside your normal user journey. Inspect the request and your legitimate traffic before choosing to block it. +**Know the bots. Own the decision.** Add bot detection and protection to your Node.js app with honeypot tripwires, rate limiting, AI crawler policies and signed-agent verification. Use local rules without an account or API key; connect WebDecoy when you want hosted analysis and dashboard reporting. -[![npm version](https://img.shields.io/npm/v/@webdecoy/node.svg)](https://www.npmjs.com/package/@webdecoy/node) -[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) +[![CI](https://github.com/WebDecoy/node/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/WebDecoy/node/actions/workflows/ci.yml) +[![npm version](https://img.shields.io/npm/v/%40webdecoy%2Fnode?color=dc318c)](https://www.npmjs.com/package/@webdecoy/node) +[![TypeScript](https://img.shields.io/badge/TypeScript-types_included-3178c6)](packages/webdecoy/src/index.ts) +[![License: MIT](https://img.shields.io/badge/License-MIT-dc318c)](LICENSE) -[Install the Express adapter](#framework-integration-still-no-api-key) · [Verify your first detection](#verify-your-first-detection) · [Express documentation](https://docs.webdecoy.com/sdk-plugins/express/?utm_source=github&utm_medium=readme&utm_campaign=node_sdk_install) +[Quick start](#quick-start--no-api-key) · [Choose your framework](#choose-your-framework) · [Verify a detection](#verify-your-first-detection) · [SDK documentation](https://docs.webdecoy.com/sdk-plugins/node-sdk/?utm_source=github&utm_medium=readme&utm_campaign=node_sdk_install) · [Examples](#examples) + +## What you can do + +| Need | Start here | +| --- | --- | +| Detect requests to hidden decoys and scanner-bait paths | [Honeypot tripwires](#why-add-tripwires) | +| Limit repeated requests to your app or API | [Local rules](#more-local-rules) and [shared rate limits](#rate-limits-across-more-than-one-process) | +| Check supported signed AI-agent identities | [Web Bot Auth verification](#verify-ai-agents-web-bot-auth) | +| Publish and enforce an AI crawler policy | [One bot policy](#one-bot-policy-published-and-enforced) | +| Add browser interaction evidence to server decisions | [Client signals](#catching-a-real-browser-that-isnt-a-real-user) | +| See server-side analysis and detections in a dashboard | [Optional WebDecoy platform](#optional-the-webdecoy-platform-api-key) | + +WebDecoy works alongside Cloudflare and other CDN protections. Server middleware observes requests that reach your application; requests served or blocked before it runs are outside its view. A crawler name is a claim until supported identity checks establish more. + +## Choose your framework + +| Your app | Package and setup | +| --- | --- | +| Express | [Express middleware](packages/express/README.md) | +| Fastify | [Fastify plugin](packages/fastify/README.md) | +| Next.js | [Next.js middleware](packages/nextjs/README.md) | +| Hono, Cloudflare Workers, Bun or Deno | [Hono middleware](packages/hono/README.md) | +| Astro, SvelteKit, Remix or another fetch handler | [`createFetchGuard()`](#any-other-fetch-runtime--no-package-needed) | +| A custom Node.js server | [Core SDK](packages/webdecoy/README.md) | The middleware starts in **monitor mode**: it records verdicts without blocking requests. A local rules setup needs no cloud connection; an API key is needed to send detections to the WebDecoy dashboard. @@ -172,7 +200,7 @@ const wd = new WebDecoy({ ## Verify AI agents (Web Bot Auth) -AI agents like OpenAI's Operator now **cryptographically sign** their requests +Some AI agents support **cryptographically signed** requests ([Web Bot Auth](https://datatracker.ietf.org/wg/webbotauth/about/), RFC 9421). WebDecoy verifies those signatures **locally** — on Node and on the edge, with no API key and no network on the warm path — so you can tell a real verified agent diff --git a/docs/assets/README.md b/docs/assets/README.md new file mode 100644 index 0000000..591004e --- /dev/null +++ b/docs/assets/README.md @@ -0,0 +1,21 @@ +# Node SDK branding + +`node-banner.svg` is the editable source for the 1600 × 800 README banner. +`node-banner.png` is the GitHub/npm-compatible render. It uses the WebDecoy duck, +dark background and pink accents. The rule decision is explicitly an example, +not a captured customer result or an efficacy benchmark. + +The PNG also has a 2:1 aspect ratio suitable for the repository's GitHub social +preview. Upload it under **Settings → General → Social preview**. A README image +does not automatically configure that setting. + +To regenerate from the SVG with Sharp installed in your development environment: + +```js +import sharp from 'sharp'; +await sharp('docs/assets/node-banner.svg').png().toFile('docs/assets/node-banner.png'); +``` + +The core package README uses an absolute `raw.githubusercontent.com` URL so the +banner can render on npm. That URL becomes available after this asset is merged +to `main`; npm README and keyword changes require the next normal package release. diff --git a/docs/assets/node-banner.png b/docs/assets/node-banner.png new file mode 100644 index 0000000..f070a09 Binary files /dev/null and b/docs/assets/node-banner.png differ diff --git a/docs/assets/node-banner.svg b/docs/assets/node-banner.svg new file mode 100644 index 0000000..d7c3df0 --- /dev/null +++ b/docs/assets/node-banner.svg @@ -0,0 +1,23 @@ + +WebDecoy for Node.js — Know the bots. Own the decision. +Bot detection and AI agent verification for Express, Fastify, Next.js and Hono. WebDecoy duck in a pink hoodie beside a clearly labeled example rule decision. + + + +WebDecoy / Node.js SDK +Know the bots. +Own the decision. +Bot detection. AI agent verification. Local rules. + +EXAMPLE · MONITOR MODE +GET /.env +rule tripwire +verdict DENY · observed, not blocked +No account or API key for local rules. + + +npm i @webdecoy/node + +Express / Fastify / Next.js / Hono +webdecoy.com + \ No newline at end of file diff --git a/package.json b/package.json index e473474..de1c4e7 100644 --- a/package.json +++ b/package.json @@ -2,7 +2,7 @@ "name": "webdecoy-monorepo", "version": "0.1.0", "private": true, - "description": "Web Decoy Node.js SDK - Bot detection with TLS fingerprinting", + "description": "WebDecoy Node.js SDK and framework middleware for bot detection, AI agent verification and local protection rules.", "repository": { "type": "git", "url": "git+https://github.com/WebDecoy/node.git" diff --git a/packages/webdecoy/README.md b/packages/webdecoy/README.md index 27a38a2..aa9a8dc 100644 --- a/packages/webdecoy/README.md +++ b/packages/webdecoy/README.md @@ -1,6 +1,10 @@ +

+ WebDecoy for Node.js: bot detection, AI agent verification and local rules +

+ # @webdecoy/node -The core WebDecoy SDK for Node.js and edge runtimes: tripwires, rate limits and other local rules, plus optional server-side bot detection reported to your WebDecoy dashboard. +TypeScript bot detection and protection for Node.js and edge runtimes. Run honeypot tripwires, rate limits, AI crawler policies and signed-agent verification locally without an account or API key. Add optional hosted analysis and WebDecoy dashboard reporting when you need them. [![npm version](https://img.shields.io/npm/v/@webdecoy/node.svg)](https://www.npmjs.com/package/@webdecoy/node) [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT) diff --git a/packages/webdecoy/package.json b/packages/webdecoy/package.json index 660f703..8ce2830 100644 --- a/packages/webdecoy/package.json +++ b/packages/webdecoy/package.json @@ -1,7 +1,7 @@ { "name": "@webdecoy/node", "version": "0.22.3", - "description": "Web Decoy SDK for Node.js - Bot detection with TLS fingerprinting", + "description": "Bot detection and AI agent verification for Node.js and edge runtimes. Local honeypot tripwires, rate limiting and AI crawler policies, with optional WebDecoy dashboard reporting.", "main": "./dist/index.js", "types": "./dist/index.d.ts", "exports": { @@ -43,13 +43,25 @@ "check:edge": "node ../../scripts/check-edge.mjs src/index.ts src/vercel.ts" }, "keywords": [ - "web-decoy", + "webdecoy", + "nodejs", + "typescript", "bot-detection", - "tls-fingerprinting", - "ja3", - "ja4", - "security", - "middleware" + "bot-protection", + "ai-agents", + "ai-crawlers", + "web-bot-auth", + "honeypot", + "tripwire", + "rate-limiting", + "anti-scraping", + "middleware", + "express", + "fastify", + "nextjs", + "hono", + "edge-runtime", + "tls-fingerprinting" ], "author": "WebDecoy LLC", "license": "MIT",