Package category
Scraping and browser automation
Headless browsers, crawlers and HTML extraction.
373 packages7 comparisons
Packages compared
373 packages
| Package | Weekly downloads | 12-month change | 52 weeks | Gzip | Last release | Module | Types | Categories |
|---|---|---|---|---|---|---|---|---|
| express-nobots Keep Bots Away From Your Express App | 2.3k | +1115% | - | 6 years ago 1.0.5 | CommonJS | None | HTTP servers and web frameworks, Scraping and browser automation | |
| open-graph-scraper-lite Javascript scraper module for Open Graph and Twitter Card info | 2.2k | +126% | - | 2 years ago 2.1.0 | ESM + CommonJS | Bundled | Scraping and browser automation | |
| @apidojo/x-scraper The fastest and cheapest way to scrape tweets from X (Twitter). Wraps the Apify Twitter Scraper Lite actor with a developer-friendly API. | 2.2k | - | - | - | 6 months ago 1.1.0 | ESM + CommonJS | Bundled | Scraping and browser automation |
| crawler Crawler is a ready-to-use web spider that works with proxies, asynchrony, rate limit, configurable request pools, jQuery, and HTTP/2 support. | 2.2k | -38% | - | 3 months ago 2.1.1 | ESM + CommonJS | None | Scraping and browser automation, HTTP clients | |
| @avocadostudio-ai/migration-sdk Utilities for migrating existing site content into the Avocado Studio PageDoc/BlockInstance shape | 2.2k | - | - | - | 2 days ago 0.21.1 | ESM only | Bundled | Scraping and browser automation, Testing |
| spider-detector A tiny node module to detect spiders/crawlers quickly and comes with optional middleware for ExpressJS | 2.2k | -13% | - | 2 years ago 2.1.0 | CommonJS | None | Scraping and browser automation | |
| safari-mcp Safari browser automation for AI agents — native macOS, zero Chrome overhead. 98 tools via AppleScript + JavaScript. | 2.2k | - | - | 1 day ago 2.21.14 | ESM only | None | Scraping and browser automation | |
| scenescout SceneScout — exploratory UI testing for AI coding agents. An MCP server that gives any agent (Claude Code, Cursor, VS Code Copilot, Codex, Gemini CLI and others) a structured view of a running web app, always-on oracles, a network-level write policy, memo | 2.1k | - | - | - 3.7.0 | ESM only | None | Scraping and browser automation, Testing | |
| @divriots/cheerio The fast, flexible & elegant library for parsing and manipulating HTML and XML. | 2.1k | - | - | - | 3 years ago 1.0.0-rc.12 | ESM + CommonJS | Bundled | Parsers and serialisers, DOM and browser utilities |
| out-url Cross platform Node.js Utility to open urls in browser | 2k | -11% | - | 5 days ago 1.5.0 | CommonJS | Bundled | CLI tools and terminal utilities, DOM and browser utilities | |
| automate-google-login-scraper End-to-end test harness for Google sign-in: persist a Playwright storageState once, reuse it everywhere, and replay it from a Cloudflare Workers Browser Rendering Durable Object | 1.9k | - | - | today 0.1.35 | ESM only | Bundled | Testing, Authentication and authorisation | |
| spider-browser Browser automation client for Spider's pre-warmed browser fleet with smart retry and browser switching | 1.9k | - | - | 3 months ago 0.3.0 | ESM + CommonJS | Bundled | Scraping and browser automation, DOM and browser utilities | |
| outscraper The library provides convenient access to the Outscraper API. Allows using Outscraper's services from your code. See https://outscraper.com for details. | 1.9k | +527% | - | 1 day ago 2.2.5 | CommonJS | Bundled | Scraping and browser automation | |
| @mradex77/google-play-scraper Google Play scraper for Node.js with a fully typed TypeScript API. Fetch app details, search results, top charts, reviews, permissions and data safety from the Play Store. | 1.9k | - | - | - | 1 day ago 1.3.0 | ESM + CommonJS | Bundled | Scraping and browser automation, TypeScript tooling |
| @consumet/extensions Nodejs library that provides high-level APIs for obtaining information on various entertainment media such as books, movies, comic books, anime, manga, and so on. | 1.9k | - | - | - | 8 months ago 1.8.8 | CommonJS | Bundled | Scraping and browser automation |
| very-happy-dom Like a light-weight version of `happy-dom` powered by Bun. | 1.9k | +1147% | - | 2 months ago 0.1.10 | ESM only | Bundled | Scraping and browser automation, Testing | |
| x-ray-scraper Scraper next gen based on x-ray (2.3.2) | 1.8k | +572% | - | 6 years ago 3.0.6 | CommonJS | None | Scraping and browser automation | |
| ag-webscrape Generic TypeScript web scraper with a headless-browser fallback for anti-scraping protection | 1.8k | +1585% | - | 6 days ago 0.0.38 | CommonJS | Bundled | Scraping and browser automation, TypeScript tooling | |
| gifted-dls Gifted-Dls: Social Media(Youtube, Tiktok, Facebook, Instagram, Twitter, Spotify, +18) Downloaders and Some Api Tools | 1.7k | +4% | - | 1 year ago 1.3.5 | CommonJS | None | Scraping and browser automation | |
| @crawlee/http-client The scalable web crawling and scraping library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer. | 1.7k | - | - | - | 7 months ago 4.0.0-beta.23 | ESM only | Bundled | Scraping and browser automation |
| testreel Programmatic video infrastructure for web apps | 1.7k | - | - | 5 months ago 0.2.0 | ESM + CommonJS | Bundled | Scraping and browser automation, Testing | |
| scrapegraph-js Official JavaScript/TypeScript SDK for the ScrapeGraph AI API — smart web scraping powered by AI | 1.7k | +153% | - | 1 month ago 2.2.1 | ESM only | Bundled | Scraping and browser automation, TypeScript tooling | |
| @scrapecreators/cli CLI for the ScrapeCreators API — use 180+ endpoints across 30+ platforms from the terminal | 1.6k | - | - | - | 1 day ago 1.0.43 | ESM only | None | CLI tools and terminal utilities, Scraping and browser automation |
| torrent-search-api Yet another node torrent scraper based on x-ray. (Support iptorrents, torrentleech, torrent9, Yyggtorrent, ThePiratebay, torrentz2, 1337x, KickassTorrent, Rarbg, TorrentProject, Yts, Limetorrents, Eztv) | 1.6k | +264% | - | 5 years ago 2.1.4 | CommonJS | None | Scraping and browser automation | |
| @nodebb/spider-detector A tiny node module to detect spiders/crawlers quickly and comes with optional middleware for ExpressJS | 1.6k | - | - | - | 2 years ago 2.0.3 | CommonJS | None | Scraping and browser automation |
| udger-nodejs NodeJS User-Agent String Parser based on Udger SQLite databases https://udger.com/products/local_parser | 1.6k | +1093% | - | 10 months ago 1.5.1 | CommonJS | None | Scraping and browser automation | |
| crawler-request HTTP request module customized for crawlers. | 1.5k | -22% | - | 8 years ago 1.2.2 | CommonJS | None | Scraping and browser automation | |
| sitemap-generator-cli Create xml sitemaps from the command line. | 1.5k | +27% | - | 6 years ago 7.5.0 | CommonJS | None | Scraping and browser automation, CLI tools and terminal utilities | |
| puppeteer-infinite-scroller Provides a simple and efficient solution for scraping data loaded through infinite scrolling on web pages using Puppeteer. | 1.5k | -25% | - | 2 years ago 1.0.2 | CommonJS | Bundled | Scraping and browser automation | |
| open-agents-ai AI coding agent powered by open-source models (Ollama/vLLM) — interactive TUI with agentic tool-calling loop | 1.5k | - | - | 4 months ago 0.187.596 | ESM only | Bundled | Scraping and browser automation, Testing | |
| crawlbase Dependency free module for scraping and crawling websites using [Crawlbase](https://crawlbase.com) API | 1.5k | +87% | - | 2 years ago 1.0.2 | CommonJS | Bundled | Scraping and browser automation | |
| puppeteer-mass-screenshots This package creates massive amount of screenshots automatically, using Chrome API screencast, | 1.5k | -69% | - | 5 years ago 1.0.15 | CommonJS | None | Scraping and browser automation | |
| playwright-persona Authentication in Playwright using personas. | 1.5k | +6963% | - | 5 months ago 0.3.0 | ESM only | None | Authentication and authorisation, Testing | |
| @enricai/barnacle Barnacle turns any website into an API. POST a structured payload to a typed endpoint and Barnacle drives a browser session through the target site, returning a structured result. | 1.5k | - | - | - | 1 day ago 1.12.65 | CommonJS | Bundled | Scraping and browser automation, HTTP servers and web frameworks |
| @microlink/mcp MCP server for Microlink API | 1.4k | - | - | - | 4 days ago 2.8.5 | ESM only | None | Scraping and browser automation, CLI tools and terminal utilities |
| protractor-http-client HTTP client to be used in protractor tests | 1.4k | -74% | - | 8 years ago 1.0.4 | CommonJS | None | Testing, Scraping and browser automation | |
| @electrovir/rebrowser-playwright-core A drop-in replacement for playwright-core patched with rebrowser-patches. It allows to pass modern automation detection tests. | 1.4k | - | - | - | 2 months ago 1.61.101 | ESM + CommonJS | Bundled | Scraping and browser automation |
| @zorilla/puppeteer-extra-plugin-stealth Stealth mode: Applies various techniques to make detection of headless puppeteer harder. | 1.3k | - | - | - | 2 months ago 2.0.1 | ESM only | Bundled | Scraping and browser automation |
| soundcloud-scraper Get data from soundcloud easily. | 1.2k | +101% | - | 4 years ago 5.0.3 | CommonJS | Bundled | Scraping and browser automation | |
| qiksy-mcp Browser MCP server for the Chrome tab you already have open — your session, your logins. Gives Claude Code, Cursor, Codex and VS Code the live page: findings, forms, failed requests with server bodies. | 1.2k | - | - | - 1.55.0 | ESM only | None | Accessibility, Testing | |
| is-antibot Detect antibot protection from 30+ providers — Cloudflare, Akamai, DataDome, PerimeterX, and more. | 1.2k | - | - | 2 days ago 2.5.17 | CommonJS | Bundled | Scraping and browser automation, Cloud SDKs | |
| ai-ready-pw-codegen AI-Ready PW Codegen — offline Playwright recorder with snapshots for AI-powered test generation | 1.2k | - | - | - 1.7.0 | CommonJS | None | Testing, Scraping and browser automation | |
| jquery-test-runner A test runner built by the jQuery team to run QUnit tests in real browsers using Selenium and BrowserStack | 1.2k | +55% | - | 2 months ago 0.3.1 | ESM only | None | Testing, Scraping and browser automation | |
| @bacnh85/pi-web Pi extension for web search, page extraction, Firecrawl scraping/crawling, Crawl4AI headless browser crawling, real-browser interaction (trusted click/type/evaluate via CDP), Gemini web-tier research, free upstream image generation (Gemini/ChatGPT web/Z.a | 1.2k | - | - | - | today 0.17.5 | ESM only | None | Scraping and browser automation |
| html-pdf-chrome HTML to PDF and image converter via Chrome/Chromium | 1.2k | -31% | - | 3 years ago 0.8.4 | CommonJS | Bundled | PDF and documents, TypeScript tooling | |
| node-scrapy Simple, lightweight and expressive web scraping with Node.js | 1.2k | -28% | - | 6 years ago 0.5.0 | CommonJS | None | Scraping and browser automation | |
| metafetch Metafetch fetches a given URL's title, description, images, links etc. | 1.2k | -58% | - | 5 months ago 6.0.0 | ESM only | Bundled | Scraping and browser automation | |
| @crawlee/fs-storage A file-system storage implementation of the Apify API | 1.2k | - | - | - | 2 months ago 4.0.0-beta.67 | ESM only | Bundled | Scraping and browser automation, Files and file systems |
| @spider-rs/spider-rs The [spider](https://github.com/spider-rs/spider) project ported to Node.js | 1.2k | - | - | - | 8 months ago 0.0.163 | CommonJS | Bundled | Scraping and browser automation |
| pixiv-token-getter Node.js Pixiv credential manager and authentication facade - token cache, refresh-token lifecycle, profiles, PKCE OAuth + Puppeteer login, optional gppt interoperability. Library and CLI (ptg). TypeScript types included. | 1.2k | - | - | 13 days ago 2.6.1 | ESM + CommonJS | Bundled | Authentication and authorisation, CLI tools and terminal utilities |
12-month change compares the average of the last 4 weeks of downloads with the first 4 weeks of the 52-week series. Gzip size is for the whole package, as measured by Bundlephobia. "-" means the value has not been fetched.
- express-nobotsKeep Bots Away From Your Express App
- open-graph-scraper-liteJavascript scraper module for Open Graph and Twitter Card info
- @apidojo/x-scraperThe fastest and cheapest way to scrape tweets from X (Twitter). Wraps the Apify Twitter Scraper Lite actor with a developer-friendly API.
- crawlerCrawler is a ready-to-use web spider that works with proxies, asynchrony, rate limit, configurable request pools, jQuery, and HTTP/2 support.
- @avocadostudio-ai/migration-sdkUtilities for migrating existing site content into the Avocado Studio PageDoc/BlockInstance shape
- spider-detectorA tiny node module to detect spiders/crawlers quickly and comes with optional middleware for ExpressJS
- safari-mcpSafari browser automation for AI agents — native macOS, zero Chrome overhead. 98 tools via AppleScript + JavaScript.
- scenescoutSceneScout — exploratory UI testing for AI coding agents. An MCP server that gives any agent (Claude Code, Cursor, VS Code Copilot, Codex, Gemini CLI and others) a structured view of a running web app, always-on oracles, a network-level write policy, memo
- @divriots/cheerioThe fast, flexible & elegant library for parsing and manipulating HTML and XML.
- out-urlCross platform Node.js Utility to open urls in browser
- automate-google-login-scraperEnd-to-end test harness for Google sign-in: persist a Playwright storageState once, reuse it everywhere, and replay it from a Cloudflare Workers Browser Rendering Durable Object
- spider-browserBrowser automation client for Spider's pre-warmed browser fleet with smart retry and browser switching
- outscraperThe library provides convenient access to the Outscraper API. Allows using Outscraper's services from your code. See https://outscraper.com for details.
- @mradex77/google-play-scraperGoogle Play scraper for Node.js with a fully typed TypeScript API. Fetch app details, search results, top charts, reviews, permissions and data safety from the Play Store.
- @consumet/extensionsNodejs library that provides high-level APIs for obtaining information on various entertainment media such as books, movies, comic books, anime, manga, and so on.
- very-happy-domLike a light-weight version of `happy-dom` powered by Bun.
- x-ray-scraperScraper next gen based on x-ray (2.3.2)
- ag-webscrapeGeneric TypeScript web scraper with a headless-browser fallback for anti-scraping protection
- gifted-dlsGifted-Dls: Social Media(Youtube, Tiktok, Facebook, Instagram, Twitter, Spotify, +18) Downloaders and Some Api Tools
- @crawlee/http-clientThe scalable web crawling and scraping library for JavaScript/Node.js. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.
- testreelProgrammatic video infrastructure for web apps
- scrapegraph-jsOfficial JavaScript/TypeScript SDK for the ScrapeGraph AI API — smart web scraping powered by AI
- @scrapecreators/cliCLI for the ScrapeCreators API — use 180+ endpoints across 30+ platforms from the terminal
- torrent-search-apiYet another node torrent scraper based on x-ray. (Support iptorrents, torrentleech, torrent9, Yyggtorrent, ThePiratebay, torrentz2, 1337x, KickassTorrent, Rarbg, TorrentProject, Yts, Limetorrents, Eztv)
- @nodebb/spider-detectorA tiny node module to detect spiders/crawlers quickly and comes with optional middleware for ExpressJS
- udger-nodejsNodeJS User-Agent String Parser based on Udger SQLite databases https://udger.com/products/local_parser
- crawler-requestHTTP request module customized for crawlers.
- sitemap-generator-cliCreate xml sitemaps from the command line.
- puppeteer-infinite-scrollerProvides a simple and efficient solution for scraping data loaded through infinite scrolling on web pages using Puppeteer.
- open-agents-aiAI coding agent powered by open-source models (Ollama/vLLM) — interactive TUI with agentic tool-calling loop
- crawlbaseDependency free module for scraping and crawling websites using [Crawlbase](https://crawlbase.com) API
- puppeteer-mass-screenshotsThis package creates massive amount of screenshots automatically, using Chrome API screencast,
- playwright-personaAuthentication in Playwright using personas.
- @enricai/barnacleBarnacle turns any website into an API. POST a structured payload to a typed endpoint and Barnacle drives a browser session through the target site, returning a structured result.
- @microlink/mcpMCP server for Microlink API
- protractor-http-clientHTTP client to be used in protractor tests
- @electrovir/rebrowser-playwright-coreA drop-in replacement for playwright-core patched with rebrowser-patches. It allows to pass modern automation detection tests.
- @zorilla/puppeteer-extra-plugin-stealthStealth mode: Applies various techniques to make detection of headless puppeteer harder.
- soundcloud-scraperGet data from soundcloud easily.
- qiksy-mcpBrowser MCP server for the Chrome tab you already have open — your session, your logins. Gives Claude Code, Cursor, Codex and VS Code the live page: findings, forms, failed requests with server bodies.
- is-antibotDetect antibot protection from 30+ providers — Cloudflare, Akamai, DataDome, PerimeterX, and more.
- ai-ready-pw-codegenAI-Ready PW Codegen — offline Playwright recorder with snapshots for AI-powered test generation
- jquery-test-runnerA test runner built by the jQuery team to run QUnit tests in real browsers using Selenium and BrowserStack
- @bacnh85/pi-webPi extension for web search, page extraction, Firecrawl scraping/crawling, Crawl4AI headless browser crawling, real-browser interaction (trusted click/type/evaluate via CDP), Gemini web-tier research, free upstream image generation (Gemini/ChatGPT web/Z.a
- html-pdf-chromeHTML to PDF and image converter via Chrome/Chromium
- node-scrapySimple, lightweight and expressive web scraping with Node.js
- metafetchMetafetch fetches a given URL's title, description, images, links etc.
- @crawlee/fs-storageA file-system storage implementation of the Apify API
- @spider-rs/spider-rsThe [spider](https://github.com/spider-rs/spider) project ported to Node.js
- pixiv-token-getterNode.js Pixiv credential manager and authentication facade - token cache, refresh-token lifecycle, profiles, PKCE OAuth + Puppeteer login, optional gppt interoperability. Library and CLI (ptg). TypeScript types included.