Skip to content
JS
Package

robots-txt-parser

A lightweight robots.txt parser for Node.js with support for wildcards, caching and promises.

v2.0.3, 6 May 2022MITCommonJSTypes: NoneParsers and serialisersScraping and browser automation
$ npm install robots-txt-parser

Key numbers

As of 25 Sept 2026
Weekly downloads
14.7k
+348% over 12 months
Gzip bundle size
-
No Bundlephobia data
Dependencies
3
Direct, from package.json
Last release
4 years ago
v2.0.3
GitHub stars
17
10 forks
Open issues / PRs
7 / 3
Contributors
-
Commits, last 52 weeks
0

Downloads

Last week 2026-09-15 to 2026-09-21
020k40k60kSept 2025Mar 2026Sept 2026High 41.1k (1 Sept 2026)Low 1.8k (11 Nov 2025)
Weekly npm downloads, one point per week, weeks starting 23 Sept 2025 to 15 Sept 2026. The y axis starts at zero.

14,721 downloads last week, 85,853 in the last month. The 12-month change of +348% compares the average of the last 4 weeks with the first 4 weeks of the series.

Releases, last 12 months
-
Latest version
2.0.3, 6 May 20224 years ago
First published
-
Last push to GitHub
23 Jun 20231,189 days before the snapshot
Repository archived
No
Open issues
7
Open pull requests
3
Commits, last 52 weeks
0
Dependents on npm
8

Major versions

-

Recent releases

-

Module format
CommonJSFrom package.json type, main and exports fields
TypeScript types
None
Node.js engine range
Not declared
Runtimes
-
Unpacked size (npm)
58 kB
Dependencies (3)
simple-getfast-url-parseris-absolute-url
npm install robots-txt-parser
  • jsDelivrhttps://cdn.jsdelivr.net/npm/robots-txt-parser@2.0.3/
  • unpkghttps://unpkg.com/robots-txt-parser@2.0.3/
  • esm.shhttps://esm.sh/robots-txt-parser@2.0.3
PackageWeekly downloadsGzipLast releaseModuleComparison
cheerio19.8m-8 months agoESM + CommonJS-
serpapi112.9k-1 year agoESM + CommonJS-
domparser-rs60.6k-5 months agoCommonJS-
sitemapd63.4k-1 month agoESM only-
dom-parser51.5k-2 years agoCommonJS-
@metascraper/helpers132.1k-1 month agoCommonJS-
html-table-to-json6.1k----
Sources: npm registry, npm downloads API, GitHub API. Data fetched 25 Sept 2026.

Indexability: Indexed.