We may earn a commission if you buy through links on this page. It never changes a score or a ranking. How we make money
The bottom line
Scrapy offers strong free crawling tools, but expect hands-on engineering and extra work for dynamic sites.
Scrapy is worth considering if your small team can build and maintain crawlers in code. Its free framework includes selectors, an interactive shell and feed exports; users report that dynamic sites can take extra work.
Scrapy Cloud has a free tier for limited hosted jobs, then charges $9 per unit monthly. One unit is $9; three and five units cost $27 and $45 monthly, not per person.
Good fit if
- Teams building custom site crawlers
- Budget-conscious developers comfortable coding
- Structured exports to JSON, CSV or S3
Skip it if
- Teams looking for no-code scraping
- Projects needing easy JavaScript rendering
- Buyers who need published seat pricing

ScoresHow it rates
What works
- Open-source framework is free
- CSS and XPath selectors for extraction
- Interactive shell for trying selectors
- Exports include JSON, CSV and S3
- Hosted units include one concurrent crawl each
What doesn't
- Setup and maintenance require engineering work
- Dynamic pages may take extra effort
- Hosted free jobs run up to one hour
- Hosted free data is retained for seven days
- Seat and open-source usage limits are unpublished
01Who Scrapy is for
Scrapy fits a small team that needs custom website crawlers and has someone comfortable writing and maintaining code. It is an open-source framework for crawling sites, extracting structured data and exporting results, rather than a ready-made scraper for nontechnical users.
The framework coordinates crawling through components such as spiders, downloaders, schedulers and pipelines. Spiders can use CSS or XPath selectors to extract data and follow links. That flexibility suits recurring collection jobs where the team wants control over how data is gathered and exported. The interactive shell can help developers try selectors before building a spider.
For a team of one to ten, the open-source framework costs $0, with no seat or usage limits published. Scrapy Cloud is an option if you want hosted crawls; its free tier is constrained, while paid units start at $9 monthly. If your team mainly needs a point-and-click tool, Scrapy is likely more work than you need.
02Scrapy free plan and trial
There are two free ways in. Scrapy Open Source is free under the BSD-3-Clause license. The pricing information lists no seat or usage limits for it, but those limits are not published. You will need to build and run your own crawlers rather than treat it as hosted crawl capacity.
Scrapy Cloud Free includes one low-resource unit, one concurrent crawl, jobs up to one hour and data retention up to seven days. That can be enough to try a hosted crawl, but the job-time and retention limits matter for longer or recurring work. The listed paid option is Scrapy Cloud Units at $9 per unit per month. A free trial is not listed. Annual billing prices are not published.
03What a small team pays for Scrapy
Scrapy Open Source costs $0 per month. The pricing page does not publish seat limits or usage limits, so there is no listed per-person charge to calculate for a one-, three- or five-person team.
For hosted production crawls, Scrapy Cloud Units cost $9 per unit per month. The listing specifies one concurrent crawl and 1 GB of RAM per unit, so one unit is $9 monthly, three units are $27, and five are $45. Those figures are for units, not seats; the pricing information does not say how many team members can use a unit. Each unit includes unlimited job run time and data retention up to 120 days.
The free hosted tier costs $0 and includes one low-resource unit. Annual prices are not published, and the listed pricing includes no add-ons. Before budgeting, ask Scrapy how many units your expected workload needs and whether there are team-access charges.
04Getting started with Scrapy
Plan for engineering setup, not a quick no-code configuration. Scrapy is a framework built from components such as spiders, schedulers, downloaders, pipelines and feed exports, so someone on the team needs to shape the crawl and maintain its code. The interactive shell helps with one early task: loading pages and trying CSS or XPath selectors before writing a spider.
For a first crawl, decide what pages to follow, what fields to extract and where to export the results. The listed feed destinations include JSON, CSV and S3. If pages rely on JavaScript, the scrapy-playwright extension can render them in a real browser, but that adds another component to set up.
One Reddit thread describes Scrapy as fast on static sites while noting that dynamic-content support took extra work. The available pricing and feature details do not publish setup times or a step-by-step onboarding process, so a team should budget for hands-on development.
05When small teams outgrow Scrapy
The open-source framework has no published seat or usage limits, so growing headcount does not automatically mean a higher listed subscription. The pressure is more likely to come from the work of building and maintaining crawlers, or from needing hosted capacity. Scrapy Cloud Free allows one concurrent crawl, jobs of up to one hour and seven days of data retention. Paid units add one concurrent crawl each, unlimited job run time and retention up to 120 days.
Teams may also need extra browser-rendering support for JavaScript-heavy sites; the feature list points to scrapy-playwright for that. One Reddit user says adding dynamic-content support took extra work, while another describes Scrapy as restrictive and unintuitive. A separate commenter reports a 403 fetch error on a site, illustrating that a framework does not guarantee access to every target. If the team needs managed anti-bot handling or a less code-heavy workflow, compare its requirements before expanding the build.
From the threadsWhat users say
Reddit sentiment is mixed: users praise Scrapy's speed and asynchronous crawling, while others describe a learning curve or extra work on dynamic sites. A reported 403 error also shows that Scrapy does not make every site accessible.
Scrapy: Super fast on static sites, though adding support for dynamic content took extra work
I find it restrictive and not intuitive to me
Scrapy handle a lot of requests with asynchronous way, with very easy, fast written codes
I'm encountering a 403 fetch error after the search and I'm not sure how to get around that with Scrapy
Questions people ask
Is Scrapy free?
Yes. Scrapy Open Source is listed as free under the BSD-3-Clause license. Its seat and usage limits are not published. Scrapy Cloud also has a free tier for hosted crawls, with one low-resource unit, one concurrent crawl, jobs up to one hour and data retention up to seven days.
What can Scrapy export?
Scrapy's listed feed-export destinations include JSON, CSV and S3. The framework extracts structured data, and feed exports let a team send crawl results to a destination it can use. The available pricing and feature information does not list a complete set of export formats, so check whether you need another destination.
Can Scrapy crawl JavaScript-heavy pages?
The scrapy-playwright extension can render JavaScript-heavy pages in a real browser. Scrapy's core features also include CSS and XPath selectors and an interactive shell. A Reddit user reports that adding dynamic-content support took extra work, so teams should account for the additional setup rather than assume it is included in a basic crawl.
How much does Scrapy Cloud cost?
Scrapy Cloud Free costs $0 and includes one low-resource unit and one concurrent crawl. Scrapy Cloud Units are listed at $9 per unit per month. One, three and five units therefore cost $9, $27 and $45 monthly. Annual pricing is not published, and unit pricing is not the same as per-seat pricing.
Does Scrapy charge per seat?
The pricing information does not publish seat limits or per-seat charges for Scrapy Open Source. Scrapy Cloud Units are priced per unit per month, not per person in the listed pricing. Ask Scrapy whether your team has any account or access charges, especially if several people need to manage hosted crawls.
Sources
- Scrapy homepage
- Scrapy pricing, compiled via web search from coldiq.com, scrapy.org, thunderbit.com, webharvy.com, zyte.com, geekflare.com
- Reddit: Best web scraping tools I’ve tried (and what I learned from each)
- Reddit: From Scrapy to Apify: how a retail data agency saved 90% costs on web scraping
- Reddit: I don't use Scrapy. Am I missing out?
- Reddit: How effective is Scrapy?
