GHSA-23j4-mw76-5v7h: Scrapy allows redirect following in protocols other than HTTP
Impact Scrapy was following redirects regardless of the URL protocol, so redirects were working for data://, file://, ftp://, s3://, and any other scheme defined in the DOWNLOADHANDLERS setting. However, HTTP redirects should only work between URLs that use the http:// or https:// schemes. A malicious actor, given write access to the start requests (e.g.
Published May 14, 2024
Affected packages
Scrapy
- Ecosystem
- PyPI
- Affected
- < 2.11.2
- Fixed in
- 2.11.2
- Check
pip show Scrapy
Advisory details
From the GitHub Advisory Database (CC BY 4.0).
Impact
Scrapy was following redirects regardless of the URL protocol, so redirects were working for data://, file://, ftp://, s3://, and any other scheme defined in the DOWNLOAD_HANDLERS setting.
However, HTTP redirects should only work between URLs that use the http:// or https:// schemes.
A malicious actor, given write access to the start requests (e.g. ability to define start_urls) of a spider and read access to the spider output, could exploit this vulnerability to:
- Redirect to any local file using the
file://scheme to read its contents. - Redirect to an
ftp://URL of a malicious FTP server to obtain the FTP username and password configured in the spider or project. - Redirect to any
s3://URL to read its content using the S3 credentials configured in the spider or project.
For file:// and s3://, how the spider implements its parsing of input data into an output item determines what data would be vulnerable. A spider that always outputs the entire contents of a response would be completely vulnerable, while a spider that extracted only fragments from the response could significantly limit vulnerable data.
Patches
Upgrade to Scrapy 2.11.2.
Workarounds
Replace the built-in retry middlewares (RedirectMiddleware and MetaRefreshMiddleware) with custom ones that implement the fix from Scrapy 2.11.2, and verify that they work as intended.
References
This security issue was reported by @mvsantos at https://github.com/scrapy/scrapy/issues/457.
References
How this page is made: the data panel comes from the GitHub Advisory Database, CISA’s KEV catalog, the NVD and FIRST EPSS, and updates twice a day. No analysis has been written for this entry yet. Methodology
More PyPI vulnerabilities
All PyPI entriesCVE-2026-105782
Scrapy: Arbitrary Module Import via Referrer-Policy Header in RefererMiddleware
high7.5Mar 13, 2026CVE-2025-6176
Scrapy is vulnerable to a denial of service (DoS) attack due to flaws in brotli decompression implementation
high7.5Oct 31, 2025CVE-2024-1968
Scrapy leaks the authorization header on same-domain but cross-origin redirects
medium5.9May 14, 2024GHSA-jm3v-qxmh-hxwv
Scrapy's redirects ignoring scheme-specific proxy settings
medium4.3May 14, 2024
