GHSA-jm3v-qxmh-hxwv: Scrapy's redirects ignoring scheme-specific proxy settings
Impact When using system proxy settings, which are scheme-specific (i.e. specific to http:// or https:// URLs), Scrapy was not accounting for scheme changes during redirects. For example, an HTTP request would use the proxy configured for HTTP and, when redirected to an HTTPS URL, the new HTTPS request would still use the proxy configured for HTTP instead of switching to the proxy configured for HTTPS.
Published May 14, 2024
Affected packages
Scrapy
- Ecosystem
- PyPI
- Affected
- < 2.11.2
- Fixed in
- 2.11.2
- Check
pip show Scrapy
Advisory details
From the GitHub Advisory Database (CC BY 4.0).
Impact
When using system proxy settings, which are scheme-specific (i.e. specific to http:// or https:// URLs), Scrapy was not accounting for scheme changes during redirects.
For example, an HTTP request would use the proxy configured for HTTP and, when redirected to an HTTPS URL, the new HTTPS request would still use the proxy configured for HTTP instead of switching to the proxy configured for HTTPS. Same the other way around.
If you have different proxy configurations for HTTP and HTTPS in your system for security reasons (e.g., maybe you don’t want one of your proxy providers to be aware of the URLs that you visit with the other one), this would be a security issue.
Patches
Upgrade to Scrapy 2.11.2.
Workarounds
Replace the built-in retry middlewares (RedirectMiddleware and MetaRefreshMiddleware) and the HttpProxyMiddleware middleware with custom ones that implement the fix from Scrapy 2.11.2, and verify that they work as intended.
References
This security issue was reported by @redapple at https://github.com/scrapy/scrapy/issues/767.
References
How this page is made: the data panel comes from the GitHub Advisory Database, CISA’s KEV catalog, the NVD and FIRST EPSS, and updates twice a day. No analysis has been written for this entry yet. Methodology
More PyPI vulnerabilities
All PyPI entriesCVE-2026-105782
Scrapy: Arbitrary Module Import via Referrer-Policy Header in RefererMiddleware
high7.5Mar 13, 2026CVE-2025-6176
Scrapy is vulnerable to a denial of service (DoS) attack due to flaws in brotli decompression implementation
high7.5Oct 31, 2025CVE-2024-1968
Scrapy leaks the authorization header on same-domain but cross-origin redirects
medium5.9May 14, 2024GHSA-23j4-mw76-5v7h
Scrapy allows redirect following in protocols other than HTTP
medium6.5May 14, 2024
