v3.13.0

3.13.0 (2025-03-04)

Bug Fixes

• cheerio: don't decode HTML entities in context.body (#2838) (32d6d0e), closes #2401
• install browser in the camoufox template correctly (#2864) (a9d008c), closes #2863
• Make log message in RequestQueue.isFinished more accurate (#2848) (3d124ae)
• Simplified RequestQueueV2 implementation (#2775) (d1a094a), closes #2767 #2700

Features

• Camoufox-based crawler template (#2842) (7f08de4), closes #2836
• improved cross platform metric collection (#2834) (e41b2f7), closes #2771
• playwright: add handleCloudflareChallenge helper (#2865) (9a1725f)
• remove old docker CI (#2831) (7f09d56)
• use native impit streaming (#2833) (af2fe23), closes #2756

v3.12.2

3.12.2 (2025-01-27)

Bug Fixes

• core: type definition of Dataset.reduce (#2774) (59bc6d1), closes #2773
• destructure CrawlerRunOptions before passing them to addRequests (#2803) (02a598c), closes #2802
• graceful BasicCrawler tidy-up on CriticalError (#2817) (53331e8), closes #2807

Features

• impit-based HttpClient implementation (#2787) (61d7ffa)
• add support for parsing comma-separated list environment variables (#2765) (4e50c47)
• stopping the crawlers gracefully with BasicCrawler.stop() (#2792) (af2966f), closes #2777

v3.12.1

3.12.1 (2024-12-04)

Bug Fixes

• log status message timeouts to debug level (55ee44a)
• social: support new URL formats for Facebook, YouTube and X (#2758) (4c95847), closes #525

Features

• tieredProxyUrls accept null for switching the proxy off (#2743) (82f4ea9), closes #2740

v3.12.0

3.12.0 (2024-11-04)

Bug Fixes

• .trim() urls from pretty-printed sitemap.xml files (#2709) (802a6fe), closes #2698
• ensure correct column order in CSV export (#2734) (b66784f), closes #2718
• ignore errors from iframe content extraction (#2714) (627e5c2), closes #2708
• update fingerprintGeneratorOptions types (#2705) (fcb098d), closes #2703

Features

• allow using other HTTP clients (#2661) (568c655), closes #2659

v3.11.5

3.11.5 (2024-10-04)

Bug Fixes

• core: fix forefront request fetching in RQv2 (#2689) (03951bd), closes #2669
• core: respect forefront option in prolong- and deleteRequestLock (#2690) (cba8da3), closes #2681 #2689 #2669
• core: check .isFinished() before RequestList reads (#2695) (6fa170f)
• core: accept UInt8Array in KVS.setValue() (#2682) (8ef0e60)
• core: trigger errorHandler for session errors (#2683) (7d72bcb), closes #2678
• core: decode special characters in proxy username and password (#2696) (0f0fcc5)
• http-crawler: avoid crashing when gotOptions.cache is on (#2686) (1106d3a)
• puppeteer: rename ignoreHTTPSErrors to acceptInsecureCerts to support v23 (#2684) (f3927e6)
• memory-storage: respect forefront option in RequestQueue (#2681) (b0527f9), closes #2669

v3.11.4

3.11.4 (2024-09-23)

Bug Fixes

• SitemapRequestList.teardown() doesn't break persistState calls (#2673) (fb2c5cd), closes #2672

v3.11.3

3.11.3 (2024-09-03)

Bug Fixes

• improve FACEBOOK_REGEX to match older style page URLs (#2650) (a005e69), closes #2216
• RequestQueueV2: reset recently handled cache too if the queue is pending for too long (#2656) (51a69bc)

v3.11.2

3.11.2 (2024-08-28)

Bug Fixes

• RequestQueueV2: remove inProgress cache, rely solely on locked states (#2601) (57fcb08)
• use namespace imports for cheerio to be compatible with v1 (#2641) (f48296f)
• Use the correct mutex in memory storage RequestQueueClient (#2623) (2fa8a29)

Features

• globs & regexps for SitemapRequestList (#2631) (b5fd3a9)
• resilient sitemap loading (#2619) (1dd7660)

---

This release is pinning the dependency on cheerio to the last RC version, we might postpone the official support for v1 to next major, or at least wait for them to fix their stuff. Nice demonstration of how not to maintain popular open source projects 😞

v3.11.1

3.11.1 (2024-07-24)

Bug Fixes

• use getHTML in the shadow root expansion (#2587) (a244d62), closes #2583

v3.11.0

3.11.0 (2024-07-09)

Features

• add iframe expansion to parseWithCheerio in browsers (#2542) (328d085), closes #2507
• add ignoreIframes opt-out from the Cheerio iframe expansion (#2562) (474a8dc)
• Sitemap-based request list implementation (#2498) (7bf8f0b)