Commits

Breadcrumbs

History for

onmaster

User selector

All users

All time

Commit History

Commits on Jul 29, 2025

• fix: Fix retry count to not count the original request (#1328)

  Show description for 74fa1d9
  Pijukatel

  authored
  74fa1d9

  Copy full SHA for 74fa1d9

• fix: Fix memory estimation not working on MacOS (#1330)

  Show description for ab020eb
  Pijukatel

  authored
  ab020eb

  Copy full SHA for ab020eb

Commits on Jul 25, 2025

• refactor!: Replace `HttpxHttpClient` with `ImpitHttpClient` as default HTTP client (#1307)

  Show description for c803a97
  Mantisus

  authored
  c803a97

  Copy full SHA for c803a97

Commits on Jul 23, 2025

• feat: Persist RequestList state (#1274)

  Show description for cc68014
  janbuchar

  authored
  cc68014

  Copy full SHA for cc68014

Commits on Jul 17, 2025

• feat: Prevent overloading system memory when running locally (#1270)

  Show description for 30de3bd
  janbuchar

  authored
  30de3bd

  Copy full SHA for 30de3bd

Commits on Jul 15, 2025

• chore: Refactor `HttpClient` implementations tests (#1299)

  Show description for 47380ef
  Pijukatel

  authored
  47380ef

  Copy full SHA for 47380ef

• feat: add `ImpitHttpClient` http-client client using the `impit` library (#1151)

  Show description for 0d0d268
  

  Mantisus

  and

  vdusek

  authored
  0d0d268

  Copy full SHA for 0d0d268

Commits on Jul 11, 2025

• feat: Add basic `OpenTelemetry` instrumentation (#1255)

  Show description for a92d8b3
  

  Pijukatel

  and

  vdusek

  authored
  a92d8b3

  Copy full SHA for a92d8b3

Commits on Jul 10, 2025

• refactor!: Change method `HttpResponse.read` from sync to async (#1296)

  Show description for 83fa8a4
  Mantisus

  authored
  83fa8a4

  Copy full SHA for 83fa8a4

Commits on Jul 9, 2025

• feat: Add crawlee-cli option to skip project installation (#1294)

  Show description for 4d5aef0
  Pijukatel

  authored
  4d5aef0

  Copy full SHA for 4d5aef0

Commits on Jul 7, 2025

• chore: Fix flaky `test_memory_estimation` (#1289)

  Show description for c319f8a
  Pijukatel

  authored
  c319f8a

  Copy full SHA for c319f8a

Commits on Jul 3, 2025

• refactor!: Split `BrowserType` literal into two different literals based on context (#1070)

  Show description for 72b5698
  Pijukatel

  authored
  72b5698

  Copy full SHA for 72b5698

Commits on Jul 2, 2025

• feat: add periodic status logging and `status_message_callback` parameter for customization (#1265)

  Show description for b992fb2
  Mantisus

  authored
  b992fb2

  Copy full SHA for b992fb2

• chore: adapt to ruff v0.12 (#1283)

  Show description for 4da38c2
  vdusek

  authored
  4da38c2

  Copy full SHA for 4da38c2

• chore: Bump minimum Python version to 3.10 (#1281)

  Show description for e5d0fdf
  vdusek

  authored
  e5d0fdf

  Copy full SHA for e5d0fdf

Commits on Jul 1, 2025

• feat: add utility for load and parse Sitemap and `SitemapRequestLoader` (#1169)

  Show description for 66599f8
  Mantisus

  authored
  66599f8

  Copy full SHA for 66599f8

• refactor!: Introduce new storage client system (#1194)

  Show description for de1c03f
  vdusek

  authored
  de1c03f

  Copy full SHA for de1c03f

Commits on Jun 26, 2025

• feat: Add `retire_browser_after_page_count` parameter for `BrowserPool` (#1266)

  Show description for 603aa2b
  Mantisus

  authored
  603aa2b

  Copy full SHA for 603aa2b

• fix: Use `perf_counter_ns` for request duration tracking (#1260)

  Show description for 9e92f6b
  

  Pijukatel

  and

  vdusek

  authored
  9e92f6b

  Copy full SHA for 9e92f6b

Commits on Jun 19, 2025

• feat: Add `stream` method for `HttpClient` (#1241)

  Show description for 95c68b0
  Mantisus

  authored
  95c68b0

  Copy full SHA for 95c68b0

Commits on Jun 11, 2025

• chore: Reduce flakiness of `test_timeout_in_handler` in CI (#1243)

  Show description for c25a0a5
  Pijukatel

  authored
  c25a0a5

  Copy full SHA for c25a0a5

Commits on Jun 10, 2025

• chore: fix `GuessedAtParserWarning` in BeautifulSoup (#1240)

  Show description for e7168b1
  vdusek

  authored
  e7168b1

  Copy full SHA for e7168b1

• chore: fix `PytestCollectionWarning` by adding `__test__=False` (#1236)

  Show description for 69fd7aa
  vdusek

  authored
  69fd7aa

  Copy full SHA for 69fd7aa

Commits on Jun 9, 2025

• fix: Use `PSS` instead of `RSS` to estimate children process memory usage on Linux (#1210)

  Show description for 436032f
  Pijukatel

  authored
  436032f

  Copy full SHA for 436032f

Commits on Jun 4, 2025

• fix: Fix `ClientSnapshot` overload calculation (#1228)

  Show description for a4fc1b6
  Pijukatel

  authored
  a4fc1b6

  Copy full SHA for a4fc1b6

Commits on May 27, 2025

• fix: respect `EnqueueLinksKwargs` for `extract_links` function (#1213)

  Show description for c9907d6
  Mantisus

  authored
  c9907d6

  Copy full SHA for c9907d6

Commits on May 26, 2025

• fix: add support non-GET requests for `PlaywrightCrawler` (#1208)

  Show description for dbb9f44
  Mantisus

  authored
  dbb9f44

  Copy full SHA for dbb9f44

Commits on May 20, 2025

• fix: fix match check for specified enqueue strategy for requests with redirect (#1199)

  Show description for d84c30c
  Mantisus

  authored
  d84c30c

  Copy full SHA for d84c30c

• fix: add `payload` to `SendRequestFunction` to support `POST` request (#1202)

  Show description for e7449f2
  Mantisus

  authored
  e7449f2

  Copy full SHA for e7449f2

Commits on May 12, 2025

• fix: Allow config change on `PlaywrightCrawler` (#1186)

  Show description for f17bf31
  mylank

  authored
  f17bf31

  Copy full SHA for f17bf31

Commits on May 2, 2025

• feat: add `on_skipped_request` decorator, to process links skipped according to `robots.txt` rules (#1166)

  Show description for bd16f14
  

  4 peopleauthored
  bd16f14

  Copy full SHA for bd16f14

Commits on Apr 30, 2025

• refactor: RecoverableState abstraction (#1172)

  Show description for de11d07
  janbuchar

  authored
  de11d07

  Copy full SHA for de11d07

Commits on Apr 29, 2025

• fix: fix handle error without `args` in `_get_error_message` for `ErrorTracker` (#1181)

  Show description for 21944d9
  

  Mantisus

  and

  Copilot

  authored
  21944d9

  Copy full SHA for 21944d9

Commits on Apr 28, 2025

• feat: Make timeout error log cleaner (#1170)

  Show description for 78ea9d2
  Pijukatel

  authored
  78ea9d2

  Copy full SHA for 78ea9d2

• chore: Update `test_enqueue_links` to use test server (#1177)

  Show description for c40f318
  Pijukatel

  authored
  c40f318

  Copy full SHA for c40f318