Skip to content

(wip) Logs: Internalization of Logs service #13521

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
pinzon wants to merge 9 commits into
base: main
Choose a base branch
from
Draft

(wip) Logs: Internalization of Logs service #13521

pinzon wants to merge 9 commits into from
+839 −4

Conversation

@pinzon
Copy link
Member

@pinzon pinzon commented Dec 13, 2025 •
edited
Loading

Motivation

This PR implements a new provider for the CloudWatch Logs service. This is done with the objective of internalize the implementation of the service and improve the storage of log events.

This new provider uses LocalStack Stores which improves the parity of the service and makes easier to implement changes for future service API updates.

Also, this new provider uses sqLite which improves the storage of log events by using disk storage instead of in-memory, fixing a memory leak issue that occurs in long rung instances of LocalStack when they use services that generate a lot of Log Events.

Changes

  • New Logs Provider
  • New LogGroup, LogStream and SubscriptionFilter definitions for the logs Store.
  • New database helper that handles database and table creation. Also manages the storage of log events

TODOs for this PR

  • Add integration with CW Metrics.
  • Fix events id generation.
  • Implement correctly provider selection.

@github-actions
Copy link

github-actions bot commented Dec 13, 2025 •
edited
Loading

Test Results - Preflight, Unit

22 982 tests   - 19   21 140 ✅  - 18   6m 4s ⏱️ -32s
     1 suites ± 0    1 842 💤  -  1 
     1 files   ± 0        0 ❌ ± 0 

Results for commit faced77. ± Comparison against base commit ee8518d.

This pull request removes 24 and adds 5 tests. Note that renamed tests count towards both. 
tests.unit.aws.test_connect.TestClientFactory ‑ test_typed_client_creation[iotanalytics]
tests.unit.aws.test_connect.TestClientFactory ‑ test_typed_client_creation[mediastore]
tests.unit.test_dns_server.TestDNSServer ‑ test_delete_operations_of_nonexistent_entries
tests.unit.test_dns_server.TestDNSServer ‑ test_dns_server_add_alias_lifecycle_with_ids
tests.unit.test_dns_server.TestDNSServer ‑ test_dns_server_add_host_lifecycle
tests.unit.test_dns_server.TestDNSServer ‑ test_dns_server_add_host_lifecycle_with_ids
tests.unit.test_dns_server.TestDNSServer ‑ test_dns_server_add_multiple_hosts
tests.unit.test_dns_server.TestDNSServer ‑ test_dns_server_alias_health_checks
tests.unit.test_dns_server.TestDNSServer ‑ test_dns_server_alias_lifecycle
tests.unit.test_dns_server.TestDNSServer ‑ test_dns_server_clear
…

tests.unit.aws.protocol.test_parser ‑ test_rpc_v2_cbor_timestamp_parsing
tests.unit.aws.test_service_router ‑ test_service_router_works_for_every_service[account-rest-json-GetGovCloudAccountInformation]
tests.unit.aws.test_service_router ‑ test_service_router_works_for_every_service[ce-json-ListCostCategoryResourceAssociations]
tests.unit.aws.test_service_router ‑ test_service_router_works_for_every_service[quicksight-rest-json-GetIdentityContext]
tests.unit.aws.test_service_router ‑ test_service_router_works_for_every_service[redshift-serverless-json-GetIdentityCenterAuthToken]

♻️ This comment has been updated with latest results.

@github-actions
Copy link

github-actions bot commented Dec 13, 2025 •
edited
Loading

Test Results (amd64) - Acceptance

7 tests  ±0   5 ✅ ±0   3m 6s ⏱️ -8s
1 suites ±0   2 💤 ±0 
1 files   ±0   0 ❌ ±0 

Results for commit faced77. ± Comparison against base commit ee8518d.

♻️ This comment has been updated with latest results.

@github-actions
Copy link

github-actions bot commented Dec 13, 2025 •
edited
Loading

Test Results (amd64) - Integration, Bootstrap

    5 files  ± 0      5 suites  ±0   2h 54m 1s ⏱️ + 19m 36s
5 555 tests +34  4 985 ✅ +20  558 💤 +2  12 ❌ +12 
5 561 runs  +34  4 985 ✅ +20  564 💤 +2  12 ❌ +12 

For more details on these failures, see this check.

Results for commit faced77. ± Comparison against base commit ee8518d.

♻️ This comment has been updated with latest results.

@github-actions
Copy link

github-actions bot commented Dec 13, 2025 •
edited
Loading

LocalStack Community integration with Pro

    2 files  ± 0      2 suites  ±0   1h 55m 15s ⏱️ + 1m 22s
5 153 tests +12  4 756 ✅ +11  397 💤 +1  0 ❌ ±0 
5 155 runs  +12  4 756 ✅ +11  399 💤 +1  0 ❌ ±0 

Results for commit faced77. ± Comparison against base commit ee8518d.

♻️ This comment has been updated with latest results.

@pinzon pinzon changed the title wip: Logs/experiment sqlite Logs: Internalization of Logs service Dec 18, 2025
@pinzon pinzon changed the title Logs: Internalization of Logs service (wip) Logs: Internalization of Logs service Dec 18, 2025
@pinzon pinzon added type: feature New feature, or improvement to an existing feature aws:logs AWS CloudWatch Logs semver: minor Non-breaking changes which can be included in minor releases, but not in patch releases docs: needed Pull request requires documentation updates notes: needed Pull request should be mentioned in the release notes labels Dec 18, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@steffyP steffyP Awaiting requested review from steffyP steffyP will be requested when the pull request is marked ready for review steffyP is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

aws:logs AWS CloudWatch Logs docs: needed Pull request requires documentation updates notes: needed Pull request should be mentioned in the release notes semver: minor Non-breaking changes which can be included in minor releases, but not in patch releases type: feature New feature, or improvement to an existing feature

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@pinzon