Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 14 additions & 6 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,20 @@
When navigating the codebase, look at relevant README.md for more project context.
- To gather more context beyond README.md:
- Look at the outstanding changes in the tree
- If on a branch check the last 2-3 to commits
- Look at relevant README.md in sub-folders

- If on a branch maybe check the last 2-3 to add commits for more context
- Look at the outstanding changes in the tree
- Write production quality code
- Make sure the code compiles
- Write production quality code.
- Adhere to rules in "Code Complete" by Steve McConnell.
- Adhere to rules in "The Art of Readable Code" by Dustin Boswell & Trevor Foucher.
- Make sure the code compiles.
- When adding code always ensure that tests cover the newly added code:
- Unit tests that validate for regular and exceptional inputs
- Use property based testing/model based testing/fuzzing when appropriate
- Integration tests for big platform-level features (in @python/tests)

At the start of every conversation, offer the user to run `scripts/claude.sh` to pull in shared LLM context files as unstaged changes. These should not be committed outside the `claude-context` branch.
- Add/update documenation and comments.
- Adhere to rules in "Bugs in Writing: A Guide to Debugging Your Prose" by Lyn Dupre.
- Adhere to rules in "The Elements of Style, Fourth Edition" by William Strunk Jr. & E. B. White

At the start of every conversation, offer the user to run `scripts/claude.sh` to pull in shared LLM context files as unstaged changes.
These should not be committed outside the `claude-context` branch.
2 changes: 1 addition & 1 deletion crates/dbsp/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ futures = { workspace = true }
tokio = { workspace = true, features = ["macros", "rt", "rt-multi-thread"] }
paste = { workspace = true }
seq-macro = { workspace = true }
derive_more = { workspace = true, features = ["add", "not", "from", "debug"] }
derive_more = { workspace = true, features = ["add", "add_assign", "sum", "not", "from", "debug"] }
dyn-clone = { workspace = true }
rand_chacha = { workspace = true }
tempfile = { workspace = true }
Expand Down
32 changes: 31 additions & 1 deletion crates/dbsp/src/circuit/metadata.rs
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,12 @@ pub const BLOOM_FILTER_MISSES_COUNT: MetricId =
pub const BLOOM_FILTER_HIT_RATE_PERCENT: MetricId =
MetricId(Cow::Borrowed("bloom_filter_hit_rate_percent"));
pub const BLOOM_FILTER_SIZE_BYTES: MetricId = MetricId(Cow::Borrowed("bloom_filter_size_bytes"));
pub const RANGE_FILTER_HITS_COUNT: MetricId = MetricId(Cow::Borrowed("range_filter_hits_count"));
pub const RANGE_FILTER_MISSES_COUNT: MetricId =
MetricId(Cow::Borrowed("range_filter_misses_count"));
pub const RANGE_FILTER_HIT_RATE_PERCENT: MetricId =
MetricId(Cow::Borrowed("range_filter_hit_rate_percent"));
pub const RANGE_FILTER_SIZE_BYTES: MetricId = MetricId(Cow::Borrowed("range_filter_size_bytes"));
pub const SPINE_BATCHES_COUNT: MetricId = MetricId(Cow::Borrowed("spine_batches_count"));
pub const SPINE_STORAGE_SIZE_BYTES: MetricId = MetricId(Cow::Borrowed("spine_storage_size_bytes"));
pub const MERGING_SIZE_BYTES: MetricId = MetricId(Cow::Borrowed("merging_size_bytes"));
Expand All @@ -161,7 +167,7 @@ pub const PREFIX_BATCHES_STATS: MetricId = MetricId(Cow::Borrowed("prefix_batche
pub const INPUT_INTEGRAL_RECORDS_COUNT: MetricId =
MetricId(Cow::Borrowed("input_integral_records_count"));

pub const CIRCUIT_METRICS: [CircuitMetric; 66] = [
pub const CIRCUIT_METRICS: [CircuitMetric; 70] = [
// State
CircuitMetric {
name: USED_MEMORY_BYTES,
Expand Down Expand Up @@ -289,6 +295,30 @@ pub const CIRCUIT_METRICS: [CircuitMetric; 66] = [
advanced: false,
description: "Hit rate of the Bloom filter.",
},
CircuitMetric {
name: RANGE_FILTER_SIZE_BYTES,
category: CircuitMetricCategory::State,
advanced: true,
description: "Size of the cached range filter in bytes.",
},
CircuitMetric {
name: RANGE_FILTER_HITS_COUNT,
category: CircuitMetricCategory::State,
advanced: false,
description: "The number of hits across all range filters. The hits are summed across the range filters for all batches in the spine.",
},
CircuitMetric {
name: RANGE_FILTER_MISSES_COUNT,
category: CircuitMetricCategory::State,
advanced: false,
description: "The number of misses across all range filters. The misses are summed across the range filters for all batches in the spine.",
},
CircuitMetric {
name: RANGE_FILTER_HIT_RATE_PERCENT,
category: CircuitMetricCategory::State,
advanced: false,
description: "Hit rate of the range filter.",
},
CircuitMetric {
name: RETAINMENT_BOUNDS,
category: CircuitMetricCategory::State,
Expand Down
1 change: 1 addition & 0 deletions crates/dbsp/src/storage.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ pub mod backend;
pub mod buffer_cache;
pub mod dirlock;
pub mod file;
pub mod filter_stats;
pub mod tracking_bloom_filter;

use fdlimit::{Outcome::LimitRaised, raise_fd_limit};
Expand Down
Loading
Loading