Conversation
3dd2197 to
1317b7e
Compare
| std::memcpy(key.data(), s.data(), key.size()); | ||
| return key; | ||
| } | ||
| typedef std::unordered_map<guid_key, V, guid_key_hash> map_type; |
There was a problem hiding this comment.
I would prefer if we just make this the type in the ifcopenshell::file/storage class and remove this file. I find that more illustrative of what we do and its limitations (std::array<char, 22>) than introducing a seemingly opaque type that also might hint at a numeric map instead of a textual map.
|
Thanks for your patience I think with the last comments addressed we're nearly at a place where important logic is reused rather than reintroduced :) |
|
It does seem that lazy got quite a bit slower, slower than non-lazy?
once we have the PR in a state of good human comprehension let's hypothesise what we can do optimising with a profiler and really see where the allocation of samples is different wrt to the fully inlined scan blob from the other PR. |
…ndex full_buffer_impl::size(), get(), get_u32() and get_u64() (and their mmap_impl twins) were defined out of line, so every character the lexer read crossed a call boundary with its own bounds check. Callgrind put the three at 5.6% of parse self time; inlining them lets the compiler hoist the checks out of the scanning loops, which is worth more than their own cost. Also reserve the streamer's inverse vector from the file size (about one record per 32 bytes of SPF on real models) and shrink it once the bulk load is sorted, so the doubling copies and the capacity slack go away. Parse time, C++ file constructor, 12-core Linux box: TXG 58 MB 1.35 s -> 1.15 s 210_King 148 MB 3.70 s -> 3.09 s OKgate22 232 MB 6.18 s -> 5.30 s Python ifcopenshell.open(): 1.40 -> 1.22, 3.70 -> 3.24, 6.27 -> 5.52 s. Memory unchanged. The removed exported symbols mean the Python wrapper must be rebuilt against this library. This commit was written by an AI coding tool and has not been verified by a human. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013wcN7XquTfUi4vsKQ4KchL
byguid_ was a std::map<std::string, ...>: a red-black node plus a heap-allocated 22-character string per rooted instance, and a lookup that walks ~18 levels of string comparisons on a 200k-entry file. guid_map keeps keys of up to 23 characters inline in an unordered_map node (every valid GlobalId is 22), and routes anything longer to an ordered map so invalid files still work. Same std::string-keyed interface as before. Parse, C++ file constructor, on top of the previous commits: TXG 58 MB 1.08 s -> 1.03 s 341 -> 335 MB 210_King 148 MB 2.80 s -> 2.72 s 836 -> 830 MB OKgate22 232 MB 4.25 s -> 3.94 s 1271 -> 1253 MB This commit was written by an AI coding tool and has not been verified by a human. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013wcN7XquTfUi4vsKQ4KchL
Each instance was four separate allocations: the instance_data record, the attribute array object it pointed at, that array's index bytes, and its slot storage. A 58 MB model made 10.4 million mallocs to load 918k instances, and massif attributed 99 MB of its 450 MB peak to malloc bookkeeping alone. variant_array now allocates the size byte, the per-slot type indices and the slots as one block, and instance_data holds the array in a std::optional instead of behind a pointer (an empty optional keeps the meaning the null pointer had: attribute storage constructed on the fly from the RocksDB backend). No ownership or lifetime changes; the same object owns the same data. Parse, C++ file constructor, on top of the previous commits: TXG 58 MB 0.96 -> 0.94 s steady 310 -> 262 MB peak 383 -> 335 MB 210_King 148 MB 2.74 -> 2.63 s steady 758 -> 630 MB peak 948 -> 820 MB OKgate22 232 MB 3.70 -> 3.61 s steady 1165 -> 971 MB peak 1447 -> 1252 MB mallocs while loading TXG: 10.39M -> 7.56M. This commit was written by an AI coding tool and has not been verified by a human. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013wcN7XquTfUi4vsKQ4KchL
Every peek(), get() and SWAR word read on a paged reader went through the
page cache's hash map and LRU list, so tokenizing through pages ran at a
fifth of the speed of the in-memory buffer. The reader now remembers the
page its cursor was last on and serves reads that fall inside it from the
pointer, revalidated against an eviction counter on the implementation
so a page that left the cache is never read through a stale pointer.
Reads that straddle a page boundary take the existing paths.
The whole tokenizer over each file, 64 KB pages, 64 cached (4 MB):
in-memory buffer paged before paged after
TXG 58 MB 200 MB/s 42 MB/s 142 MB/s
210_King 148 MB 181 MB/s 34 MB/s 118 MB/s
OKgate22 232 MB 197 MB/s 38 MB/s 128 MB/s
This is the step that makes reading in pages a candidate for the default
path rather than a fallback; the remaining gap is the page fetch itself.
This commit was written by an AI coding tool and has not been verified by
a human.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013wcN7XquTfUi4vsKQ4KchL
callgrind on the tokenizer showed SWAR::has_special_char and eq_mask compiled as calls, one per eight bytes; paged_file_impl::size() out of line behind every eof() and remaining(); and the cursor's page-cache check not inlined into peek() because it shared a function with the page fetch. The SWAR helpers are forced inline, size() is defined in the class, and cached_() is split into an inline check and an out-of-line refresh. TXG (58 MB), single thread: tokenizer 196 -> 204 MB/s in memory and 136 -> 193 MB/s through 64 KB pages; strict parse through pages 1.15 -> 0.97 s against 0.93 s in memory; lazy index pass over pages 218 -> 311 MB/s; lazy open 0.52 -> 0.43 s. This commit was written by an AI coding tool and has not been verified by a human. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013wcN7XquTfUi4vsKQ4KchL
spf_lexer::next() becomes next<Policy>(). full_tokens, the default, is what the parser has always had. index_tokens is what the lazy index needs: a string is ended but not decoded, and a number, enumeration or binary comes back as Token_LITERAL with only its position; names, keywords and operators are read as before. Each policy compiles to its own loop from the one implementation, so there is no second tokenizer. character_decoder gains skip(): the same state machine as the conversion with the collection compiled out, so an escape such as \S\' (an apostrophe as the page character) ends the string at the same byte under both policies. A byte-level scan would have ended it early. Also fixes a comment that follows a token without whitespace, ",/* x */", which skip_comment() never saw because the slash had been consumed. TXG (58 MB), single thread: tokenizing the whole file 194 MB/s with full_tokens, 249 MB/s with index_tokens; through 64 KB pages 196 and 205 MB/s. The parse itself is unchanged. This commit was written by an AI coding tool and has not been verified by a human. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013wcN7XquTfUi4vsKQ4KchL
The parser could not resolve a #name when it read it, because the instance may be defined further down the file, so it left the slot empty and appended (owner, attribute, name) to a side table that a second pass walked. The table held one entry per reference for the whole read: 64 MB on a 58 MB model, the high-water mark of opening. Now the reference stays where the tokenizer put it: the attribute slot holds the instance_reference, or the reference_or_simple_type aggregate for a list (mixed with inline typed values or not), until every instance has been read, and resolve_instance_references() walks each instance's slots and swaps names for instances. Ordering is what the tokenizer produced; nothing is re-derived. A missing name becomes null in a scalar and is dropped from an aggregate, as before; the error keeps its offset. The three transient alternatives are appended to the attribute pack and to argument_type in lock step and are never visible once a file is loaded. Simple type instances read inline (IfcPropertySetDefinitionSet) have their own slots, so their references need no diversion. The table remains for the header entities and for streaming consumers of instance_streamer::references(), which leave resolve_references_in_place off. TXG 58 MB / 210_King 147 MB / OKgate22 231 MB, single thread: time unchanged (1.05 / 2.69 / 4.99 s), memory after the parse 287 -> 274, 698 -> 654, 1086 -> 1036 MB, peak 400 -> 365, 965 -> 871, 1471 -> 1347 MB. This commit was written by an AI coding tool and has not been verified by a human. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013wcN7XquTfUi4vsKQ4KchL
…True) A lazy open reads the DATA section once with the tokenizer's index policy and builds what indexes the file: a shell per instance (name and declaration, no attribute array), the complete inverse index with attribute indices, the GlobalId map and the by-type lists. No attribute value is decoded. The first time an instance's attributes are touched, ensure_loaded() seeks the retained paged reader to the instance and runs the same load_attributes() the full parse runs, with inverse registration off, then resolves that instance's references from its own slots. A modified instance is materialised first, so writing works. There is no scanner of its own: the index pass consumes next<index_tokens>() and counts parentheses and commas on the operator tokens; a keyword where an instance should start, or a token the tokenizer rejects, stops the index and the file is parsed in full. The offset of each instance's attribute list is kept in one sorted vector that exists only in lazy mode, so a full parse pays nothing for it. Materialising from several threads at once is not safe. TXG 58 MB / 210_King 147 MB / OKgate22 231 MB, single thread: lazy open 0.61 / 1.73 / 2.86 s against the full parse's 1.05 / 2.69 / 4.99 s, at 141 / 374 / 534 MB against 274 / 654 / 1036 MB; reading one attribute of every instance afterwards costs a further 0.56 / 1.44 / 4.86 s. This commit was written by an AI coding tool and has not been verified by a human. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013wcN7XquTfUi4vsKQ4KchL
The DATA section is split into one chunk per thread and each worker runs the same per-instance reader as the serial parse over its own reader, storage, inverse records and simple-type list; the results are merged in file order, so instance order, GlobalId precedence and inverse records are identical to the serial parse. Reference resolution then splits over the same threads: each instance's slots are its own and the name table is complete and read-only by then. The default is one thread per core, capped at 16; IFCOPENSHELL_PARSE_THREADS or file::parse_threads() overrides it, and 1 parses as before. The instance headers are read by one loop, for_each_instance_header(), shared with the lazy index: it looks declarations up once per keyword, passes over a bypassed instance's attribute list and slides past a stray keyword the way the serial reader does (the lazy index therefore no longer falls back on one). Finding the split points is the one place that looks at raw bytes rather than tokens, because tokenizing the file serially first would leave nothing to parallelise. It applies three rules: a string starts and ends at a quote and cannot span a line, and a comment runs from /* to */; a split is a '#' that starts a line outside both. Getting a string's end wrong can only lose a candidate, never accept a wrong one, since no string contains a newline. The equality test puts a comment holding a fake instance and a string holding "/*" between the chunks. file_reader gains for_each_span(), which hands a byte range out span by span (one span for a buffer, one per page for the paged reader), and reopen(), a reader over the same file for another thread. TXG 58 MB / 210_King 147 MB / OKgate22 231 MB, 12 threads: 0.44 / 1.24 / 2.01 s against 1.07 / 2.75 / 5.12 s on one thread; memory after the parse within 1–4%, peak +5–4%. This commit was written by an AI coding tool and has not been verified by a human. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013wcN7XquTfUi4vsKQ4KchL
file::paged_reading(true), set before initialize(), runs the full parse (serial or parallel) through the paged reader with 64 KB pages and a 4 MB cache instead of reading the whole file into memory; the whole file is then never held. Every stage already reads through the reader, so nothing else changes. The equality test now runs the same file paged, serially and with five workers each holding its own page cache. TXG 58 MB / 210_King 147 MB / OKgate22 231 MB: one thread 1.09 / 2.89 / 5.25 s against 1.07 / 2.75 / 5.12 s in memory, twelve threads 0.48 / 1.27 / 1.99 s against 0.44 / 1.24 / 2.01 s; peak memory 311 / 727 / 1119 MB against 365 / 871 / 1347 MB, that is, down by the size of the file. Whether this should become the default is a decision the numbers on the PR are meant to inform. This commit was written by an AI coding tool and has not been verified by a human. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013wcN7XquTfUi4vsKQ4KchL
-fastproxy binds each wrapped method directly to its compiled function instead of going through a generated Python def, and -fastdispatch shortens overload dispatch. Reading is_a(), id() and an attribute of every one of TXG's 918k instances takes 2.15 s instead of 2.36 s (-8%); the tight loop over 133 walls reading GlobalId, Name and is_a() twenty times over 9 ms instead of 11 ms. validate_stub compared the stub against the wrapper's def signatures by parsing both files; a fast-proxy wrapper has assignments instead of defs. It now rebuilds the signature from the compiled function's autodoc docstring (the C++ prototype) with the rule SWIG itself uses: one prototype whose defaults are Python literals becomes named parameters, several prototypes or a non-literal default such as an enum become *args. Checked against every def of a wrapper built without -fastproxy: 773 of 773 signatures rebuild identically. The autodoc feature moves above the SWIG library includes so the iterator and container classes carry prototypes too. This commit was written by an AI coding tool and has not been verified by a human. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013wcN7XquTfUi4vsKQ4KchL
…ying it Inside an attribute list the index looks only at operators and names, so a third tokenizer policy, attribute_tokens, returns a keyword (an inline typed value such as IFCLABEL), an enumeration or a binary as Token_LITERAL without copying its text; only a name's digits are kept. The instance headers still go through index_tokens, which keeps the keyword. Same next(), one more compile-time branch. Found by callgrind on the lazy open (see the PR): string-pool access and keyword text copying for tokens the index never read. Whole-file tokenizing of TXG (58 MB) with the index policy 265 MB/s -> 293 MB/s; lazy open TXG / 210_King / OKgate22 0.58 / 1.73 / 2.86 s -> 0.57 / 1.65 / 2.78 s. Small: the per-token call is the larger cost, which the next commit addresses by splitting the work over threads. This commit was written by an AI coding tool and has not been verified by a human. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013wcN7XquTfUi4vsKQ4KchL
On a file large enough the DATA section is split at the same boundaries the parallel parse uses (chunk_bounds(), now shared) and each chunk is indexed by its own worker with its own paged reader, lexer, shells, offsets, GlobalIds and inverse records; the results are merged in file order, so instance order, GlobalId precedence and inverse records are identical to the serial index. The serial index is the same code run on one chunk. The name table is reserved before the merge, which also helps the serial case. The default thread count is the one the full parse uses. TXG 58 MB / 210_King 147 MB / OKgate22 231 MB, lazy open: 12 threads 0.37 / 1.01 / 1.34 s against 0.55 / 1.59 / 2.11 s on one thread and 0.44 / 1.17 / 1.91 s for the default (parallel full) open; memory after the open +15 to +50 MB at 12 threads for the workers' page caches. The equality test now opens the 12 MB replicated fixture lazily with five workers and compares it instance by instance with the serial parse. This commit was written by an AI coding tool and has not been verified by a human. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013wcN7XquTfUi4vsKQ4KchL
|
(AI-written, not verified by Dion.) guid_map.h is gone. The GlobalId index is now the spelled-out Lazy slower than non-lazy. Two things were mixed in that table: the default column is the 12-thread parallel parse and the lazy open was single-threaded; and lazy did get slower than #9473 by exactly the difference between the byte scanner and the tokenizer. I profiled it as you suggested before changing anything. Callgrind on the lazy open of TXG, this branch against the byte scanner: the index pass is 0.28 s through
(a) and (b) recover about a fifth of the tokenizer gap: the per-token call is the cost, not what the token carries, and while All fourteen models, this branch, C++ (memory after the open):
From Python, lazy is now 4–11% faster than the default open on every model at 55–70% of its memory. Full tables, the per-commit code and test line counts, and the profile numbers are in the PR body; 13 commits, net +1982 −176 lines (+1542 −164 outside tests). |
What I think would be interesting to benchmark is on what token types the difference is most pronounced. We can also skip converting the reals to float for example on the lazy path (if not already). But it could well be that a major driver for the performance difference is the string handling, for cases such as (as you noted):
So maybe benchmark the difference on files with:
Because profiling otherwise averages everything out. This will indicate more clearly where the difference exists. As I said I have a suspicion its the strings and the character decoder that kicks in to determine the string literal - in case of complex encoding, which is often not there. Maybe we can SWAR or otherwise handle this more efficiently. Is see you already added back character_decoder::skip() (which we previously had) which is a step in the right direction. I don't know if the consumer model matches the structure well, but otherwise maybe a lexer::next_n(vector&) call can also be investigated - or maybe even just exhaust tokenize one page at a time. Or maybe omit tokens in next() that we don't care about for lazy. Because you can keep the structural ones such as |
|
(AI-written, not verified by Dion.) Measured per token type as you suggested, on 50 MB synthetic files (IfcPerson with doubled quotes, The columns: byte scan is the standalone scanner from #9473, for reference. The other three are the one
Two things fall out. Strings are not the driver: the string-heavy file is the fastest per byte and the cheapest per token, because I then tried your last suggestion, passing over the non-identifier tokens inside So the honest reading is that the gap to the byte scanner is the per-token structure itself, not any token type, and closing it single-threaded would need the attribute structure (depth, commas) folded into the skip loop, which is the byte scanner in another place. I would rather not do that. What the branch does instead is chunk the index over threads (commit 13), which puts lazy under the default open on every model; the fourteen-model column is in the previous comment and the PR body. For the record, the synthetic files also confirm the decoder cost you had in mind is real but small: with strings decoded (full tokenizer) the string file is the slowest at 166 MB/s; with them only ended it is the fastest. |
|
You're right. Never realized that ifc models are so sparse on literal values. On Duplex_A I get to 12%.
But I think the fact that all tokens are more or less in the same order of magnitude tells us that the overhead is likely in the inability to optimise across the next() boundary or the allocation of the token object etc. So let's go ahead with the inversion to a Consumer model.
Make the function body exactly that of lexer::next() but wrap it with a while loop and do not allocate Token objects but call the appropriate function - other than that exact equal. Then remove next(). struct lazy_index_consumer {
static constexpr bool decode_strings = false;
static constexpr bool decode_numbers = false;
// ...
uint32_t depth = 0;
uint32_t attribute = 0;
FORCEINLINE void operator_(size_t pos, char c) {
switch (c) {
case '(':
++depth;
break;
case ')':
--depth;
break;
case ',':
if (depth == 1)
++attribute;
break;
}
}
FORCEINLINE void identifier(size_t pos, uint32_t id) {
// directly build inverse index
}
};
struct full_parse_consumer {
static constexpr bool decode_strings = true;
static constexpr bool decode_numbers = true;
IFC_FORCEINLINE void operator_(size_t, char);
IFC_FORCEINLINE void identifier(size_t, uint32_t);
IFC_FORCEINLINE void string(size_t, std::string_view);
...
};Also a completely different subject, but since SPF structure is so regular and IFC entity types follow typical usage patterns we should really try and investigate profile guided optimizations. If a local test confirms this helps, please investigate a separate PR where we build this into the build_win / build_rocky / build_osx github action build pipelines. |
…onsumer The body of next() becomes spf_lexer::scan(Consumer&), the same code wrapped in a loop that hands each token to the consumer's callbacks (operator_, identifier, string, keyword, enumeration, binary, boolean, integer, real, literal) instead of building a token object; each callback returns whether to go on. The consumer's constexpr flags say what is decoded: decode_strings, decode_values, keep_keywords. It lives in spf_scan.h, with the SWAR helpers and number parsing it needs, so a consumer inlines into the loop. next<Policy>() is kept as the consumer that stops after one token: the attribute reader, header parser and streamer pull tokens recursively and stay as they are. The lazy index is now attribute_consumer: depth and attribute index from the operators, every name straight into the inverse index, the bounds of the first attribute if it is a string, done at the closing semicolon. The attribute_tokens policy it replaces is gone. Tokenizing 50 MB files with nothing decoded, in memory: index policy through next() 214–247 MB/s, scan() with the inlined consumer 314–403 MB/s (TXG 247 → 345); through 64 KB pages 245–284 MB/s. Lazy open on one thread TXG / 210_King / OKgate22 0.55 / 1.59 / 2.11 s → 0.52 / 1.57 / 2.05 s. The full tokenizer through the adapter is unchanged (TXG 202–210 MB/s against 195–219 before), as is the strict parse. This commit was written by an AI coding tool and has not been verified by a human. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013wcN7XquTfUi4vsKQ4KchL
|
(AI-written, not verified by Dion.) The consumer model is in as commit 14. The body of One deviation from what you wrote, and I'd rather state it than hide it: Tokenizing 50 MB files with nothing decoded, in memory: the index policy through PGO, tested locally as you suggested: IfcParse and the schema plugins built with
So 6–13% on ordinary models, about 4% on one dominated by long real lists, and it holds out of sample. Two things to know before putting it in the pipelines: a first attempt showed no gain at all because the |
|
(AI-written, not verified by Dion.) A clarification on commit 14, so the numbers are not misread: it is an architecture change, not a performance one. Same fourteen models, the full matrix before the commit against the one after:
Run-to-run scatter on this machine is about ±3%, so the open times are flat and the strict parse is unchanged by construction ( What the commit does deliver is the shape you asked for: one tokenizer body, the lazy index inlined into it as a consumer instead of a policy returning a token per call, and one policy fewer. The PR body's profile section says the same. The end-to-end before/after tables at the top of the PR body are unchanged by it. |
The inverse index is sorted once after a parse (and after a parallel merge). std::sort on 1.7–5 million 12-byte records was the largest serial phase left after the parse itself. sort_records() now does a stable LSD radix sort on referenced_id, 11 bits per pass and as many passes as the largest id needs, then applies record_less within each run of equal ids, so the order is exactly what std::sort produced. Inputs under 4096 records still use std::sort. Experiment on top of the series; measured in isolation against the previous commit, five models, best of three (strict parse 1 / 12 threads, lazy open 1 / 12 threads): TXG 58 MB −7% / −16% / −10% / −19%, 210_King 147 MB −8% / −17% / −15% / −22%, OKgate22 231 MB −6% / −12% / −11% / −17%, a 107 MB model −8% / −18% / −12% / −19%, a 523 MB model of few large instances −3% / −4% / 0% / −5%. This commit was written by an AI coding tool and has not been verified by a human. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013wcN7XquTfUi4vsKQ4KchL
After the parallel parse (and the parallel lazy index) the main thread did, per instance, the by-type list push, the GlobalId extraction and the name-table insert, then concatenated every worker's inverse records and sorted the lot: on a 12-core box that serial merge was most of the open. Now each worker sorts its own records, builds its own by-type lists and reads its own GlobalIds; the main thread appends the by-type lists, inserts names and GlobalIds, and merges the sorted runs pairwise (inverse_index::merge_sorted, O(n log k)) instead of sorting again. Instance order, GlobalId precedence and record order are unchanged. The workers sort their runs in place (inverse_index::sort_in_place, no radix buffer, no shrink): merge_sorted() copies the runs anyway, and a buffer allocated on a worker's arena stays there after the open, which showed as 3–10% more resident memory after a 12-thread open until the runs sorted in place; peak memory at 12 threads is one transient copy of the records higher (the merge), single-threaded it is unchanged. Experiment on top of the series; measured in isolation against the previous commit, five models, best of three: single-threaded within ±2% (by construction); 12 threads, strict / lazy: TXG −16% / −25%, 210_King −23% / −26%, OKgate22 −25% / −20%, a 107 MB model −25% / −23%, a 523 MB model of few large instances −9% / −7%. Together with the radix sort: strict 12 threads −20% to −29%, lazy 12 threads −31% to −35% on the ordinary models. This commit was written by an AI coding tool and has not been verified by a human. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013wcN7XquTfUi4vsKQ4KchL
|
(AI-written, not verified by Dion.) Two more commits at the end of the branch, 15 and 16. They are experiments, kept separate so they are easy to drop: the series up to commit 14 is the one under review, and we assume 15 and 16 are discarded unless you want them. Nothing above them changed and the numbers in the PR body up to commit 14 are as they were; this comment's numbers are on top of those. Why these two. With the tokenizer inlined, a lazy open of TXG on one thread (0.54 s) breaks down roughly as follows, from the callgrind profile and the isolated timings:
Everything below the first row was serial even on 12 threads, which is why the lazy open only went from 0.54 s to 0.37 s on twelve cores, and the same serial merge is what held the strict parse at 2–3x on 12 cores: of its 0.44 s on TXG, roughly 0.3 s was the merge and the sort. So the two levers with contained changes were the sort and the merge. Commit 15, radix sort of the inverse records.
Commit 16, sort and index per worker, merge sorted runs. Each parser (and index) worker sorts its own records, builds its own by-type lists and reads its own GlobalIds; the main thread appends the lists, inserts names and GlobalIds, and merges the sorted runs pairwise (
Single-threaded it is neutral by construction. A third experiment, deferring the lazy GlobalId map to first use, gained 0–8% on the 12-thread lazy open only and is not included. Both together, against commit 14, all fourteen models:
Memory. Single-threaded nothing changes. At 12 threads the peak is one transient copy of the records higher, for the merge: +1–4% strict, +0–10% lazy (the 787 MB model: 2684 → 2863 MB). Resident memory after the open is within ±2% once the allocator's free pages are trimmed, and 0–7% higher before trimming, which is glibc keeping the freed per-worker runs on the worker arenas ( If you would rather keep the PR at commit 14, say so and I will drop these two; they stand on their own and can come back later. |
2 participants
This is the fresh series asked for in #9473: one tokenizer, minimal commits in the order (1) inlining and minimal optimisations, GlobalId map, heap allocation reduction; (2) template context on the existing tokenizer; (3) unresolved reference storage and lazy loading; (4) multithreading; plus the SWIG fast-proxy with
validate_stubtaught to read signatures from the compiled functions, two commits from profiling the lazy open, and the tokenizer restructured asscan(Consumer&)withnext()as its one-token consumer (below). It replaces #9473, which stays open only as the reference for the numbers posted there. 14 commits; net +2427 −415 lines, of which +1950 −403 outside tests (#9473 added 2169). Two further commits, 15 and 16, are experiments on the serial phases and are assumed to be dropped unless wanted; they are described in their own section at the end and none of the numbers below include them.dense_id_mapis not included.guid_map.his gone as reviewed: the GlobalId index is the spelled-outstd::unordered_map<std::array<char, 22>, ...>on the storage class.Before and after
C++
fileconstructor on fourteen models: three public (TXG, 210_King, OKgate22) and eleven supplied privately for benchmarking, identified by size only. Wall clock on a 12-core Linux box, one run each; memory is anonymous RSS after the parse, peak is VmHWM; percentages are against v0.9.0 head (148ff02) in the same row. Every run logged zero parser messages.Python
ifcopenshell.openon the same fourteen models, wall clock and RSS growth over the bare interpreter as a multiple of file size. This PR's default open is the parallel strict parse; v0.8.5 is the last release; v0.9.0 head is a wrapper built from 148ff02 with the same options.The commits
Lines are per commit, added and removed, with the tests counted separately so the cost of each item is visible.
std::unordered_map<std::array<char, 22>, ...>on the storage, withvariant_mapconverting fromstd::stringat the file interface; only a 22-character GlobalId can be stored or foundstd::optionalstoragesize()and the page-cache check forced inline; found with callgrindspf_lexer::next<Policy>():full_tokensis the parser,index_tokensends strings without decoding them and passes over numbers, enumerations and binaries;character_decoder::skip()is the same state machine with collection compiled out; also fixes a comment that follows a token without whitespaceinstance_reference/reference_or_simple_typeaggregate until every instance is read, mixed lists included; no side table for the parsenext<index_tokens>(); offsets in one lazy-only sorted vector, nothing added toinstance_datafor the full parse-fastproxy -fastdispatch;validate_stubrebuilds signatures from the compiled functions' docstringsscan(Consumer&)inspf_scan.h, callbacks inlining into the loop;next()kept as the one-token consumer for the pull parser; the lazy index is a consumerinverse_index::sort_records(): LSD radix sort onreferenced_id,record_lesswithin each run of equal ids, same order as beforeinverse_index::merge_sorted)Each commit message carries its own numbers on the three public models.
One tokenizer
There is no byte-level scanner left. The lazy index consumes
next<index_tokens>()and counts parentheses and commas on the operator tokens; the parallel chunker and the lazy index share one instance-header loop,for_each_instance_header(), which looks declarations up once per keyword, passes over a bypassed instance's attribute list and slides past a stray keyword the way the serial reader does. Under both policies a string ends at the byte the decoder's state machine says it ends at; the equivalence test includes'',\X2\and\S\', an apostrophe used as the page character, which any quote-counting byte scan (including the one in #9473) ended early.The one place that looks at raw bytes is the split-point scan for parallel parsing, because tokenizing the file serially to find the split points would leave nothing to parallelise. It applies three rules: a string starts and ends at a quote and cannot span a line, and a comment runs from
/*to*/; a split is a#that starts a line outside both. Getting a string's end wrong can only lose a candidate, never accept a wrong one, since no string contains a newline. The equality test puts a comment holding a fake instance and a string holding/*between the chunks. Comments in the DATA section therefore no longer force the serial path.Throughput of the stages
Single thread, whole file. The index policy is the tokenizer with strings ended but not decoded and values passed over; the paged columns read through 64 KB pages with a 4 MB cache.
Reading across: the index policy is 20–35% faster than the full tokenizer; through pages both run at 90–105% of their in-memory speed; the full parse through pages costs a few percent on one thread and is level at 12 threads, with peak memory down by the size of the file. A lazy open on one thread is 50–60% of the strict single-thread parse and holds 130–160 bytes per instance; on 12 threads it is under the default (parallel full) open on every model. Reading every attribute afterwards costs about the strict parse again, so lazy is for touching a fraction of the model.
What lazy loading is
One pass over the DATA section through the tokenizer's index policy builds everything that indexes the file: a shell per instance (name and declaration, no attribute array), the complete inverse index with attribute indices, the GlobalId map and the by-type lists. No attribute value is decoded. The first time anything touches an instance's attributes,
ensure_loaded()seeks the retained paged reader to the instance and runs the sameload_attributes()the full parse runs, with inverse registration off, then resolves that instance's references from its own slots. A modified instance is materialised first, so writing works. The offset of each instance's attribute list lives in one sorted vector that exists only in lazy mode. Materialising from several threads at once is not safe.Where the lazy open's time went, and what was done about it
Callgrind on the lazy open of TXG, on this branch against the byte scanner of #9473: the index pass is 0.28 s through
next<index_tokens>()against 0.055 s for the byte scan, and the rest of a lazy open (shells, inverse sort, GlobalIds) is the same 0.33 s in both. The samples in the tokenizer: 37% in the index-policynext()itself, 6.7% infile_reader::peek()as a non-inlined call per byte from the whitespace skip, about 5% copying keyword and enumeration text the index never reads, 2% parsing names. The byte scanner did none of that: seven characters through one SWAR mask. Roughly 17 ns per token times 13 million tokens is the gap. Three hypotheses, each measured on the three public models (TXG / 210_King / OKgate22):(a) and (b) recover about a fifth of the tokenizer gap: the per-token call is the cost, not what the token carries (measured per token type on synthetic files: 9–16 ns per token whatever the token is, strings the cheapest). (c) is what puts lazy back under the default open. Skipping non-structural tokens inside
next()was also tried and reverted: on real files the structural tokens and names are the majority, so it changed nothing measurable.Commit 14 then inverts the tokenizer as asked: the body of
next()becomesscan(Consumer&)inspf_scan.h, handing each token to the consumer's callbacks (which inline into the loop) instead of building a token;next<Policy>()is kept as the consumer that stops after one token, so the recursive pull parser is untouched; the lazy index isattribute_consumer. Index pass, nothing decoded, in memory: 214–247 MB/s throughnext()→ 266–343 MB/s throughscan()(the byte scanner it replaces: 700–1300 MB/s); lazy open on one thread TXG / King / OKgate22 0.55 / 1.59 / 2.11 → 0.54 / 1.57 / 2.07 s. The full tokenizer through the adapter is unchanged. End to end this commit is an architecture change, not a performance one: on the fourteen models the lazy open moved by −1% (1 thread) and +2% (12 threads) on average, within run-to-run noise, the strict parse and memory not at all; the tokenizer pass is a small share of a lazy open and the strict parse still pulls tokens through the adapter.SWIG fast-proxy
-fastproxybinds each wrapped method directly to its compiled function instead of a generated Python def;-fastdispatchshortens overload dispatch. Readingis_a(),id()and an attribute of every one of TXG's 918k instances: 2.36 s → 2.15 s (−8%).validate_stubcompared the stub against the wrapper's def signatures; a fast-proxy wrapper has assignments, so it now rebuilds each signature from the compiled function's autodoc docstring with the rule SWIG itself uses (one prototype with literal defaults → named parameters; overloads or a non-literal default such as an enum →*args), checked against every def of a non-fast-proxy wrapper: 773 of 773 identical. The autodoc feature moves above the SWIG library includes so the iterator and container classes carry prototypes too.Experiments on top: commits 15 and 16
Kept at the end of the branch so they are easy to drop. After commit 14 the tokenizer pass parallelises, and what is left of a 12-thread open is the serial merge: the inverse-record sort, the by-type lists, the name and GlobalId tables. Commit 15 replaces the introsort of the inverse records by a radix sort with the same resulting order; commit 16 moves the sort, the by-type lists and the GlobalId decode into the workers and merges sorted runs. Single-threaded, 16 is neutral by construction and 15 is the sort alone. Same fourteen models, same bench, commit 14 → commit 16 (the tables above stay as measured at commit 14):
Single-threaded nothing changes. At 12 threads the peak is one transient copy of the records higher, for the merge: +1–4% strict, +0–10% lazy (the 787 MB model: 2684 → 2863 MB). Resident memory after the open is within ±2% once the allocator's free pages are trimmed, and 0–7% higher before trimming, which is glibc keeping the freed per-worker runs on the worker arenas (
MALLOC_ARENA_MAX=1removes the difference). A first form of commit 16 ran the radix sort inside the workers; its buffers stayed on those arenas and resident memory after a 12-thread open was 3–10% higher, so the workers now sort their runs in place withstd::sort, which costs nothing at 12 threads (the runs are a twelfth of the records each) and keeps the radix sort for the single-threaded paths. A third experiment, deferring the lazy GlobalId map to first use, gained 0–8% on the 12-thread lazy open only and is not included.Verification
BUILD_IFCOPENSHELL_PARSE_TESTS=ON): token-for-token equivalence of the two policies including the escapes above; references resolved in place for scalars, lists, nested lists, mixed selects, missing and bypassed names; a lazily opened file against the full parse, per-instanceto_string(), inverse counts and GlobalId lookups, on the fixture and on an inline file, plus the stray-keyword and fallback cases; serial, 5-thread, paged and paged 5-thread equality on a 12 MB replicated fixture with comments and/*-strings between the chunks.test/apitree and the newtest/test_lazy.py: 2063 passed, 31 skipped, with the default parallel parse and the fast-proxy wrapper; black and ruff clean. Four existing tests looked walls up by GlobalIds such as "id1"; they now use real 22-character ids, andtest_file.pyadditionally checks that a short key is rejected.ensure_loaded()calls added in the serializer, paths otherwise untouched), Windows/macOS, Bonsai (fast-proxy changes what the wrapper's methods are at the Python level: assignments instead of defs, so anything that introspects them the wayvalidate_stubdid would need the same change). The Python wrapper must be rebuilt against the library.🤖 Generated with Claude Code
https://claude.ai/code/session_013wcN7XquTfUi4vsKQ4KchL