ci(mirror): 每资源计时 + 2m 上传上限(超时 warn 并跳过) - #219
Merged
Merged
Conversation
v0.0.94 发布时 publish-ecosystem 被 30min job timeout 杀掉,日志里看不出是 哪个资产在慢 —— 事后查是 gtc 跨境传 linux-x86_64(8.4MB)/linux-aarch64 (30.5MB)两个大件卡住,最后靠本地 gtc 手工补齐。 - **每资源计时**:上传返回即打印 `<github|gitcode> <资产> (<大小>) uploaded in Ns`; 已 serving 的仍先 probe → 打印 skipping,不重传(原有行为,补上 host 标签)。 - **2m 上限**(`MIRROR_UPLOAD_TIMEOUT`,默认 120s):超时打印 WARN + **放弃该资产**, 后续轮次不再重试。 - 每条 mirror leg 结束打印耗时;失败时 hint 出手工补传命令。 **为何这个 cap 与 v0.0.90 事故不同**:那次 `timeout 300` 杀掉慢但**在推进**的上传 后**重传**,每次从字节零开始,永不收敛,把 20min job 预算拖垮。本 cap 的安全性来自 **超时即放弃、绝不重试**;完整性 gate 仍是唯一 pass/fail,漏传会响亮报错而不是 静默发半个镜像。脚本顶部的血泪注释已按此改写。 验证: - 真实 0.0.94 镜像干跑 → 16 资产两端全 200、已存在全部 skip、206s 通过。 - stub 双分支(PATH 上的真可执行文件,非 shell function —— timeout 对函数无效): 快传 → `github big.tar.gz (2.9MB) uploaded in 1s`; 慢传 → `WARN: gitcode ... exceeded the 2s cap after 2s — skipping`。
按 review 定档: - `MIRROR_UPLOAD_TIMEOUT` 120 → **180s**。mcpp 最大资产约 30MB(单包不超 100M), 3 分钟还没传完不是「慢」是「卡住」,不值得继续占 CI —— 跳过并由本地补传 (gate 会打印确切的补传命令)。 - Mirror 步骤加 **`timeout-minutes: 10`** 硬顶。健康的 run 远在此之内;10min 是 兜底,防止某一端病态时烧光 job 预算(v0.0.94 就是 30min 挂死且零可见性)。
Sunrisepeak
added a commit
that referenced
this pull request
Aug 19, 2026
…plementation plan (#461) * docs: deep review of #455-#459 and the bare-metal ecosystem Covers what the five PRs did, the four releases they produced, and the ecosystem work alongside them (xim-pkgindex #651/#652/#653, mcpp-index #219/#220, two new mcpplibs repos), assessed on architecture, compatibility, simplicity, stability and cross-platform. Every claim carries its source — a PR number, a file, or the command that measured it — because the point of the document is to be checkable rather than summarised. Three of its assertions were re-verified against the tree while writing it. The uncomfortable half is deliberate: * five defects in this round were self-inflicted, two of them found only AFTER a release; * three test criteria were themselves wrong — green tests that were not testing the thing they named; * the largest architectural error (packages declaring the target's C library) was found by review, not by me, and I had written 'this cannot be done' about the std subset while my own research document had measured that it could. * docs: user-facing bare-metal scenarios, and the package-naming/ownership answers Adds a scenario document — what a user types, what they see, and what they no longer have to write — with the commands and outputs taken from a real run of the released binary rather than sketched. Folds two review questions into the analysis: * `picolibc-riscv` / `qemu-riscv` carrying the target in the name follows xim's existing rule, which is visible across the index: aarch64-linux-musl-gcc, riscv64-linux-musl-gcc, mingw-cross-gcc, musl-gcc. The NAME carries the target; the `archs` axis carries the host, which is why `llvm` has no target in its name at all. * picolibc belongs on the xim side, resolved at build time from the target's row. It is not importable, it is chosen by the target rather than by a dependency graph, and every other C library in the ecosystem — glibc, musl, musl-cross-make — is already there; mcpp-index carries zero libc packages. Moving it would put the libc back into the package graph, undoing #459. Both answers came with a gap worth recording: `[target.X]` has toolchain, linkage, runner and cxx_runtime but no `sysroot`, so a project cannot swap picolibc for newlib today. * docs: openkal design — a two-sided kernel ABI specification Written from the discussion that followed the #455-#459 review, and grounded in what is measurable today rather than sketched: every claim marked with a source was verified against the shipped payloads (llvm 22.1.8, gcc 16.1.0, picolibc 1.8.12) while writing. The load-bearing decisions: * Partitioning by RESOURCE KIND, not by which standard-library facility it lights up. The latter couples a kernel ABI to C++ and to today's library, and inverts the dependency — it is the same mistake POSIX made for C and WASIp1 made for POSIX, one generation further on. 'Lights up std::X' is demoted to the upward admission criterion, which is where it belongs. * An opaque one-word handle, because that is what makes openkal indifferent to sitting above or below libc. An int fd forces it below (Windows needs a table); a FILE* forces it above. Measured: four backends store their native thing with zero bridging. * fs and net dissolve. Naming goes to openkal.namespace, and what it hands back is the same stream resource a file, a socket or a UART gives you. Cleaner than 'everything is a file', because naming failure and I/O failure end up in different interfaces. * core is abort + stream + memory. Memory is core because a bump allocator over a static arena is an IMPLEMENTATION, not an emulation — the test being whether a fake would make callers silently wrong, which is true of a clock that does not advance but not of an allocator that can fail. Also records the caps-can-lie problem with four defences ordered by strength, led by making unsupported operations unrepresentable in the type system rather than false in a bool — the same conclusion K1/K2 reached for the MMU. * docs(openkal): retract two decisions after review, and record six open questions Two things the design got wrong, both retracted with the reasoning that made them look right at the time: * `openkal.namespace` replacing fs and net. It violated this document's own §5.1 rule (a stream whose caps are the union of file and socket operations is precisely the 'present but useless' antipattern), it required every backend to carry a URI parser — which is an emulation layer by the downward admission criterion — and the WASIp2 precedent it cited was a misreading: WASIp2 separates resource KINDS and shares only the stream type. * Extending cfg() with capability predicates. That conclusion was about openarch's AddressSpace; going through openkal interface by interface, core has no semantic axis at all, and the triple already carries most of what cfg(mmu) would have. The design is now a zero-engine-change proposal. Also corrects the module wiring: `reexport` propagates DOWNSTREAM, so an interface package cannot use it to reach a backend the consumer chose. The backend reexports the interface instead, which is what that mechanism is for. Six open questions the draft did not take a position on, led by one that is measured rather than theoretical: picolibc's vfprintf references free, so routing operator new to kal_alloc while printf keeps picolibc's malloc puts two allocators on the same RAM. The spec has to require that kal_alloc be built over a libc allocator where one exists, not beside it. * docs(openkal): capabilities are ADL-probeable — the caps struct and its config file are gone The design's §4 rested on one measurement: `requires { mcpp::runner("x") }` is a hard error when the name is absent. The measurement was right; the quantifier in the conclusion was not. It is QUALIFIED names that cannot be probed — unqualified lookup through ADL is dependent inside a template and evaluates to false, exactly as wanted. Verified on llvm 22.1.8 against real C++20 modules, not headers, with all three behaviours holding at once: * backend present → the concept is true and the call resolves to it * backend absent → the concept is FALSE, so `if constexpr` degrades * backend absent, called anyway → a compile error carrying the spec's own wording, which is what 'build the diagnostic in' was asking for⚠️ One trap worth the record: if the fallback overload returns the same type as the real one, the concept is true even with no backend — a requires-expression does not instantiate the body, so the static_assert never fires. The fallback must return a distinct type. Measured, after writing it the other way first. Consequences: the caps struct, the generated caps module and capabilities.toml are all deleted. A backend's module interface IS its capability declaration, so claim and implementation become the same artifact by construction and the whole 'caps can lie' problem shrinks from structural+behavioural to behavioural only. Nothing leaves mcpp.toml — backend selection stays a conditional dependency. * docs(openkal): the backend owns the interface module name, and consumers declare both Answers two review questions that turned out to be the same question. The draft had the application write `import openkal.uart;`. That is wrong — it pins the source to a backend, which is the one thing openkal exists to avoid. But it papered over a real constraint, now measured on mcpp 2026.8.19.4 with gcc 16.1.0: * a transitive dependency's module IS importable *⚠️ but ADL does NOT reach a module the translation unit did not import ('seek' was not declared in this scope) So the backend's declarations must live in the module the app imports, which forces the backend to own the well-known name `openkal.stream` while the interface package provides `openkal.abi.stream`. Verified end to end: the app writes one import, names no backend, and ADL resolves to the backend's seek.⚠️ And a trap worth the record: the interface module cannot be called `openkal.stream.abi` — the module graph reads the dots as hierarchy and ninja reports a self-cycle on openkal.stream.gcm. `openkal.abi.stream` is fine. On dependencies: two, not one. The backend alone would work, but declaring the contract is what lets the APPLICATION pin the contract version, turning a mismatch into a resolution error instead of a pile of signature errors at compile time. Same shape as embedded-hal plus a board crate. * docs(openkal): openkal IS the ABI, and the fragmentation risk is mechanically bounded Naming: the interface package is `openkal`, not `openkal-abi` — openkal is the specification, so saying it twice is noise. Two module names are still forced by the language (§4.3), but the qualifier now lands only where implementers see it: applications write `import openkal.stream;`, backend authors write `export import openkal.decl.stream;`. The backend owning the application-visible module name is the one real cost of that shape, and it is a fragmentation risk: a backend could put non-standard names into the standard module and applications would not notice. What bounds it is that most of the surface is not the backend's to touch — ⓘ measured, a backend redefining the interface's types is rejected outright: error: redeclaring 'struct kal::io_result@openkal.decl.stream' in module 'openkal.stream' conflicts with import so the only freedom left is adding overloads, and THAT is statically checkable: conformance diffs the module's exported name set — and signatures, since an `unsigned long` offset would still win ADL through a conversion — against the spec list. Vendor extensions must live under a different module name, which makes 'I used an extension' visible in the source. Second review pass adds two findings: the cardinality that matters is one implementation per INTERFACE rather than one backend per program (a program may take stream from one provider and memory from another), and the fallback overload is too greedy — unconstrained, it catches every kal type and tells a socket it is not a seekable stream. * docs(openkal): add a complete Linux reference implementation Two identities: a backend that works today, and the thing other implementers copy. It does not move the D0 gate — that gate is whether a THIRD PARTY writes a third backend — but it turns 'guess the shape and write the implementation' into 'write the implementation'. Writing it out surfaced three things the design document had not: * core operations need no ADL at all. They are declared `extern "C"` by the interface and defined by the backend; missing means a link error. The ADL mechanism serves optional capabilities only, which makes the common path simpler than the draft implied. * short writes are a spec question nobody had asked. ::write(2) may write less than requested, so openkal must choose: write-all-or-error (the loop lives once, in the backend) or allow short writes (every caller writes the loop — which is exactly where POSIX has tripped programs up for decades). * ⭐ it independently confirms the fs/net decomposition. On Linux, seekability is a property of the HANDLE, not of the backend — lseek succeeds on a file and returns ESPIPE on a pipe. If openkal.stream had seek, the Linux backend could not answer honestly: claiming it means always failing on pipes, which is precisely the 'present but useless' antipattern. Because §2.3 puts seek on openkal.fs's descriptor instead, the question does not arise. That last point is the strongest argument for writing a complete reference at all: it is the only way to find a decomposition error, and it finds it earlier than a conformance suite would. * docs(openkal): module naming is normative, and decl is not interchangeable with impl `openkal.impl.*` would be semantically backwards: that module belongs to the interface package and holds declarations — types, the extern "C" surface, the fallback overloads, the concepts. What an implementation provides is `openkal.<interface>` itself. The name has to be in the spec rather than left to taste, for three reasons that are all load-bearing: every backend must `export import` that exact name, so it is part of the contract; the guarantee that a backend cannot redefine the interface's types only holds while all backends import the SAME module; and conformance's exported-name diff needs to know which names came from the shared module.⚠️ The rationale has to ship with the rule. A spec reader will naturally reach for `openkal.stream.decl` — the dotted extension — and that one was measured to produce a ninja self-cycle on openkal.stream.gcm. A rule without its reason sends the first implementer straight into it. Also records a simplification that was considered and rejected: one `openkal` module holding every interface's declarations. It costs a naming level but breaks per-interface independent versioning, and drags task/fs declarations into a backend that only provides streams. * docs: openkal 0.1 implementation plan, and the design document now points at the shipped packages openkal 0.1 exists as two published packages: mcpplibs/openkal carries the specification and the modules that declare it, and mcpplibs/openkal-linux is the reference implementation, maintained as the worked example other implementations follow. Both are mirrored, and the mirrored archives were verified byte-identical. The plan document records the task dependencies, the criteria applied to each decision, and what verification established. Two results are worth separating from the rest. Writing a complete reference implementation confirmed the decomposition independently of the reasoning that produced it: on Linux, whether a stream can be repositioned is a property of the individual descriptor rather than of the implementation, so an openkal.stream that offered positioning could have been neither claimed honestly nor withheld usefully. A decomposition error of that kind is invisible in specification text and would have surfaced later. The exported-surface checker required by clause 9.3 was verified in both directions, and the negative direction mattered: an earlier version of it was vacuous, comparing a set of C++ symbols that inline functions never emit. The design document is now marked as the record of derivation, including withdrawn proposals and their reasons, while the specification records only conclusions. Where they disagree the specification governs. --------- Co-authored-by: speak-agent <248744407+speak-agent@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
动机
v0.0.94 发布时
publish-ecosystem被 30min job timeout 杀掉,日志里看不出是哪个资产在慢。事后查:gtc 跨境传linux-x86_64(8.4MB)与linux-aarch64(30.5MB)两个大件卡住,GitHub 端 8 个全好、GitCode 端只差这两个,最后靠本地gtc release upload手工补齐(已核验两端 16 个资产全 200 + 字节一致)。改动
already mirrored, skipping,绝不重传。<github|gitcode> <资产> (<大小>) uploaded in Ns。MIRROR_UPLOAD_TIMEOUT,默认 120s):超时 →WARN ... exceeded the 120s cap after Ns — skipping,放弃该资产、后续轮次不再重试。为何这个 cap 与 v0.0.90 事故不同(重要)
脚本里原有一条血泪注释:v0.0.90 用
timeout 300包上传,杀掉慢但在推进的 PUT 然后重传,每次从字节零重来,永不收敛,把 20min job 预算拖垮。本 cap 的安全性来自 超时即放弃、绝不重试。完整性 gate 仍是唯一 pass/fail —— 漏传会响亮报错,而不是静默发半个镜像。顶部注释已按此改写。
120s 对 GitHub runner → GitCode 跨境传 30.5MB 的 aarch64 包大概率不够。真这样的话每次发版该资产都会被 warn+skip → gate 红 → 仍需人工补传(即这次的实际流程)。
差别是:从「30 分钟静默挂死」变成「2 分钟内明确报出是哪个资产、多大、跳过了」 —— 可观测性是净赚。若希望 CI 全自动闭环,可 (a) 调大
MIRROR_UPLOAD_TIMEOUT,或 (b) 按资产大小分档给 cap。当前实现已把它做成 env 变量,改一行即可。验证
200、已存在全部 skip、206s 通过。timeout对函数无效,第一版测试正是栽在这):[mirror] github big.tar.gz (2.9MB) uploaded in 1s[mirror] WARN: gitcode big.tar.gz (2.9MB) exceeded the 2s cap after 2s — skipping (not retried; the verify gate below decides the release)