fix: return error when {{ source }} references a skipped source - #9095
fix: return error when {{ source }} references a skipped source#9095ambikeesshh wants to merge 9 commits into
Conversation
5fa3757 to
06f0a18
Compare
|
hey @olblak PTAL |
|
@olblak can you review it ? when you get a chance |
|
Of course, I'll try to find some time thanks for the fix |
| Config: config.Spec.Sources[id], | ||
| Result: &result.Source{ | ||
| Result: result.SKIPPED, | ||
| Result: "", |
There was a problem hiding this comment.
Why is it changed from result.SKIPPED to empty string?
There was a problem hiding this comment.
empty string = not yet run (retry later). SKIPPED = explicitly skipped (error)
both were the same value before this change
dfca649 to
8bf3606
Compare
|
@olblak rebased on latest main |
|
@olblak friendly ping |
|
@ambikeesshh Sorry very busy days at work. |
|
@olblak yep, i replied inline on the lemme know if you'd like any other changes |
There was a problem hiding this comment.
Pull request overview
This pull request fixes a confusing failure mode where {{ source "id" }} could remain unresolved and silently leak into runtime configuration when the referenced source was skipped via depends_on. It makes “not yet run” distinguishable from “skipped”, and updates the runtime template behavior to return a clear error when a skipped source is referenced.
Changes:
- Initialize sources with an empty result (instead of
SKIPPED) so the runtime template engine can distinguish “pending” vs “skipped”. - Explicitly mark sources as
SKIPPEDwhendepends_oncauses them to be skipped. - Update the runtime
sourcetemplate function to error when the referenced source isSKIPPED, and add/extend tests covering this behavior.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| pkg/core/pipeline/sources_test.go | Adds an end-to-end test ensuring a target referencing a skipped source via {{ source "..." }} fails clearly. |
| pkg/core/pipeline/main.go | Changes initial source result to empty and explicitly sets SKIPPED when a source is skipped by dependency logic. |
| pkg/core/config/main_test.go | Adds a unit test asserting the runtime template error message for a skipped source. |
| pkg/core/config/funcsUpdatecli.go | Adds explicit SKIPPED handling in the runtime source template function, returning a clear error. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
Hi @ambikeesshh, I am a bit annoyed by this change because if something unexpected happened, the resource wouldn't have a clear state. I think I would prefer to add a new Boolean named "IsRun" on the source/condition/target like Once The source function is exectued like here updatecli/pkg/core/pipeline/source/main.go Line 112 in 094d96c then we set it to true |
|
hey @olblak
one note: a skipped source never hits thanks for the review! |
|
thanks for the update, I'll review this pr in the coming days |
|
sorry for the confusion on this pullrquest, I now realize that all the comments that I made on this pr where only visible to me because they were "pending" |
|
@olblak |
olblak
left a comment
There was a problem hiding this comment.
I manually tested using the following manifest
name: Test sourceid {{ source "shell" }}
sources:
shell:
name: Show 1.2.3
kind: shell
dependson:
- condition#default
spec:
command: 'echo "1.2.3"'
shell2:
name: Show source output {{ source "shell" }}
kind: shell
spec:
command: 'echo {{ source "shell" }}'
conditions:
default:
name: "False"
kind: shell
disablesourceinput: true
spec:
command: |
false
And it works as expected.
While testing it, I spotted a minor issue which I'll fix in a follow up pr
|
@olblak turns out a bunch of the repo's own is that covered by the follow-up you mentioned, or would you rather skipped sources also skip their dependents instead of erroring? happy to go either way, just lmk which you prefer |
|
This is not related to my follow up. I think it would make more sense to not error on skipped source but just to skip dependents resources |
Fix #7919
Summary
{{ source "id" }}returns the literal string{{ source "id" }}when the referenced source is skipped viadepends_on. Since a skipped source never transitions to SUCCESS, this template literal never resolves. It silently leaks into config values causing confusing downstream failures with no clear indication of what went wrong.What changed
Sources initialized with
Result: SKIPPED, same state as an explicitly skipped source, so the template function couldn't tell them apart. Two changes fix this:result.SKIPPEDto empty string, so "not yet run" and "skipped" are distinguishableshouldSkipResourceskips a sourceThen the template function can safely error on SKIPPED:
case result.SKIPPEDin the runtimesourcetemplate function that returns a clear error, consistent with existingFAILUREhandlingTest
New test cases:
pkg/core/config/main_test.go- unit test for SKIPPED source in template functionpkg/core/pipeline/sources_test.go- end-to-end test with a source skipped viadepends_onand a target referencing it via{{ source "..." }}