Skip to content

DPL: Reconstruct the full command and propagate it to workflow dump tools#5778

Merged
ktf merged 1 commit intoAliceO2Group:devfrom
knopers8:propagate-dpl-command
Mar 25, 2021
Merged

DPL: Reconstruct the full command and propagate it to workflow dump tools#5778
ktf merged 1 commit intoAliceO2Group:devfrom
knopers8:propagate-dpl-command

Conversation

@knopers8
Copy link
Copy Markdown
Collaborator

@ktf I took a stab on providing to the AliECS the full command used to run the workflow (including all the workflow merging). This would enable the AliECS to avoid generating intermediate files to run a topology and thus streamline it as much as possible.

@knopers8 knopers8 requested a review from a team as a code owner March 24, 2021 13:14
Copy link
Copy Markdown
Member

@ktf ktf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is fine with me, however it's not clear to my what is missing from the current list of arguments. Could you please elaborate?

@knopers8
Copy link
Copy Markdown
Collaborator Author

The argument list doesn't provide the command used to generate the full workflow. So if I have something like:

o2-dpl-a | o2-dpl-b

Then the execution of each process gives me only this:

o2-dpl-b --id <some-device-from-b> --other-arguments

But I need this:

o2-dpl-a | o2-dpl-b | o2-dpl-b --id <some-device-from-b> --other-arguments

There is a solution with using intermediate files (cat dump.json), but I would like to avoid relying on generated files.

In principle, I could reconstruct the full command from DataProcessorInfos, which have exec name and args, but there is no unique identifier of a part of the merged workflow. So having duplicated executables in the same merged workflow would be quite troublesome:

o2-qc --config json:///a/b/c.json | o2-qc --config json:///d/e/f.json

@ktf
Copy link
Copy Markdown
Member

ktf commented Mar 24, 2021

Ok, thank you for your explanations.

@knopers8
Copy link
Copy Markdown
Collaborator Author

I don't understand why o2-dataflow failed.

@ktf
Copy link
Copy Markdown
Member

ktf commented Mar 25, 2021

What was the error? It was transient in any case.

@ktf ktf merged commit 577f91b into AliceO2Group:dev Mar 25, 2021
@knopers8
Copy link
Copy Markdown
Collaborator Author

There was no error or failed test, but the check was marked as failed. Thanks for merging

@knopers8 knopers8 deleted the propagate-dpl-command branch March 25, 2021 14:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants