Skip to content

[DCS] Helpers for (fake) DCS aliases generation#4863

Merged
shahor02 merged 4 commits intoAliceO2Group:devfrom
aphecetche:dcs-aliases-generation
Nov 23, 2020
Merged

[DCS] Helpers for (fake) DCS aliases generation#4863
shahor02 merged 4 commits intoAliceO2Group:devfrom
aphecetche:dcs-aliases-generation

Conversation

@aphecetche
Copy link
Copy Markdown
Collaborator

@aphecetche aphecetche commented Nov 18, 2020

See example usage in e.g. DCSRandomDataGeneratorSpec.h (generate method)

Note that the actual upload to CCDB is failing as far as I can tell (as in the current version ?) because the path used (inferred from class name) contains invalid characters '<' and ','

Might want to retain the 3 commits (i.e. do not squash) as the first two are "fixes" that are not only useful for data point generation IMO.

@aphecetche aphecetche requested a review from a team as a code owner November 18, 2020 15:39
@chiarazampolli
Copy link
Copy Markdown
Collaborator

chiarazampolli commented Nov 18, 2020

Hello @aphecetche ,
Many thanks! I will try your code ASAP. As for the CCDB, lately there were changes by @sawenzel , I don't know if they matter. Do you have a simple example showing the issue?
In the past, we had problems with special characters, but it was solved (maybe now it is something else).

Chiara

@aphecetche aphecetche force-pushed the dcs-aliases-generation branch 2 times, most recently from d048f0e to a343081 Compare November 18, 2020 21:45
@aphecetche
Copy link
Copy Markdown
Collaborator Author

aphecetche commented Nov 18, 2020

@chiarazampolli maybe wait for the builds to be green before looking at the code ;-)

Concerning the ccdb upload failure. For example :

o2-dcs-data-workflow --max-timeframes=10 | o2-calibration-ccdb-populator-workflow --ccdb-path http://localhost:6464

(or the same with o2-dcs-random-data-workflow introduced by this PR)

gives me messages like :

[9468:ccdb-populator]: Usage of /*:
[9468:ccdb-populator]:   GET:
[9468:ccdb-populator]:     task name / detector name / start time / <UUID> - return the content of that UUID, or
[9468:ccdb-populator]:     task name / detector name / [ / time [ / key = value]* ]
[9468:ccdb-populator]:
[9468:ccdb-populator]:   POST:
[9468:ccdb-populator]:     task name / detector name / start time [ / end time ] [ / UUID ] [ / key = value ]*
[9468:ccdb-populator]:     binary blob as multipart parameter called 'blob'
[9468:ccdb-populator]:
[9468:ccdb-populator]:   PUT:
[9468:ccdb-populator]:     task name / detector name / start time [ / new end time ] [ / UUID ] [? (key=newvalue&)* ]
[9468:ccdb-populator]:
[9468:ccdb-populator]:   DELETE:
[9468:ccdb-populator]:     task name / detector name / start time / UUID
[9468:ccdb-populator]:     or any other selection string, the matching object will be deleted
[9468:ccdb-populator]:
[9468:ccdb-populator]: Usage of /browse/* or /latest/*:
[9468:ccdb-populator]:   GET:
[9468:ccdb-populator]:     task name / detector name / [start time, default = now] [/key=value]*
[9468:ccdb-populator]:     Use the Accept header to control the output format (one of text/plain, text/html, text/xml, application/json)
[9468:ccdb-populator]:
[9468:ccdb-populator]: This call was made with:
[9468:ccdb-populator]:   servlet path:
[9468:ccdb-populator]:   context path:
[9468:ccdb-populator]:   HTTP method: POST
[9468:ccdb-populator]:   path info: /0/0/ObjectType=unordered_map<string,float>/
[9468:ccdb-populator]:   query string: null
[9468:ccdb-populator]:   request URI: //0/0/ObjectType=unordered_map%3Cstring%2Cfloat%3E/

and I don't see new objects in the CCDB, so I interpret this as a failure.

@chiarazampolli
Copy link
Copy Markdown
Collaborator

chiarazampolli commented Nov 18, 2020

Ciao @aphecetche ,

I can reproduce the issue. I can try to have a look, it could be related, as I said, to the recent changes to the CCDB classes.

Chiara

@chiarazampolli chiarazampolli mentioned this pull request Nov 18, 2020
Merged
@chiarazampolli
Copy link
Copy Markdown
Collaborator

chiarazampolli commented Nov 18, 2020 •
edited
Loading

Hello @aphecetche ,

See #4867. There was indeed a mistake in DCSDataProcessorSpec.cxx, and I was sending to the CCDB populator not the correct data (I was not sending from the correct DCSProcessor, which is an element of a vector - I think it is a remnant of previous implementation without the vector). Can you try if this fixes your issue? It should, since you use the same data processor in your workflow, if I read well.

Chiara

@aphecetche
[DCS] Add helper functions to generate (random) DCS data points.
1605216 
The main functions are `o2::dcs::generateRandomDataPoints` and
`o2::dcs::expandAliases`, the latter being used to create easily large
lists of data point aliases from basic patterns. For instance
the compact description "DET/Crate[0..3]/Channel[000.042]/[iMon,vMon]"
would be expanded to 344 (4x43x2) aliases.
@aphecetche aphecetche force-pushed the dcs-aliases-generation branch from a343081 to 1605216 Compare November 19, 2020 06:19
@aphecetche
Copy link
Copy Markdown
Collaborator Author

aphecetche commented Nov 19, 2020

@chiarazampolli #4867 is not really working (for me at least), but I would say this one can be merged anyway as the origin of the problem is not related to this new code, right ? (as, as you've seen, I'm just reusing the data processor spec part (just changed the generation spec).

@aphecetche
Copy link
Copy Markdown
Collaborator Author

aphecetche commented Nov 19, 2020 •
edited
Loading

@chiarazampolli don't know exactly what I've done wrong the first time but rebuilding and re-trying it now works indeed. Sorry for the false alarm.

@chiarazampolli
Copy link
Copy Markdown
Collaborator

chiarazampolli commented Nov 19, 2020

Hello Laurent, very good!

@aphecetche
Copy link
Copy Markdown
Collaborator Author

aphecetche commented Nov 23, 2020

@shahor02 I'm sure this is not the only PR you have to review, but just to be sure it's on your radar ;-) Thanks.

shahor02
shahor02 previously approved these changes Nov 23, 2020
View reviewed changes
Copy link
Copy Markdown
Collaborator

@shahor02 shahor02 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @aphecetche
Thanks, looks perfect. I would just add to DetectorsDCSLinkDef.h

#pragma link C++ function o2::dcs::expandAlias(const std::string&);
#pragma link C++ function o2::dcs::expandAliases(const std::vector<std::string>&);

to have corresponding dictionaries: I think it would be useful to be able to test the aliases from root command line

@aphecetche
Copy link
Copy Markdown
Collaborator Author

aphecetche commented Nov 23, 2020

@shahor02 I've added the expandAlias(es) to the dictionary.

shahor02
shahor02 approved these changes Nov 23, 2020
View reviewed changes
Copy link
Copy Markdown
Collaborator

@shahor02 shahor02 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aphecetche I saw, thanks! Merging.

@shahor02 shahor02 merged commit 50125c6 into AliceO2Group:dev Nov 23, 2020
@aphecetche aphecetche deleted the dcs-aliases-generation branch November 23, 2020 21:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@shahor02 shahor02 shahor02 approved these changes

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@aphecetche @chiarazampolli @shahor02