Skip to content

Use JAliEn credentials in CCDB API#5826

Merged
pzhristov merged 8 commits intoAliceO2Group:devfrom
Atlantic777:feature-jalien-credentials
Sep 3, 2021
Merged

Use JAliEn credentials in CCDB API#5826
pzhristov merged 8 commits intoAliceO2Group:devfrom
Atlantic777:feature-jalien-credentials

Conversation

@Atlantic777
Copy link
Copy Markdown
Contributor

This PR adds JAliEn credentials manager to CCDB API and decorates curl handle with JAliEn credentials. Depends on: alisw/alidist#2932

cc @costing @iraklic @yuw726

@Atlantic777 Atlantic777 requested review from a team, Barthelemy, costing and sawenzel as code owners March 30, 2021 11:51
costing
costing previously approved these changes Mar 30, 2021
jgrosseo
jgrosseo previously approved these changes Apr 1, 2021
#include <TMessage.h>
#include "CCDB/CcdbObjectInfo.h"

#if !defined(__CINT__) && !defined(__MAKECINT__) && !defined(__ROOTCLING__) && !defined(__CLING__)
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need this?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The libjalien isn't a ROOT library and doesn't have a ROOT dictionary. I noticed problems with ROOT interpreter when CcdbApi.h was included because it couldn't understand this type.

There is similar code in JAliEn-ROOT / AliEn-ROOT-Legacy, but maybe there is another way around it. In any case, if this header is not explicitly included in a ROOT interpreter session then maybe it works without it.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sawenzel What do you advise?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No hard opinion here. But usually this should not be necessary. One can already use o2::ccdb::CcdbApi in a ROOT session without including the header. And the protection could be put outside as well.
We can also merge as is and remove later.

@jgrosseo
Copy link
Copy Markdown
Collaborator

@TimoWilken The CI errors seems unrelated, or am I mistaken?

@TimoWilken
Copy link
Copy Markdown
Contributor

TimoWilken commented Apr 12, 2021

@jgrosseo the libjalienO2 dependency (added in this PR) isn't picked up by QC. I'll open a PR in alidist and restart these checks when that's merged. While the O2 build runs fine, could you hold off on merging for now so we don't break the QC build?

@Barthelemy
Copy link
Copy Markdown
Collaborator

QC should not need to be touched. Dependencies should be transitive in both aliBuild and CMake.

@Barthelemy
Copy link
Copy Markdown
Collaborator

I don't have a solution but this is not going to work:
HINTS "${LIBJALIENO2}"

This variable is unknown in downstream packages. You could use the environment variable LIBJALIENO2_ROOT

@jgrosseo
Copy link
Copy Markdown
Collaborator

@costing Do you know who could follow this up after @Atlantic777 departure. My understanding is that just the cmake has to be fixed.

@iraklic
Copy link
Copy Markdown

iraklic commented Apr 29, 2021

@jgrosseo that would probably be me. I read through this thread but I'm not entirely sure what is going on. If you know what needs to be fixed in cmake please let me know and I will have a look.

@jgrosseo
Copy link
Copy Markdown
Collaborator

Apart from resolving the merge conflict which is probably simple, follow the comments by @Barthelemy about the different variable name

@jgrosseo
Copy link
Copy Markdown
Collaborator

Thanks for looking into it!

@yuw726
Copy link
Copy Markdown

yuw726 commented Apr 29, 2021

Hi @Barthelemy @jgrosseo @iraklic,
Isn't ${LIBJALIENO2} passed to cmake in o2 alidist script? Please correct me if I am wrong.

@Barthelemy
Copy link
Copy Markdown
Collaborator

@yuw726 the cmake variable LIBJALIENO2 is set to the value of the environment variable LIBJALIENO2_ROOT when configuring O2.

Later in the build, QualityControl is built and when it loads the dependencies it tries to use the FindlibjalienO2.cmake and fails because, at this stage, LIBJALIENO2 is not knows. You should try to use the environment variable LIBJALIENO2_ROOT in the Find script. Not guaranteed to work but it is worth trying.

To test do aliBuild build QualityControl.

@jgrosseo
Copy link
Copy Markdown
Collaborator

jgrosseo commented May 6, 2021

@iraklic Any news?

@iraklic
Copy link
Copy Markdown

iraklic commented May 7, 2021

@iraklic Any news?

Sorry, I am having very busy week. It is on my todo. BTW: do I understand it right that you expect a new PR from me with the conflict in CCDB/src/CcdbApi.cxx resolved or there is some other way of modifying this PR (I have never done that)?

@jgrosseo
Copy link
Copy Markdown
Collaborator

jgrosseo commented May 7, 2021

@iraklic Any news?

Sorry, I am having very busy week. It is on my todo. BTW: do I understand it right that you expect a new PR from me with the conflict in CCDB/src/CcdbApi.cxx resolved or there is some other way of modifying this PR (I have never done that)?

You cannot push to the branch which made this PR, but you can just start from that branch, push that into your own fork, and then open a PR based on that one. This will include all these changes. We will at that point close this PR.

@iraklic
Copy link
Copy Markdown

iraklic commented May 7, 2021

@iraklic Any news?

Sorry, I am having very busy week. It is on my todo. BTW: do I understand it right that you expect a new PR from me with the conflict in CCDB/src/CcdbApi.cxx resolved or there is some other way of modifying this PR (I have never done that)?

You cannot push to the branch which made this PR, but you can just start from that branch, push that into your own fork, and then open a PR based on that one. This will include all these changes. We will at that point close this PR.

Thank you. That's what I was going to do, but wanted to confirm.

@jgrosseo
Copy link
Copy Markdown
Collaborator

@iraklic Any news?

Sorry, I am having very busy week. It is on my todo. BTW: do I understand it right that you expect a new PR from me with the conflict in CCDB/src/CcdbApi.cxx resolved or there is some other way of modifying this PR (I have never done that)?

You cannot push to the branch which made this PR, but you can just start from that branch, push that into your own fork, and then open a PR based on that one. This will include all these changes. We will at that point close this PR.

Thank you. That's what I was going to do, but wanted to confirm.

Any news?

@jgrosseo
Copy link
Copy Markdown
Collaborator

Fixed conflict

@costing
Copy link
Copy Markdown
Collaborator

costing commented May 27, 2021

Done, thank you @TimoWilken ! I think it's also requiring a review from @Barthelemy

@Barthelemy
Copy link
Copy Markdown
Collaborator

@costing could you give me a bit of context ?

@jgrosseo
Copy link
Copy Markdown
Collaborator

@costing could you give me a bit of context ?

It is fine. I approved it.

@jgrosseo
Copy link
Copy Markdown
Collaborator

fullCI is unrelated. I'll ignore the space checker on this one (apologies @vkucera , we'll fix this later), and merge when the other two CIs have passed.

@Barthelemy
Copy link
Copy Markdown
Collaborator

I tried to compile and got the following error:

2021-05-27@11:25:43:DEBUG:QualityControl:QualityControl:0: -- The following REQUIRED packages have not been found:
2021-05-27@11:25:43:DEBUG:QualityControl:QualityControl:0: 
2021-05-27@11:25:43:DEBUG:QualityControl:QualityControl:0:  * libjalienO2
2021-05-27@11:25:43:DEBUG:QualityControl:QualityControl:0:    For CCDB API
2021-05-27@11:25:43:DEBUG:QualityControl:QualityControl:0: 

The command I used on CC7 is

aliBuild build QualityControl --defaults o2 --debug

alidist is up to date with master

@jgrosseo
Copy link
Copy Markdown
Collaborator

@TimoWilken do you know why QC misses this?

@iraklic
Copy link
Copy Markdown

iraklic commented Jun 14, 2021

I am trying to understand nuts and bolts of what is going on here, but I am having trouble to compile dev at this point. It crushes with this:
/home/iraklic/jAliEn_Clean/sw/SOURCES/Monitoring/v3.8.6/v3.8.6/src/Transports/Unix.cxx:18:10: fatal error: filesystem: No such file or directory #include <filesystem> ^~~~~~~~~~~~ compilation terminated.

I checked out dev and building with aliBuild build O2 --defaults o2

Any idea?

@Barthelemy
Copy link
Copy Markdown
Collaborator

some of your stuff is not up to date. Make sure you update O2

@iraklic
Copy link
Copy Markdown

iraklic commented Jun 15, 2021

What kind of stuff? I pulled O2 using aliBuild init O2@dev --defaults o2 and even did git pull today just in case and tried to rebuild. But have the same issue.

@iraklic
Copy link
Copy Markdown

iraklic commented Jun 21, 2021

Is there a mattermost/slack channel or something similar when I can ask about my building error? Or should I use alice-project-analysis-task-force mailing list? Or is there a fail-safe tag for O2 I can check out that will surely build? Any help is appreciated.

@iraklic
Copy link
Copy Markdown

iraklic commented Jun 21, 2021

OK. looks like I am not alone in this: https://alice.its.cern.ch/jira/browse/O2-2327. But the problem is that the workaround yields an entirely new issue:

In file included from /home/iraklic/jAliEn_Clean/sw/SOURCES/O2/master/0/Common/Utils/src/ShmManager.cxx:19:
/home/iraklic/jAliEn_Clean/sw/SOURCES/O2/master/0/Common/Utils/include/CommonUtils/ShmManager.h:40:20: error: field 'counter' has incomplete type 'std::atomic<int>'
   40 |   std::atomic<int> counter = 0; // atomic counter .. counter number of attached processes
      |                    ^~~~~~~

Any suggestions are appreciated

@jgrosseo
Copy link
Copy Markdown
Collaborator

jgrosseo commented Jul 7, 2021

The JIRA ticket you quote is closed and it is said " This should not be the case anymore, provided you move to alibuild 1.8.4 " .
Which alibuild do you have?

@jgrosseo
Copy link
Copy Markdown
Collaborator

jgrosseo commented Jul 7, 2021

You can ask in O2 release integration or O2 analysis challenge on mattermost any additional questions.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Aug 7, 2021

This PR did not have any update in the last 30 days. Is it still needed? Unless further action in will be closed in 5 days.

@github-actions github-actions bot added the stale label Aug 7, 2021
@iraklic
Copy link
Copy Markdown

iraklic commented Aug 7, 2021

@jgrosseo has this been merged through other PR? I am asking because there are mentions of conflicts mentioned here being fixed and so on. The compilation problem that I had I fixed using the workaround suggested in the ticket that I mentioned. Anyways please let me know if this has been merged though some other PR or if you want me to check it on my own and open new PR with this change. Thanks.

@github-actions github-actions bot closed this Aug 13, 2021
@jgrosseo
Copy link
Copy Markdown
Collaborator

@iraklic I don't think this has been merged through another PR. It is just that other changes had been made to the same code and the conflicts need to be resolved.

@jgrosseo jgrosseo reopened this Aug 16, 2021
@jgrosseo
Copy link
Copy Markdown
Collaborator

I reopened it and do not see a conflicts actually.
However, there are failures in the CI? Could you update this PR with what needs to be done, so that it goes green and we can merge it?

@github-actions github-actions bot closed this Aug 22, 2021
@costing costing reopened this Aug 30, 2021
@pzhristov
Copy link
Copy Markdown
Contributor

I will try to fix the CMake part and provide modifications in the alidist recipes since the CI issues are related to the discovery of the package.

@pzhristov
Copy link
Copy Markdown
Contributor

The following PR alisw/alidist#3275 should solve the issues we observe in the CI.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Development

Successfully merging this pull request may close these issues.

9 participants