Skip to content

[FIT] Follow data format changes in O2 #8344#1092

Merged
knopers8 merged 2 commits intoAliceO2Group:masterfrom
mslupeck:master
Mar 17, 2022
Merged

[FIT] Follow data format changes in O2 #8344#1092
knopers8 merged 2 commits intoAliceO2Group:masterfrom
mslupeck:master

Conversation

@mslupeck
Copy link
Copy Markdown
Contributor

This is necessary for the CI tests in O2 PR #8344 to pass

@knopers8 knopers8 enabled auto-merge (squash) March 15, 2022 11:19
@mslupeck
Copy link
Copy Markdown
Contributor Author

Hi @knopers8 thanks for approving.

But I'm not sure how to deal with the CI tests in this case. This is a chicken-and-egg problem. I mean the CI tests of this PR in QC require that the O2 PR #8344 is merged. On the other hand, the CI tests in O2 need this PR to be merged first.

I'll ping also @shahor02 for a second opinion, but I think it would be best to merge the QC first, even with building errors, simply because it is a much smaller PR, involving only 8 files, as opposed to the PR in O2, which is a major one involving over 100 files.
I've tested both of these PRs locally on my laptop and together they do build and all the CI tests pass.

Another approach could be to split this QC PR into two parts

  • in the first one all 8 offending files would be excluded from building. Then the O2 PR could be merged without errors.
  • in the second PR the 8 offending files are again included

What do you think?

@shahor02
Copy link
Copy Markdown
Contributor

Hi @mslupeck
The clean way would be indeed to restore the O2 files which are needed for the QC compilation, merge the AliceO2Group/AliceO2#8344 after CI, then to merge this PR.
Cheers,
Ruben

@mslupeck
Copy link
Copy Markdown
Contributor Author

Hi @shahor02, I think there is a misunderstanding. I've created this QC PR only because the O2 CI tests fail (the O2 tests apparently also require a successful QC build).

So if I restore the O2 files that are needed for this QC PR to pass, there is no need for this QC PR.

@shahor02
Copy link
Copy Markdown
Contributor

@mslupeck The AliceO2Group/AliceO2#8344 fails in QC compilation at

cd /sw/BUILD/8a7c80d79030a681448394104621fd7e548bc025/QualityControl/Modules/FV0 && /sw/BUILD/8a7c80d79030a681448394104621fd7e548bc025/QualityControl/rootcling_wrapper.sh --rootmap_file /sw/BUILD/8a7c80d79030a681448394104621fd7e548bc025/QualityControl/lib/libO2QcFV0.rootmap --dictionary_file /sw/BUILD/8a7c80d79030a681448394104621fd7e548bc025/QualityControl/Modules/FV0/G__O2QcFV0.cxx --ld_library_path /sw/slc8_x86-64/ROOT/v6-24-06-local1/lib:/sw/slc8_x86-64/GCC-Toolchain/v10.2.0-alice2-5/lib:/sw/slc8_x86-64/GCC-Toolchain/v10.2.0-alice2-5/lib64 --rootmap_library_name libO2QcFV0.so --include_dirs "-I/sw/SOURCES/QualityControl/v1.49.0/v1.49.0/Modules/FV0/include;-I/sw/SOURCES/QualityControl/v1.49.0/v1.49.0/Modules/FV0/src;-I/sw/SOURCES/QualityControl/v1.49.0/v1.49.0/Modules/FV0/;-I/sw/SOURCES/QualityControl/v1.49.0/v1.49.0/Framework/include;-I/sw/slc8_x86-64/boost/v1.75.0-30/include;-I/sw/slc8_x86-64/FairLogger/v1.10.4-6/include;-I/sw/slc8_x86-64/fmt/8.0.1-6/include;-I/sw/slc8_x86-64/FairMQ/v1.4.49-8/include/fairmq;-I/sw/slc8_x86-64/FairMQ/v1.4.49-8/include;-I/sw/slc8_x86-64/boost/v1.75.0-30/include;-I/sw/slc8_x86-64/boost/v1.75.0-30/include;-I/sw/slc8_x86-64/boost/v1.75.0-30/include;-I/sw/slc8_x86-64/boost/v1.75.0-30/include;-I/sw/slc8_x86-64/FairMQ/v1.4.49-8/include;-I/sw/slc8_x86-64/FairMQ/v1.4.49-8/include;-I/sw/slc8_x86-64/ROOT/v6-24-06-local1/include;-I/sw/slc8_x86-64/ROOT/v6-24-06-local1/include;-I/sw/slc8_x86-64/ROOT/v6-24-06-local1/include;-I/sw/slc8_x86-64/ROOT/v6-24-06-local1/include;-I/sw/slc8_x86-64/ROOT/v6-24-06-local1/include;-I/sw/slc8_x86-64/ROOT/v6-24-06-local1/include;-I/sw/slc8_x86-64/ROOT/v6-24-06-local1/include;-I/sw/slc8_x86-64/ROOT/v6-24-06-local1/include;-I/sw/slc8_x86-64/ROOT/v6-24-06-local1/include;-I/sw/slc8_x86-64/Common-O2/v1.6.0-33/include;-I/sw/slc8_x86-64/libInfoLogger/v2.4.3-5/include;-I/sw/slc8_x86-64/Monitoring/v3.10.1-24/include;-I/sw/slc8_x86-64/Configuration/v2.6.3-18/include;-I/sw/slc8_x86-64/Control-OCCPlugin/v0.35.0-2/include/Occ;-I/sw/slc8_x86-64/grpc/v1.34.0-alice2-10/include;-I/sw/slc8_x86-64/protobuf/v3.15.8-6/include;-I/sw/slc8_x86-64/zlib/v1.2.8-10/include;-I/sw/slc8_x86-64/grpc/v1.34.0-alice2-10/include;-I/usr/include;-I/usr/include;-I/sw/slc8_x86-64/c-ares/v1.17.1-12/include;-I/sw/slc8_x86-64/grpc/v1.34.0-alice2-10/include;-I/sw/slc8_x86-64/re2/2019-09-01-18/include;-I/sw/slc8_x86-64/grpc/v1.34.0-alice2-10/include;-I/sw/slc8_x86-64/grpc/v1.34.0-alice2-10/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/abseil/20200225.2-12/include;-I/sw/slc8_x86-64/grpc/v1.34.0-alice2-10/include;-I/sw/slc8_x86-64/O2/8344-slc8_x86-64-local1/include;-I/usr/include;-I/sw/slc8_x86-64/O2/8344-slc8_x86-64-local1/include;-I/sw/slc8_x86-64/ROOT/v6-24-06-local1/include;-I/sw/slc8_x86-64/boost/v1.75.0-30/include;-I/sw/slc8_x86-64/O2/8344-slc8_x86-64-local1/include;-I/sw/slc8_x86-64/O2/8344-slc8_x86-64-local1/include;-I/sw/slc8_x86-64/O2/8344-slc8_x86-64-local1/include/GPU;-I/sw/slc8_x86-64/O2/8344-slc8_x86-64-local1/include;-I/sw/slc8_x86-64/FairRoot/v18.4.2-local1/include;-I/sw/slc8_x86-64/FairRoot/v18.4.2-local1/include;-I/sw/slc8_x86-64/FairRoot/v18.4.2-local1/include;-I/sw/slc8_x86-64/FairRoot/v18.4.2-local1/include;-I/sw/slc8_x86-64/ROOT/v6-24-06-local1/include;-I/sw/slc8_x86-64/ROOT/v6-24-06-local1/include;-I/sw/slc8_x86-64/ROOT/v6-24-06-local1/include;-I/sw/slc8_x86-64/ROOT/v6-24-06-local1/include;-I/sw/slc8_x86-64/ROOT/v6-24-06-local1/include;-I/sw/slc8_x86-64/ROOT/v6-24-06-local1/include;-I/sw/slc8_x86-64/VMC/v1-1-p1-local1/include/vmc;-I/sw/slc8_x86-64/ROOT/v6-24-06-local1/include;-I/sw/slc8_x86-64/ROOT/v6-24-06-local1/include;-I/sw/slc8_x86-64/ROOT/v6-24-06-local1/include;-I/sw/slc8_x86-64/arrow/v5.0.0-alice3-19/include;-I/sw/slc8_x86-64/O2/8344-slc8_x86-64-local1/include;-I/sw/slc8_x86-64/ROOT/v6-24-06-local1/include;-I/sw/slc8_x86-64/Vc/1.4.1-18/include;-I/sw/slc8_x86-64/ms_gsl/4.0.0-1/include;-I/sw/slc8_x86-64/O2/8344-slc8_x86-64-local1/include;-I/sw/slc8_x86-64/O2/8344-slc8_x86-64-local1/include;-I/sw/slc8_x86-64/O2/8344-slc8_x86-64-local1/include;-I/sw/slc8_x86-64/FairRoot/v18.4.2-local1/include;-I/sw/slc8_x86-64/libjalienO2/0.1.3-15/include;-I/sw/slc8_x86-64/O2/8344-slc8_x86-64-local1/include;-I/sw/slc8_x86-64/O2/8344-slc8_x86-64-local1/include;-I/sw/slc8_x86-64/RapidJSON/v1.1.0-alice2-21/include;-I/sw/slc8_x86-64/boost/v1.75.0-30/include;-I/sw/slc8_x86-64/arrow/v5.0.0-alice3-19/include;-I/sw/slc8_x86-64/libuv/v1.40.0-18/include;-I/sw/SOURCES/QualityControl/v1.49.0/v1.49.0/Framework/include;-I/sw/slc8_x86-64/O2/8344-slc8_x86-64-local1/include;-I/sw/slc8_x86-64/O2/8344-slc8_x86-64-local1/include;-I/sw/slc8_x86-64/O2/8344-slc8_x86-64-local1/include/GPU;-I/sw/slc8_x86-64/O2/8344-slc8_x86-64-local1/include;-I/sw/slc8_x86-64/O2/8344-slc8_x86-64-local1/include;-I/sw/slc8_x86-64/O2/8344-slc8_x86-64-local1/include;-I/sw/slc8_x86-64/O2/8344-slc8_x86-64-local1/include;-I/sw/slc8_x86-64/O2/8344-slc8_x86-64-local1/include;-I/sw/slc8_x86-64/O2/8344-slc8_x86-64-local1/include;-I/sw/slc8_x86-64/O2/8344-slc8_x86-64-local1/include;-I/sw/slc8_x86-64/O2/8344-slc8_x86-64-local1/include;-I/sw/slc8_x86-64/O2/8344-slc8_x86-64-local1/include;-I/sw/slc8_x86-64/O2/8344-slc8_x86-64-local1/include;-I/sw/slc8_x86-64/O2/8344-slc8_x86-64-local1/include;-I/sw/slc8_x86-64/TBB/v2021.5.0-8/include;-I/sw/slc8_x86-64/O2/8344-slc8_x86-64-local1/include;-I/sw/slc8_x86-64/O2/8344-slc8_x86-64-local1/include" --compile_defs "-DBOOST_ALL_NO_LIB;-DFMT_LOCALE;-DFMT_SHARED;-DBOOST_ERROR_CODE_HEADER_ONLY;-DFAIRMQ_HAS_STD_FILESYSTEM=1;-DBOOST_ALL_NO_LIB;-DBOOST_CONTAINER_DYN_LINK;-DBOOST_ALL_NO_LIB;-DBOOST_PROGRAM_OPTIONS_DYN_LINK;-DBOOST_ALL_NO_LIB;-DBOOST_FILESYSTEM_DYN_LINK;-DBOOST_ALL_NO_LIB;-DBOOST_REGEX_DYN_LINK;-DBOOST_ERROR_CODE_HEADER_ONLY;-DBOOST_ERROR_CODE_HEADER_ONLY;-D__CLANG_SUPPORT_DYN_ANNOTATION__;-DBOOST_ALL_NO_LIB;-DBOOST_IOSTREAMS_DYN_LINK;-DDPL_ENABLE_BACKTRACE;-DBOOST_ALL_NO_LIB;-DBOOST_SERIALIZATION_DYN_LINK" --pcmdeps "" --headers "/sw/SOURCES/QualityControl/v1.49.0/v1.49.0/Modules/FV0/include/FV0/DigitQcTask.h;/sw/SOURCES/QualityControl/v1.49.0/v1.49.0/Modules/FV0/include/FV0/Helper.h;/sw/SOURCES/QualityControl/v1.49.0/v1.49.0/Modules/FV0/include/FV0/CFDEffCheck.h;/sw/SOURCES/QualityControl/v1.49.0/v1.49.0/Modules/FV0/include/FV0/BasicPPTask.h;/sw/SOURCES/QualityControl/v1.49.0/v1.49.0/Modules/FV0/include/FV0/OutOfBunchCollTask.h;/sw/SOURCES/QualityControl/v1.49.0/v1.49.0/Modules/FV0/include/FV0/OutOfBunchCollCheck.h;/sw/SOURCES/QualityControl/v1.49.0/v1.49.0/Modules/FV0/include/FV0/TriggerQcTask.h;/sw/SOURCES/QualityControl/v1.49.0/v1.49.0/Modules/FV0/include/FV0/CalibrationTask.h;/sw/SOURCES/QualityControl/v1.49.0/v1.49.0/Modules/FV0/include/FV0/ChannelTimeCalibrationCheck.h;/sw/SOURCES/QualityControl/v1.49.0/v1.49.0/Modules/FV0/include/FV0/LinkDef.h" && /sw/slc8_x86-64/CMake/v3.19.2-16/bin/cmake -E copy_if_different /sw/BUILD/8a7c80d79030a681448394104621fd7e548bc025/QualityControl/Modules/FV0/G__O2QcFV0_rdict.pcm /sw/BUILD/8a7c80d79030a681448394104621fd7e548bc025/QualityControl/lib/G__O2QcFV0_rdict.pcm
input_line_9:88:10: fatal error: 'DataFormatsFV0/BCData.h' file not found
#include "DataFormatsFV0/BCData.h"

If these old headers would be available, why the QC should fail?

@mslupeck
Copy link
Copy Markdown
Contributor Author

Ah... so you mean that I should just restore the old headers without removing the new ones and without restoring the functionality of the old ones? Yes... that could work. I'll try it out. Thanks for the suggestion!

@shahor02
Copy link
Copy Markdown
Contributor

yes

auto-merge was automatically disabled March 15, 2022 20:55

Head branch was pushed to by a user without write access

@mslupeck
Copy link
Copy Markdown
Contributor Author

So I don't see how to do it on a reasonable time/effort scale following @shahor02 suggestion. The problem is that missing BCData.h header is just the tip of an iceberg. The next error comes from the ChannelData, in which all the data members have changed names (to follow FT0's convention).

So for now, I propose to disable FV0 QC entirely. It should get the O2 PR through and then the FV0 QC could be reenabled (we are not planning to use FV0 QC for another 2 weeks or so).

@mslupeck mslupeck requested a review from knopers8 March 15, 2022 21:07
@knopers8
Copy link
Copy Markdown
Collaborator

FV0 seems to be enabled also in asynchronous reconstructions, are you sure you want to disable it for two weeks?
https://github.com/AliceO2Group/O2DPG/blob/b196f0055e6e0e5168399c386f58f9193f40657d/DATA/production/qc-workflow.sh#L45
That might require disabling it also there.

@mslupeck
Copy link
Copy Markdown
Contributor Author

@knopers8: I've just spoken with @chiarazampolli and it seems we still have a few weeks before everyone will be ready for apass4. And in case I'm late with reenabling the FV0 QC, she agreed to exclude the fv0 qc json manually.

So yes, I'm sure about disabling the FV0 QC.

@chiarazampolli
Copy link
Copy Markdown
Contributor

Hi @mslupeck ,

I did not really say "few weeks", what I meant is that we are not ready to start right away, I have no clear ETA, but it won't be an issue to disable FV0 QC in the apass4, as you wrote.

Chiara

@mslupeck
Copy link
Copy Markdown
Contributor Author

OK - sorry for the misunderstanding about the schedule of apass4. It's clear it is not settled yet. In any case, the exact timing of apass4 is not blocking this PR from merging.

@knopers8 knopers8 enabled auto-merge (squash) March 17, 2022 08:18
@knopers8 knopers8 merged commit 3d7d765 into AliceO2Group:master Mar 17, 2022
Barthelemy pushed a commit that referenced this pull request Mar 21, 2022
* [FIT] Follow data format changes in O2 #8344

* Temporarily disable FV0 QC

Co-authored-by: Maciej Slupecki <maciej.slupecki@cern.ch>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

4 participants