Skip to content

Conversation

@marc-hb
Copy link
Collaborator

@marc-hb marc-hb commented May 2, 2023

This replaces and simplifies commit a823958 ("xtensa-build-zephyr: pass sof build version to rimage") with a constant.

EDIT: That commit was submitted to avoid a test failure in a broken, internal
test looking for an SOF_BUILD value... hardcoded to 1! (internal FW issue
257, test TestLoadFwExtended::()::test_00_01_load_fw_and_check_version")

The final goal is for this script to stop extracting anything from generated/sof_versions.h so rimage can be configured before the build has started. Other commits will deal with other parts of sof_versions.h

SOF_BUILD can be replaced by a constant because it has always been one when building with Zephyr. The optional BLD_COUNTERS feature - disabled by default in XTOS since commit 9f8cce1 ("Disable TIME and the non-reproducible build counter by default") for build reproducibility reasons - has never worked with Zephyr for the following reasons:

  • The sof_add_build_counter_rule() invocation is located in the top-level sof/CMakeLists.txt file which has never been used by the Zephyr build.

  • sof_add_build_counter_rule() registers a POST_BUILD command for the top-level "sof" CMake target but there is no "sof" build target in the Zephyr build.

  • Variables like VERSION_BUILD_COUNTER_CMAKE_PATH are not defined in the Zephyr build for some unknown reason.

  • Probably others.

@marc-hb
Copy link
Collaborator Author

marc-hb commented May 2, 2023

https://sof-ci.01.org/sofpr/PR7546/build6851/build/mtl_zph_ipc4.txt is still invoking rimage with -b 1 as it always has.

@marc-hb marc-hb marked this pull request as ready for review May 2, 2023 05:27
@marc-hb
Copy link
Collaborator Author

marc-hb commented May 2, 2023

https://sof-ci.01.org/sof-pr-viewer/#/build/PR7546/build11924100 has the usual error below.

Everything else is green including https://sof-ci.01.org/sofpr/PR7546/build6853/devicetest and https://sof-ci.01.org/sofpr/PR7546/build6852/devicetest

@wszypelt a green report would be so unusual that it would be almost worrying!

cc:

22:33:53,387 INFO  - Executing pre-execute action...
22:33:53,387 INFO  - Running step...
22:42:34,319 INFO  - Executing post-execute action...
22:42:34,319 ERROR - Step 'master>SCANS>TESTS>TRIGGER PYTHON SMOKE_IPC4 TESTS' is failed: Step is failed since the triggered build is failed, cancelled, or timed out.

@marc-hb
Copy link
Collaborator Author

marc-hb commented May 5, 2023

Someone re-run https://sof-ci.01.org/sof-pr-viewer/#/build/PR7546/build11933012 and even that is green now. Everything is.

Copy link
Contributor

@aborisovich aborisovich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, the PR looks OK but I am not fan of the solution you took for
sign_cmd += ["-b", "1" ] . It makes no sense in my opinion because it is just hardcoded value in the script without any specific reason. Rest of the PR is ok but I would like to push you for a bit more, because you wrote that

The final goal is for this script to stop extracting anything from generated/sof_versions.h so rimage can be configured before the build has started. Other commits will deal with other parts of sof_versions.h

So, if this statement is a motivation for this PR, I think you can go further and refactor a bit how the version is generated. What I mean is:

  1. From SOF configure rimage using FetchContent and pass SOF FW version to rimage CMake at configure time.
  2. Change rimage - if no command line arguments provided with version number, use the one from CMake cache.
  3. Remove completely get_sof_version(abs_build_dir) function from xtensa-build-zephyr.py.
  4. Generate sof_versions.h file using configure_file function.

This approach would finally simplify version handling and make it more robust.
We would pass if from CMake down, not from Cmake to python to west to rimage command args.
SOF versioning is not bount in any way to platform we build for, so there should be no problem with this approach. I think it matches also the idea of having rimage producing .ri images as part of "build" step not "sign" step.

Comment on lines 678 to 684
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like this one... We version SOF FW not Zephyr. If rimage has build counter that we do not use, we should stick to default values provided in rimage (and eventually consider removing this build counter from rimage if it has no use cases).

@marc-hb
Copy link
Collaborator Author

marc-hb commented May 17, 2023

Thanks for the review! Very hard to find reviewers in this area.

but I am not fan of the solution you took for sign_cmd += ["-b", "1" ] . It makes no sense in my opinion because it is just hardcoded value in the script without any specific reason

So the rationale for hardcoding sign_cmd += ["-b", "1" ] was to keep the binary 100% unchanged. Thanks for pointing this out; it's not a very good reason. In fact I'm now leaning toward dropping this line. Then rimage will default to -b 0 when there is no -b. Also, if I drop this line then users can more easily override the value using west config or whatever other CMake way they find without wondering what happens when there are two -b options on the same rimage command line; always confusing.

Would dropping sign_cmd += ["-b", "1" ] work for you?

(and eventually consider removing this build counter from rimage if it has no use cases).

Maybe. SOF_BUILD is disabled by default but it still works in XTOS builds though. Maybe some SOF_BUILD fan could also find a way to finally make it work when building Zephyr? Dunno.

pass SOF FW version to rimage CMake at configure time.

There seems to be a misunderstanding here? -b is an rimage parameter passed when running rimage.

Generate sof_versions.h file using configure_file function.

But this would make sof_versions.h more static that it is now? See commit 23d2a01. Either way it looks off-topic to me in this PR but maybe I'm missing something.


FetchContent() suggestion.

Adopting FetchContent() would be a much bigger change than this small, "no-op" PR! I mean SOF_BUILD never worked before this small PR and it still does not work after this PR and that's pretty much it...

I see where you're coming from because FetchContent() clearly looks like a tool designed to solve dependencies like SOF=>rimage . However I see a number of reasons why FetchContent would not fit here. Main reason: I think it would go against the spirit and the letter of the Zephyr build.

Please start a new issue if you want to keep recommending FetchContent()?

@aborisovich
Copy link
Contributor

aborisovich commented May 17, 2023

Would dropping sign_cmd += ["-b", "1" ] work for you?

Sure, it's my only concern.
Maybe I'll submit a feature request about my idea. Because you see, FetchContent may but does not have to clone. We may still stick to west in terms of cloning, just use functions from fetch content to populate the CMake cache of rimage and run its configure step as part of SOF configure step. Then, built rimage executable would have 'hardcoded version' of SOF inside of it. And we could use that version when no rimage input arguments related to version are provided (aka 'default'). So rimage would use not only information provided in the imput args but also information given from the build. Well, anyway that was just an idea.
I think its worth considering because:

  • rimage anyway relies on other data from SOF (headers?)
  • we parse version information from SOF using python regex just to pass it as runtime input argument of rimage executable
  • so in fact we 'pretend' rimage to be 'stand-alone' tool while it strongly depends on SOF anyway.
  • changing SOF version or revision would cause change in rimage cache and would trigger rebuilding to contain new proper version

This replaces and simplifies commit a823958 ("xtensa-build-zephyr:
pass sof build version to rimage") with a constant 1.

That commit was submitted to avoid a test failure in a broken, internal
test looking for an SOF_BUILD value... hardcoded to 1! (internal FW issue
257, test TestLoadFwExtended::()::test_00_01_load_fw_and_check_version")

The final goal is for this script to stop extracting anything from
`generated/sof_versions.h` so rimage can be configured _before_ the build
has started. Other commits will deal with other parts of sof_versions.h

SOF_BUILD can be replaced by a constant because it has always been one
when building with Zephyr. The optional BLD_COUNTERS feature - disabled
by default in XTOS since commit 9f8cce1 ("Disable __TIME__ and the
non-reproducible build counter by default") for build reproducibility
reasons - has never worked with Zephyr for the following reasons:

- The sof_add_build_counter_rule() invocation is located in the
top-level sof/CMakeLists.txt file which has never been used by the
Zephyr build.

- sof_add_build_counter_rule() registers a POST_BUILD command for the
top-level "sof" CMake target but there is no "sof" build target in the
Zephyr build.

- Variables like VERSION_BUILD_COUNTER_CMAKE_PATH are not defined in the
Zephyr build for some unknown reason.

- Probably others.

Signed-off-by: Marc Herbert <marc.herbert@intel.com>
@marc-hb marc-hb force-pushed the constant-sof-build branch from d8cf82b to 33fc7a3 Compare May 17, 2023 06:32
@marc-hb
Copy link
Collaborator Author

marc-hb commented May 17, 2023

Would dropping sign_cmd += ["-b", "1" ] work for you?

I was dropping this line and then I had second thoughts and I took a second look at #6276. I found internal issue 257. So the whole reason SOF_BUILD was added to xtensa-build-zephyr.py in the first place is apparently because some broken test is checking for SOF_BUILD to be hardcoded to 1!! So I kept the hardcoding for now and added a FIXME :-(

zmain failures are known and unrelated: #7614

Copy link
Contributor

@aborisovich aborisovich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So the whole reason SOF_BUILD was added to xtensa-build-zephyr.py in the first place is apparently because some broken test is checking for SOF_BUILD to be hardcoded to 1!!

This is just sad...

@kv2019i
Copy link
Collaborator

kv2019i commented May 17, 2023

Seems ready to go, but need a clean CI pass for required checks.

@marc-hb
Copy link
Collaborator Author

marc-hb commented May 17, 2023

Seems ready to go, but need a clean CI pass for required checks.

The Ubuntu mirrors were very unreliable yesterday. Today they worked!

https://sof-ci.01.org/sofpr/PR7546/build7928/devicetest has a couple failures as usual, nothing that could be caused by this.

The zmain branch needs the Zephyr SDK 0.16.1, known issue.

Everything else is green now.

@marc-hb marc-hb added this to the v2.6 milestone May 17, 2023
@kv2019i kv2019i merged commit 2d3c7ea into thesofproject:main May 19, 2023
@marc-hb marc-hb deleted the constant-sof-build branch May 19, 2023 22:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants