-
Notifications
You must be signed in to change notification settings - Fork 349
config: mtpm: Change Google RTC Audio Processing to Module Configuration #9730
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
config: mtpm: Change Google RTC Audio Processing to Module Configuration #9730
Conversation
lgirdwood
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@lyakh could we make all the mocks into modules and then load then into CI via UUID ?
@lgirdwood sure, but we'd need topologies that include those stub components to load and test them |
|
@tmleman please compile AEC as a LLEXT loadable module, we don't need it in production. EDIT: it may also require changes in testing, a module has to be loaded |
@marcinszkudlinski so it is run-time tested in QB? |
test for AEC + Mock is included in internal CI tests |
9965610 to
4f70175
Compare
lyakh
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd swap the commits - first convert to "m" where already enabled, and then add directly as "m" where not enabled yet, but the result is the same
|
@tmleman can you check Internal CI thanks ! |
|
@lgirdwood PR must wait until the required changes in the tests are implemented. Validation is working on it. |
727d836 to
8a77adf
Compare
@tmleman I assume we have a valid test implemented now and its being run in CI ? |
|
This module had an incorrect byte order in uuid, this was fixed by #9793. For me its good to merge. |
|
@tmleman This has one quickbuild failing. Otherwise looks good for merge. |
24dbb6b to
8048e70
Compare
|
@lgirdwood Previously, MTL CI failed in the RTC test due to an incorrect UUID. The problem was fixed by @softwarecki recent changes. Currently, the same test fails during module initialization: [ 0.692421] <inf> ipc: ipc_cmd: rx : 0x40005000|0x12010026
[ 0.692775] <wrn> ipc: ipc4_get_drv: get_drv(): the provided UUID (b780a0a6-269f-466f-b477-23dfa05af758) can't be found!
[ 0.695606] <wrn> llext: llext_link_plt: PLT: cannot find idx 53 name ipc4_update_source_format
[ 0.697148] <wrn> llext: llext_link_plt: PLT: cannot find idx 39 name comp_is_current_data_blob_valid
[ 0.701273] <wrn> llext: llext_link_plt: PLT: cannot find idx 53 name ipc4_update_source_format
[ 0.710821] <wrn> llext: llext_link_plt: PLT: cannot find idx 39 name comp_is_current_data_blob_valid
[ 0.719726] <inf> llext: llext_load: Loaded extension RTC_AEC
[ 0.721906] <err> lib_manager: lib_manager_module_create: lib_manager_allocate_module() failed!
[ 0.721948] <err> ipc: ipc4_init_module_instance: error: failed to init module 5000 : 0
[ 0.721991] <err> ipc: ipc_cmd: ipc4: MODULE_MSG failed with err 104
[ 0.956415] <inf> ipc: ipc_cmd: rx : 0x12010000|0x0 |
@lyakh do we need a west update ? |
@lgirdwood I suppose this is an excerpt from an Internal CI log? I don't think there's anything relevant in recent Zephyr changes, that would fix this, and we don't have a topology with this module, so I cannot test, sorry. But since it's a proper reproducible error with a full log, it should be rather easy to debug |
8048e70 to
77b1587
Compare
|
@wszypelt good to merge ? |
|
@lgirdwood Unfortunately, we still have an issue with RTC_AEC (exception) |
@tmleman @wszypelt is this with the stub or with the real module ? Can we share here to help debug ? Thanks |
@lgirdwood @lyakh In the tests, we use a mock. [ 1.681920] <inf> ipc: ipc_cmd: rx : 0x40008000|0x12010026
[ 1.682308] <wrn> ipc: ipc4_get_drv: get_drv(): the provided UUID (b780a0a6-269f-466f-b477-23dfa05af758) can't be found!
[ 1.686383] <wrn> llext: llext_link_plt: PLT: cannot find idx 22 name comp
[ 1.700713] <wrn> llext: llext_link_plt: PLT: cannot find idx 22 name comp
[ 1.705181] <wrn> llext: llext_link_plt: PLT: cannot find idx 22 name comp
[ 1.714426] <wrn> llext: llext_link_plt: PLT: cannot find idx 37 name module_ada
[ 1.716853] <inf> llext: llext_load: Loaded extension RTC_AEC
[ 1.721130] <inf> google_rtc_audio_processing_init: comp:1 0x8000 google_rtc_audio_processing_init()
[ 1.721228] <err> os: print_fatal_exception: ** FATAL EXCEPTION
[ 1.721283] <err> os: print_fatal_exception: ** CPU 2 EXCCAUSE 12 (instr PIF data error)
[ 1.721356] <err> os: print_fatal_exception: ** PC (nil) VADDR (nil)
[ 1.721385] <err> os: print_fatal_exception: ** PS 0x60d20
[ 1.721413] <err> os: print_fatal_exception: ** (INTLEVEL:0 EXCM: 0 UM:1 RING:0 WOE:1 OWB:13 CALLINC:2)
[ 1.721445] <err> os: xtensa_dump_stack: ** A0 0xa068b220 SP 0xa00c74e0 A2 (nil) A3 0x2
[ 1.721473] <err> os: xtensa_dump_stack: ** A4 0x2 A5 0xbb80 A6 0x2 A7 (nil)
[ 1.721501] <err> os: xtensa_dump_stack: ** A8 0xa068c0d6 A9 0xa006fbf0 A10 (nil) A11 0x1
[ 1.721530] <err> os: xtensa_dump_stack: ** A12 0x14 A13 0x40 A14 (nil) A15 (nil)
[ 1.721558] <err> os: xtensa_dump_stack: ** LBEG 0xa00854d8 LEND 0xa00854de LCOUNT (nil)
[ 1.721585] <err> os: xtensa_dump_stack: ** SAR 0x12
[ 1.721611] <err> os: xtensa_dump_stack: ** THREADPTR 0x34000[ 0.392040] <inf> ipc: ipc_cmd: rx : 0x13000004|0x1
[ 0.392473] <inf> copier: copier_prepare: comp:1 0x30005 copier_prepare()
[ 0.392615] <inf> copier: copier_prepare: comp:1 0x20005 copier_prepare()
[ 0.393160] <inf> copier: copier_prepare: comp:0 0x5 copier_prepare()
[ 0.393246] <inf> google_rtc_audio_processing_prepare: comp:0 0x8000 google_rtc_audio_processing_prepare()
[ 0.393291] <wrn> google_rtc_audio_processing_prepare: comp:0 0x8000 Too many mic channels: 4, truncating to 2
[ 0.393443] <inf> module_adapter: module_adapter_prepare: comp:0 0x8000 DP Module period set to 10000
[ 0.393665] <inf> copier: copier_prepare: comp:0 0x10005 copier_prepare()
[ 0.393758] <inf> pipe: pipeline_trigger: pipe:1 0x0 pipe trigger cmd 7
[ 0.393895] <inf> ll_schedule: zephyr_ll_task_schedule_common: task add 0xa00db6c0 0xa00ab394U priority 0 flags 0x0
[ 0.394031] <inf> ll_schedule: zephyr_domain_register: zephyr_domain_register domain->type 1 domain->clk 0 domain->ticks_per_ms 38400 period 1000
[ 0.394395] <inf> copier: copier_comp_trigger: comp:1 0x30005 No dai copier found, start/end offset is not calculated
[ 0.394685] <inf> copier: copier_comp_trigger: comp:1 0x30005 No dai copier found, start/end offset is not calculated
[ 0.394940] <inf> pipe: pipeline_trigger: pipe:0 0x0 pipe trigger cmd 7
[ 0.395001] <inf> clock: clock_set_freq: clock 0 set freq 393216000Hz freq_idx 1 old 0
[ 0.395006] <inf> clock: clock_set_freq: clock 1 set freq 393216000Hz freq_idx 1 old 0
[ 0.395011] <inf> clock: clock_set_freq: clock 2 set freq 393216000Hz freq_idx 1 old 0
[ 0.395021] <inf> ll_schedule: zephyr_ll_task_schedule_common: task add 0xa00dbd40 0xa00ab394U priority 0 flags 0x0
[ 0.395066] <inf> copier: copier_comp_trigger: comp:0 0x5 No dai copier found, start/end offset is not calculated
[ 0.395086] <inf> copier: copier_comp_trigger: comp:0 0x5 No dai copier found, start/end offset is not calculated
[ 0.395100] <inf> ll_schedule: zephyr_ll_task_schedule_common: task add 0xa00d38c8 0xa00aaf44U priority 0 flags 0x0
[ 0.395136] <inf> host_comp: host_get_copy_bytes_normal: comp:0 0x10005 no bytes to copy, available samples: 0, free_samples: 768
[ 0.396061] <inf> host_comp: host_get_copy_bytes_normal: comp:1 0x30005 no bytes to copy, available samples: 384, free_samples: 0
[ 0.396070] <err> os: print_fatal_exception: ** FATAL EXCEPTION
[ 0.396076] <err> os: print_fatal_exception: ** CPU 0 EXCCAUSE 12 (instr PIF data error)
[ 0.396081] <err> os: print_fatal_exception: ** PC (nil) VADDR (nil)
[ 0.396085] <err> os: print_fatal_exception: ** PS 0x60720
[ 0.396088] <err> os: print_fatal_exception: ** (INTLEVEL:0 EXCM: 0 UM:1 RING:0 WOE:1 OWB:7 CALLINC:2)
[ 0.396091] <err> os: xtensa_dump_stack: ** A0 0xa0078faf SP 0xa00c04c0 A2 (nil) A3 0xa00dba00
[ 0.396096] <err> os: xtensa_dump_stack: ** A4 (nil) A5 0xa00dba80 A6 0x1 A7 0xa00daf00
[ 0.396100] <err> os: xtensa_dump_stack: ** A8 0xa007ceba A9 0x2 A10 0xa00db244 A11 (nil)
[ 0.396103] <err> os: xtensa_dump_stack: ** A12 0xa00db384 A13 (nil) A14 0xc0 A15 0x76543210
[ 0.396106] <err> os: xtensa_dump_stack: ** LBEG 0xa0036cd5 LEND 0xa0036ce4 LCOUNT 0xa0062b5f
[ 0.396110] <err> os: xtensa_dump_stack: ** SAR 0x2
[ 0.396115] <err> os: xtensa_dump_stack: ** THREADPTR 0x5 |
|
@tmleman this looks like an assert in user mode ?? OR is our user mode stack big enough for each of these modules ? |
|
@tmleman can you rebase and push - there has been a llext fixed merged that may fix here. |
@tmleman sorry, don't understand: is the problem still there or is it gone with a rebase? If it's still there, I understand, that you reproduce it manually ATM, since PR CI results don't have that failure. Can we add that test to the PR QB test? |
0707dfc to
db8da91
Compare
|
@lgirdwood @lyakh I performed a rebase, twice. The first time, the FW passed through CI but not seeing fixes that could affect the problem I'm experiencing with this PR, I decided to hold off on removing the DNM label. After the second rebase, the problem appeared again: [ 0.901898] <inf> copier: copier_prepare: comp:3 0x50005 copier_prepare()
[ 0.902938] <inf> copier: copier_prepare: comp:1 0x30005 copier_prepare()
[ 0.903273] <inf> google_rtc_audio_processing_prepare: comp:1 0x5000 google_rtc_audio_processing_prepare()
[ 0.903326] <wrn> google_rtc_audio_processing_prepare: comp:1 0x5000 Too many mic channels: 4, truncating to 2
[ 0.903483] <err> os: print_fatal_exception: ** FATAL EXCEPTION
[ 0.903538] <err> os: print_fatal_exception: ** CPU 2 EXCCAUSE 12 (instr PIF data error)
[ 0.903568] <err> os: print_fatal_exception: ** PC (nil) VADDR (nil)
[ 0.903593] <err> os: print_fatal_exception: ** PS 0x60d20
[ 0.903621] <err> os: print_fatal_exception: ** (INTLEVEL:0 EXCM: 0 UM:1 RING:0 WOE:1 OWB:13 CALLINC:2)
[ 0.903653] <err> os: xtensa_dump_stack: ** A0 0xa0688770 SP 0xa00bdbb0 A2 0x400d8200 A3 0x2
[ 0.903680] <err> os: xtensa_dump_stack: ** A4 0x400d8140 A5 0x2 A6 0xa00d83c0 A7 0xa00bdbb0
[ 0.903706] <err> os: xtensa_dump_stack: ** A8 0xa0688cf2 A9 0xa00bdbe0 A10 0xa00d8480 A11 0x1fffffff
[ 0.903733] <err> os: xtensa_dump_stack: ** A12 0x1 A13 0x40 A14 (nil) A15 0x400cd400
[ 0.903760] <err> os: xtensa_dump_stack: ** LBEG 0xa003a2da LEND 0xa003a2e7 LCOUNT 0xa00800bc
[ 0.903786] <err> os: xtensa_dump_stack: ** SAR 0x14
[ 0.903813] <err> os: xtensa_dump_stack: ** THREADPTR 0x400cd408
Backtrace:0xfffffffd:0xa00bdbb0 |<-CORRUPTED
[ 0.904111] <err> os: z_fatal_error: >>> ZEPHYR FATAL ERROR 0: CPU exception on CPU 2
[ 0.904160] <err> os: z_fatal_error: Current thread: 0x400cd080 (unknown)
[ 0.913080] <err> coredump: coredump_mem_window_backend_start: #CD:BEGIN#
[ 0.913315] <err> pipe: pipeline_prepare: pipe:1 0x0 pipeline_prepare(): ret = -11, dev->comp.id = 0x30005
[ 0.914028] <err> coredump: coredump_mem_window_backend_end: #CD:END#
[ 0.914396] <err> ipc: ipc4_pcm_params: ipc: pipe 1 comp 0 prepare failed -11
[ 0.914673] <err> zephyr: k_sys_fatal_error_handler: Halting system
[ 0.915315] <wrn> p4wq: k_p4wq_submit: Out of worker threads, priority guarantee violated
[ 0.926428] <err> pipe: pipeline_reset: pipe:1 0x0 pipeline_reset(): ret = -11, host->comp.id = 0x30005
[ 0.927103] <err> ipc: ipc4_pcm_params: ipc: pipe 1 comp 0 reset failed -11
[ 0.927598] <err> ipc: ipc_cmd: ipc4: FW_GEN_MSG failed with err 7
The test on RTC is in CI and the above log comes from it. The exception thrown by the DSP I'm debugging appears and disappears when I perform a rebase. |
Interesting, does a pristine build fix the issue ? or even delete the build directory and build from scratch ? |
db8da91 to
84851b5
Compare
84851b5 to
eaf6dbd
Compare
| # This mock is part of official sof-bin releases because the CI that | ||
| # tests it can't use extra CONFIGs. See #9410, #8722 and #9386 | ||
| CONFIG_COMP_GOOGLE_RTC_AUDIO_PROCESSING=y | ||
| CONFIG_COMP_GOOGLE_RTC_AUDIO_PROCESSING=m |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
just to make sure this is intended - we currently don't package loadable modules on MTL, so this will exclude this module from the binary distribution
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is fine as this module only used for Chrome configs.
app/boards/intel_adsp_ace20_lnl.conf
Outdated
| # This mock is part of official sof-bin releases because the CI that | ||
| # tests it can't use extra CONFIGs. See #9410, #8722 and #9386 | ||
| CONFIG_COMP_GOOGLE_RTC_AUDIO_PROCESSING=y | ||
| CONFIG_COMP_GOOGLE_RTC_AUDIO_PROCESSING=m |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
on LNL we so far only enable DTC as a loadable modules, so this may remove this module from the binary distribution
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ditto re chrome configs.
|
looks like the CI stalled or DUTs were in use. rerun. |
|
SOFCI TEST |
eaf6dbd to
1d83b07
Compare
|
Update: enabled RTC_AEC on WCL. |
1d83b07 to
a07f205
Compare
This patch exports the symbols `comp_is_current_data_blob_valid` and `ipc4_update_source_format` to resolve issues encountered when the Google RTC module was converted into an LLEXT module. The lack of exported symbols was causing warnings and subsequent firmware exceptions when these functions were called from within the module. Changes include: - Adding `EXPORT_SYMBOL` for `comp_is_current_data_blob_valid` in src/audio/data_blob.c. - Adding `EXPORT_SYMBOL` for `ipc4_update_source_format` in src/ipc/ipc4/helper.c. Signed-off-by: Tomasz Leman <tomasz.m.leman@intel.com>
This patch modifies the configuration of the Google RTC Audio Processing component from being built-in (`y`) to a loadable module (`m`). This change allows for more flexibility in managing the component, as it can now be loaded or unloaded dynamically. The adjustment aligns with the need for modularity and adaptability in the SOF firmware, especially in environments where resource management and configuration customization are critical. Signed-off-by: Tomasz Leman <tomasz.m.leman@intel.com>
a07f205 to
5c83b22
Compare
|
Dropping the enabling of the module on ACE20 and ACE30 platforms. Debugging everything at once slows down work. Switching RTC AEC on MTL to a loadable module does not lead to any visible regression. In the next step, the module will be enabled for PTL in a separate PR. |
This pull request updates the configuration for the Google RTC Audio Processing component in the MTL firmware. The component's configuration has been changed from being statically included (
y) to a loadable module (m). This adjustment enhances the flexibility and modularity of the firmware, allowing dynamic loading and unloading of the component as needed. It aligns with the project's goals of improving resource management and adaptability in various deployment environments.Changes:
CONFIG_COMP_GOOGLE_RTC_AUDIO_PROCESSINGfromytominintel_adsp_ace15_mtpm.conf.Impact:
Please review the changes and provide feedback or approval for merging.