Skip to content

Conversation

@lyakh
Copy link
Collaborator

@lyakh lyakh commented Jun 4, 2024

Cleanly separate LLEXT code and data segments, handle .bss together with .data. Using Zephyr's automatic section grouping, based on feature flags.

lyakh added 3 commits June 4, 2024 15:51
Re-group headers more logically in eq_iir.c and mixin_mixout.c.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
When dynamically mapping memory, we need to update access flags
according to the type of the mapping.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
When an LLEXT module contains multiple Module Adapter instances,
their manifests are stored in an array in the .module section. Those
array entries contain per-instance information like module entry
points, names, UUIDs, but ELF information is common for all
instances. Store it in the first array entry to avoid confusion.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
@lyakh
Copy link
Collaborator Author

lyakh commented Jun 4, 2024

this also fixes the breakage caused by zephyrproject-rtos/zephyr#73571 and this builds with both Zephyr versions - before and after that change

@lyakh
Copy link
Collaborator Author

lyakh commented Jun 5, 2024

Before updating this PR to fix sparse warnings, just to record the present CI result: all clean apart from sparse (to be fixed) and jenkins on-device tests. The letter on MTL only had one device unavailable and the other two green, on LNL the result is strange, but must be unrelated https://sof-ci.01.org/sofpr/PR9190/build5052/devicetest/index.html - is this a known issue?

@lyakh lyakh force-pushed the llext-segment branch 2 times, most recently from 8e1570b to 2ed4f79 Compare June 5, 2024 08:12
llext_find_section() returns a negative error code when it cannot
find the requested section, not 0. Fix error checking.

Reported-by: Luca Burelli <l.burelli@arduino.cc>
Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
Copy link
Collaborator

@kv2019i kv2019i left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good except one thing in the second-last patch in the series, please see inline.


/* Writable data (.data, .bss, etc.) */
void __sparse_cache *va_base_data = (void __sparse_cache *)
ctx->segment[LIB_MANAGER_RODATA].addr;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lyakh Shouldn't this be LIB_MANAGER_DATA ? va_base_rodata already set above.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kv2019i oops, thanks for catching! The clean-up path - easy to miss without wide-scale automated testing...

Copy link
Collaborator

@kv2019i kv2019i left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, good now!

static int llext_manager_update_flags(void __sparse_cache *vma, size_t size, uint32_t flags)
{
size_t pre_pad_size = (uintptr_t)vma & (PAGE_SZ - 1);
void *aligned_vma = (__sparse_force uint8_t *)vma - pre_pad_size;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ALIGN_DOWN

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@softwarecki you could use it, but I also need pre_pad_size, so I open code it.

((uintptr_t)bss_addr + bss_size <= (uintptr_t)va_base_data ||
(uintptr_t)bss_addr >= (uintptr_t)va_base_data + data_size)) {
if ((uintptr_t)bss_addr + bss_size == (uintptr_t)va_base_data &&
!((uintptr_t)bss_addr & (PAGE_SZ - 1))) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ALIGN_DOWN

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't need to align, I check if it is aligned. So I could use IS_ALIGNED() but it's generic, using division, whereas here it's a power of 2, so a faster masking can be used.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So you can use IS_ALIGNED

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could, but I don't want to use division where masking works too.

void __sparse_cache *va_base = llext_manager_get_bss_address(module_id, mod);
ret = llext_manager_align_unmap(va_base_data, data_size);
if (ret < 0)
return ret;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't we try to unmap the rodata even if unmapping the data fails?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@softwarecki ok, this actually might make sense. Will update, thanks!

Copy link
Collaborator

@softwarecki softwarecki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My two cents

lyakh added 2 commits June 5, 2024 16:07
We map memory pages for loaded modules dynamically and we're able to
set memory flags for write access or for code execution. This commit
takes advantage of the recently added section grouping and maps the
three module parts separately: executable code, read-only data and
writable data, including zero-initialised .bss. This also cleans up
references, pointing into module storage in IMR instead of the mapped
addresses.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
Zephyr places .bss into a separate section element, still if it's
immediately adjacent to writable data we can merge and allocate them
together.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>
@lyakh
Copy link
Collaborator Author

lyakh commented Jun 6, 2024

CI failures:

@plbossart
Copy link
Member

yes, that's issue thesofproject/linux#4823. Clearly not related to the LLEXT work...

Copy link
Member

@lgirdwood lgirdwood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like all comments are addressed. @softwarecki ok for you now ?

@lgirdwood lgirdwood merged commit 01c0f18 into thesofproject:main Jun 7, 2024
@lyakh lyakh deleted the llext-segment branch June 9, 2024 15:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants