Do not allocate module memory containers one by one, but allocate them
in chunks.

The bookkeeping of allocated resources is done using containers that
are allocated from heap. This effectively doubles the amount of heap
allocations. This is not very efficient especially since the containers
are only 20 bytes in size.

This commit changes the allocation of containers so that they are
always allocated in chunks of 16 containers, or what is selected with
MODULE_MEMORY_API_CONTAINER_CHUNK_SIZE Kconfig option. The unused
containers are not freed when the associated resource is
freed. Instead the unused containers are kept in free containers
list. All the containers are freed when mod_free_all() is called, for
instance when the module unloads.

Signed-off-by: Jyri Sarha <jyri.sarha@linux.intel.com>

Add heap usage high water mark to module_adapter_heap_usage() and
shell's "sof module_heap_usage" command.

Signed-off-by: Jyri Sarha <jyri.sarha@linux.intel.com>

Add src/audio/data_blob.c to mixer cmocka test sources to fix the
dependency problem from adding comp_data_blob_handler_new_ext() to
audio/module_adapter/module/generic.c.

Signed-off-by: Jyri Sarha <jyri.sarha@linux.intel.com>

Add mod_data_blob_handler_new() to module API. The function is
otherwise the same as comp_data_blob_handler_new(), but it takes a
module pointer as the first argument, and the blob handler is
automatically freed when the module unloads.

The handler allocated with mod_data_blob_handler_new() should not be
freed with comp_data_blob_handler_free(), mod_data_blob_handler_free()
should be used.

Signed-off-by: Jyri Sarha <jyri.sarha@linux.intel.com>

Add module API versions of fast_get() and fast_put(). The SRAM copies
reserved with mod_fast_get() are released automatically when the
module unloads, and those SRAM copies should not be freed with the
regular fast_put().

Signed-off-by: Jyri Sarha <jyri.sarha@linux.intel.com>

Add safeguard to mod_alloc() and friends that checks that they are
always called from the same thread (e.g. no locking needed).

The checking code has to be also behind defined(__ZEPHYR__) to keep
cmocka tests working.

Signed-off-by: Jyri Sarha <jyri.sarha@linux.intel.com>

Take mod_fast_get() and mod_fast_put() into use.

Signed-off-by: Jyri Sarha <jyri.sarha@linux.intel.com>

Allocate all memory through module API mod_alloc() and friends
and remove all redundant rfree() calls from module unload
functions and init error branches.

Signed-off-by: Jyri Sarha <jyri.sarha@linux.intel.com>

Allocate all memory, blob handlers, and fast_get() buffers through
module API mod_alloc() and friends and remove all redundant rfree(),
comp_data_blob_handler_free(), and fast_put() calls from module unload
functions and init error branches.

Signed-off-by: Jyri Sarha <jyri.sarha@linux.intel.com>

Allocate all memory through module API mod_alloc() and friends
and remove all redundant rfree() calls from module unload
functions and init error branches.

NOTE: copier_dai.c and copier_host.c still have their shared memory
allocated through the old API. This is to be fixed once we have
decided on how the shared memory allocations should work in
user-space.

Signed-off-by: Jyri Sarha <jyri.sarha@linux.intel.com>

Allocate all memory through module API mod_alloc() and friends
and remove all redundant rfree() calls from module unload
functions and init error branches.

Signed-off-by: Jyri Sarha <jyri.sarha@linux.intel.com>

Allocate all memory, blob handlers, and fast_get() buffers through
module API mod_alloc() and friends and remove all redundant rfree(),
comp_data_blob_handler_free(), and fast_put() calls from module unload
functions and init error branches.

Signed-off-by: Jyri Sarha <jyri.sarha@linux.intel.com>

Allocate all memory, blob handlers, and fast_get() buffers through
module API mod_alloc() and friends and remove all redundant rfree(),
comp_data_blob_handler_free(), and fast_put() calls from module unload
functions and init error branches.

Signed-off-by: Jyri Sarha <jyri.sarha@linux.intel.com>

Split comp_alloc() into two parts - allocation and initialisation to
be able to re-use the initialisation code with a different allocation
method.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

Add sof_heap_alloc() and sof_heap_free() to allocate and free memory
on a private heap.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

We need to move IPC processing for DP scheduled components into their
thread context. For that the thread has to be started early. Create
it immediately when creating DP task context.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

We need to run all module callbacks in DP thread context, for this
the thread has to be started early - before the first module callback
is called.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

DP processing threads should have as long as life time as possible to
process all the relevant IPCs in the thread context. Move thread
termination to be called immediately before freeing module data.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

src_set_config() and src_get_config() aren't used, they would return
an error if ever called. It's easier to just remove them.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

Move a conditional clause inside a previous one to simplify it.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

Add functions for adding LLEXT partitions to a memory domain for
user-space modules.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

We want to be able to serve all module memory allocations from a
private heap. This commit creates such a heap for DP scheduled
modules and moves struct comp_dev and struct processing_module to it.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

Move mod_alloc() allocations, including the container pool, to the
module local heap.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

Allocate the ring-buffer object on module heap too for DP access.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

The ring-buffer data buffer has to be accessible to user-space DP
modules, allocate it on module heap.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

Pass a "heap" argument to fast_get() and fast_put() for user-space
DP allocations.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

Move the MM_DRV_INTEL_VIRTUAL_REGION_COUNT option close to another
MM_DRV option.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

Extract resource initialization into a separate function in
preparation for user-space changes.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

When running in user-space module code should only be executed in its
DP thread context. This includes IPC and any other module call-backs.
Modify the scheduler to support that flow.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

Instead of calling module hooks inline signal the DP thread to call
them in the thread context in DP-scheduled module case.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

mod_alloc(), mod_data_blob_handler_free(), mod_fast_put() and
mod_zalloc() are trivial wrappers, make them inline functions. Also
use size_t for size arguments.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

Add a heap parameter to buffer allocation functions. This makes
buffer structures accessible to the user-space.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

Add a new mod_alloc_ext() allocation function with support for
allocator flags.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

The only functions that have to be converted to syscalls are
mod_alloc_ext() and mod_free(), the rest of the API is implemented
using inline functions.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

The private part of struct struct processing_module contains a list
head, add the respective header.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

Variable size structure members are only allowed at the end.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

Some Zephyr heap allocations use per-thread heap pointers. By default
those allocations end up using the default Zephyr system heap, which
is rather small in the SOF case. To overcome that assign the common
SOF heap to EDF and IDC threads.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

When a LLEXT module is freed, its partitioins should be removed from
any memory domains. Add a function for that.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

Run DP threads in user-space. Move all the respective memory and
kobjects to a dedicated memory domain. Work around Zephyr inability
to remove memory domains on Xtensa.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

Make lib_manager_allocate_module() static also for the
CONFIG_MM_DRV=n case. Also don't drop the const qualifier needlessly.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

A Zephyr Pull Request is needed, exporting xtensa_domain_list.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

src_get_copy_limits() only returns false if both
sink_get_free_frames(sink) and
source_get_data_frames_available(source) return 0. They are
calculated in the beginning of the function, the rest of the
function is calculating intermediate values, used later for sample
calculation. But src_get_copy_limits(() is also called in
src_process(), so inside src_is_ready_to_process() it's enough to
perform a much simpler check, skipping all the calculations.

Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>