topology2: add multiple capture support with DMIC in nocodec path #6783

RanderWang · 2022-12-13T07:24:14Z

It is a simple pipeline for AEC according to the requirement of chrome team. In future one capture stream will be for AEC and anther one will be sent back to user directly.
AEC pcm id is 27. Table shared by Pierre

PCM0 playback Jack Out
PCM1 capture Jack In
PCM2 playback Speaker
PCM3 capture Amplifier Reference (typically based on I/V feedback)
PCM4 capture Microphone
PCM5 playback HDMI 1
PCM6 playback HDMI 2
PCM7 playback HDMI 3
PCM8 playback HDMI 4
PCM9 RESERVED
PCM10 capture DMIC
PCM11 capture DMIC16kHz
PCM12 capture BufferedMic (aka KPB)
PCM13 playback Bluetooth
PCM14 capture Bluetooth
PCM15 RESERVED
PCM16 RESERVED
PCM17 RESERVED
PCM18 RESERVED
PCM19 RESERVED
PCM20 capture LowLatency
PCM21 capture RawPDM
PCM22 capture Ultrasonics
PCM23 capture RawPCM
PCM24 capture Voice (Communication with a human)
PCM25 capture Speech (interaction with a machine)
PCM26 capture Context (aka Acoustic Context Analysis - ACA)
PCM27 capture Reference (e.g. for AEC. May be based on I/V feedback or on playback data)
PCM28 RESERVED
PCM29 RESERVED
PCM30 playback LowLatency
PCM31 playback DeepBuffer (for low-power)
PCM32 playback Media (e.g. movies)
PCM33 playback Voice (Communication with a human)
PCM34 playback Ultrasonics

plbossart · 2022-12-13T16:29:30Z

@RanderWang can you remind me why we need three copier.module for a simple demux functionality. Can't we just have one? Or are we missing an additional layer on top of the second copier/

ranj063 · 2022-12-13T16:32:56Z

@RanderWang can you remind me why we need three copier.module for a simple demux functionality. Can't we just have one? Or are we missing an additional layer on top of the second copier/

@plbossart this is because of the restriction with the ref FW that the gain module can only be connected to modules that are in the same pipeline as the gain module

tools/topology/topology2/platform/intel/dmic-aec.conf

tools/topology/topology2/cavs-nocodec.conf

RanderWang · 2022-12-14T06:58:37Z

@plbossart updated with second capture stream

lgirdwood · 2022-12-14T15:56:26Z

@cujomalainey table update good for you ?

lgirdwood · 2022-12-14T15:56:48Z

@dbaluta fyi

RanderWang · 2022-12-16T07:42:48Z

rebase to latest code

tools/topology/topology2/platform/intel/dmic-generic.conf

RanderWang · 2022-12-16T09:09:21Z

update for bard`s comments

RanderWang · 2022-12-16T13:18:12Z

wrap commit message to 75 char per line

tools/topology/topology2/platform/intel/dmic-default.conf

tools/topology/topology2/cavs-nocodec.conf

RanderWang · 2022-12-19T11:58:06Z

updated based on Pierre's advice
(1) change Reference to Raw
(2) remove host components in DMIC definition.

plbossart

couple more comments to make this PR shiny-clean. Thanks @RanderWang

tools/topology/topology2/cavs-nocodec.conf

tools/topology/topology2/cavs-rt5682.conf

tools/topology/topology2/cavs-sdw.conf

RanderWang · 2023-02-03T06:26:45Z

@lgirdwood it is good for me now. Only known issue on SDW, please check #7037 ipc timed out for 0x13020003 |GLB_SET_PIPELINE_STATE on TGLU_RVP_SDW_IPC4ZPH). Thanks @lyakh put great effort on this PR!

lyakh · 2023-02-03T08:23:11Z

@RanderWang haven't you forgotten to update tools/topology/topology2/include/pipelines/cavs/gain-module-copier.conf too with num_source_audio_formats and num_sink_audio_formats?

This patch will support multiple capture streams with DMIC. The pcm stream is composed of two pipelines: (1) copier host <- gain <- copier module (2) copier module <-gain <- copier DMIC. We need to use copier module to connect two pipelines because of the restriction with the ref FW that the gain module can only be connected to modules that are in the same pipeline Signed-off-by: Rander Wang <rander.wang@intel.com>

Currently copier components of the module type are used in pairs - one on each pipeline, connecting to each other. This isn't necessary. It's enough to have one such copier component on the side, that does forking. Add a single-copier capture pipeline variane and switch DMIC over to it. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

The second capture stream can be used as a reference stream for user. Signed-off-by: Rander Wang <rander.wang@intel.com>

Use the simplified pipeline with just one module-type copier connecting two pipelines. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

RanderWang · 2023-02-03T08:56:36Z

@RanderWang haven't you forgotten to update tools/topology/topology2/include/pipelines/cavs/gain-module-copier.conf too with num_source_audio_formats and num_sink_audio_formats?

Thanks! fix it.

lyakh · 2023-02-03T10:47:37Z

why have tests failed this time - on all of HDA, SDW and nocodec? I see an "IPC timed out" only in one of them?

kv2019i · 2023-02-03T11:28:10Z

Hmm, the tests still do not look good. One issue is known, this one https://sof-ci.01.org/sofpr/PR6783/build3698/devicetest/index.html?model=ADLP_RVP_NOCODEC_IPC4ZPH&testcase=multiple-pause-resume-5 is this #6864

But there are multiple other failures in https://sof-ci.01.org/sofpr/PR6783/build3698/devicetest/index.html that do not have the above signature and these don't happen in mainline. I checked #7037 but I don't see "ipc timed out for 0x13020003" in logs of the failed cases.

I do see #7025 passed the tests, so not sure what to make of it. Does this PR make cause existing bugs to be hit at higher reproduction rate? I'll kick off SOF_CI once more to get a bit more statistical data on this.

kv2019i · 2023-02-03T11:28:35Z

SOFCI TEST

RanderWang · 2023-02-03T13:11:24Z

@kv2019i , this PR is a copy of #7025. I don't find any error after retest now

lgirdwood · 2023-02-03T14:19:47Z

Tests looking good, but 3 DUTs were not available. Will rerun again in an hour.

Insert a second copier into the DMIC0 stream to generate a total of three PCMs from it. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

keqiaozhang · 2023-02-06T01:32:10Z

SOFCI TEST

RanderWang · 2023-02-06T05:34:47Z

SOFCI TEST

RanderWang · 2023-02-06T07:09:09Z

@lgirdwood @kv2019i only one failed: SDW. I checked the topology, as I know this PR doesn't affect it. @lyakh how about your idea ? Thanks!

lyakh · 2023-02-06T07:34:16Z

@lgirdwood @kv2019i only one failed: SDW. I checked the topology, as I know this PR doesn't affect it. @lyakh how about your idea ? Thanks!

I'm pretty certain https://sof-ci.01.org/sofpr/PR6783/build3727/devicetest/index.html?model=TGLU_RVP_SDW_IPC4ZPH&testcase=check-suspend-resume-with-playback is unrelated to this PR for reasons:

this PR touches only capture, the failure is with playback
the failure is where we often get failures - with suspend-resume
the failure is a timeout, not an actual run-time error

kv2019i · 2023-02-06T09:23:27Z

Thanks @RanderWang and @lyakh . The latest test results look good and together with the analysis, I think this is good to go. @plbossart good to go now?

RanderWang requested review from dbaluta, lbetlej, lgirdwood, mmaka1 and plbossart as code owners December 13, 2022 07:24

plbossart requested changes Dec 13, 2022

View reviewed changes

RanderWang force-pushed the topology2-aec branch 3 times, most recently from f2216b5 to ead0669 Compare December 14, 2022 06:56

RanderWang force-pushed the topology2-aec branch from ead0669 to ce56216 Compare December 14, 2022 07:01

RanderWang force-pushed the topology2-aec branch from ce56216 to c162c7c Compare December 16, 2022 07:42

RanderWang requested a review from bardliao December 16, 2022 08:07

bardliao reviewed Dec 16, 2022

View reviewed changes

tools/topology/topology2/platform/intel/dmic-generic.conf Outdated Show resolved Hide resolved

bardliao reviewed Dec 16, 2022

View reviewed changes

tools/topology/topology2/platform/intel/dmic-generic.conf Outdated Show resolved Hide resolved

bardliao reviewed Dec 16, 2022

View reviewed changes

tools/topology/topology2/platform/intel/dmic-generic.conf Outdated Show resolved Hide resolved

RanderWang force-pushed the topology2-aec branch from c162c7c to b8edfd5 Compare December 16, 2022 09:08

bardliao approved these changes Dec 16, 2022

View reviewed changes

RanderWang force-pushed the topology2-aec branch from b8edfd5 to e8e58cb Compare December 16, 2022 13:17

plbossart reviewed Dec 16, 2022

View reviewed changes

tools/topology/topology2/platform/intel/dmic-default.conf Outdated Show resolved Hide resolved

tools/topology/topology2/cavs-nocodec.conf Outdated Show resolved Hide resolved

RanderWang force-pushed the topology2-aec branch from e8e58cb to f88b43b Compare December 19, 2022 11:54

plbossart reviewed Dec 20, 2022

View reviewed changes

RanderWang force-pushed the topology2-aec branch 3 times, most recently from ee2c6c9 to a2bc134 Compare February 3, 2023 05:17

RanderWang and others added 4 commits February 3, 2023 16:49

topology2: add second capture stream for DMIC0

3acde7b

The second capture stream can be used as a reference stream for user. Signed-off-by: Rander Wang <rander.wang@intel.com>

topology2: remove a superfluous copier in nocodec

1d19080

Use the simplified pipeline with just one module-type copier connecting two pipelines. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

RanderWang force-pushed the topology2-aec branch 2 times, most recently from 7d09363 to b71a112 Compare February 3, 2023 08:56

topology2: nocodec: add a second level copier

db974a4

Insert a second copier into the DMIC0 stream to generate a total of three PCMs from it. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski@linux.intel.com>

RanderWang force-pushed the topology2-aec branch from b71a112 to db974a4 Compare February 6, 2023 01:43

kv2019i requested a review from plbossart February 6, 2023 09:21

lgirdwood approved these changes Feb 6, 2023

View reviewed changes

lgirdwood merged commit 0f7c8be into thesofproject:main Feb 7, 2023

This was referenced Feb 12, 2023

[BUG]: [Multiple Capture - No-codec mode]: Observed dragging and metallic noise from data captured from cascaded (second level) copier #7074

Closed

topology2: add second capture stream for DMIC0 for MTL topology #7085

Merged

yongzhi1 mentioned this pull request Jul 10, 2023

topology2: nocodec-multicore: Add connected pipelines #7847

Merged

topology2: add multiple capture support with DMIC in nocodec path #6783

topology2: add multiple capture support with DMIC in nocodec path #6783

Uh oh!

Conversation

RanderWang commented Dec 13, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

plbossart commented Dec 13, 2022

Uh oh!

ranj063 commented Dec 13, 2022

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

RanderWang commented Dec 14, 2022

Uh oh!

lgirdwood commented Dec 14, 2022

Uh oh!

lgirdwood commented Dec 14, 2022

Uh oh!

RanderWang commented Dec 16, 2022

Uh oh!

Uh oh!

Uh oh!

Uh oh!

RanderWang commented Dec 16, 2022

Uh oh!

RanderWang commented Dec 16, 2022

Uh oh!

Uh oh!

Uh oh!

RanderWang commented Dec 19, 2022

Uh oh!

plbossart left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

RanderWang commented Feb 3, 2023

Uh oh!

lyakh commented Feb 3, 2023

Uh oh!

RanderWang commented Feb 3, 2023

Uh oh!

lyakh commented Feb 3, 2023

Uh oh!

kv2019i commented Feb 3, 2023

Uh oh!

kv2019i commented Feb 3, 2023

Uh oh!

RanderWang commented Feb 3, 2023

Uh oh!

lgirdwood commented Feb 3, 2023

Uh oh!

keqiaozhang commented Feb 6, 2023

Uh oh!

RanderWang commented Feb 6, 2023

Uh oh!

RanderWang commented Feb 6, 2023

Uh oh!

lyakh commented Feb 6, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kv2019i commented Feb 6, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

RanderWang commented Dec 13, 2022 •

edited

Loading

lyakh commented Feb 6, 2023 •

edited

Loading