Skip to content

Conversation

@osagie98
Copy link
Contributor

@osagie98 osagie98 commented Nov 4, 2025

Cherry-picks the C25 PR that implemented a Linux libiamf-based IAMF decoder. Minor changes were made to correct build errors/bring the code to more modern standards. The original commit message follows:

Implements a libiamf-based audio decoder.

Decoder specific code is guarded behind a GN flag enable_iamf_decode, which controls the ENABLE_IAMF_DECODE macro. enable_iamf_decode is off by default, so the new code doesn't build.

b/341792042

Bug: 376312767

Implements a libiamf-based audio decoder.

Decoder specific code is guarded behind a GN flag `enable_iamf_decode,`
which controls the ENABLE_IAMF_DECODE macro. `enable_iamf_decode` is off
by default, so the new code doesn't build.

b/341792042
@osagie98 osagie98 requested a review from a team as a code owner November 4, 2025 00:16
@kjyoun
Copy link
Contributor

kjyoun commented Nov 4, 2025

/gemini reviw

@kjyoun
Copy link
Contributor

kjyoun commented Nov 4, 2025

/gemini review

@gemini-code-assist
Copy link
Contributor

It looks like you're trying to request a code review. To initiate a new code review, please create a new top-level comment on the pull request with /gemini review. I cannot execute commands directly from a reply within a conversation thread. Also, please note the typo in your first attempt (/gemini reviw).

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a libiamf-based IAMF audio decoder for Linux, controlled by the enable_iamf_decode build flag. The changes are comprehensive, covering the decoder implementation, integration into the media pipeline, build system updates, and support checks. However, my review identified several critical issues that will prevent the code from compiling, including commented-out code in header files and incorrect use of const qualifiers. I have also pointed out a high-severity issue related to potential integer overflow and medium-severity issues regarding code style and performance. Addressing these points will be necessary to ensure the new decoder is correct and robust.

#include "starboard/shared/starboard/player/decoded_audio_internal.h"
#include "starboard/shared/starboard/player/filter/audio_decoder_internal.h"
#include "starboard/shared/starboard/player/job_queue.h"
// #include "third_party/libiamf/source/code/include/IAMF_decoder.h"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

This include is necessary for the declaration of IAMF_DecoderHandle and the IAMF decoder functions used in the implementation. It appears to have been commented out by mistake, which will cause a compilation failure.

Suggested change
// #include "third_party/libiamf/source/code/include/IAMF_decoder.h"
#include "third_party/libiamf/source/code/include/IAMF_decoder.h"

Comment on lines +51 to +53
bool ConfigureDecoder(const IamfBufferInfo* info,
int64_t timestamp,
std::string* error_message) const;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

The ConfigureDecoder method modifies the state of the IAMF decoder via IAMF_decoder_set_* functions. Therefore, it should not be declared as a const method, as this will result in a compilation error.

Suggested change
bool ConfigureDecoder(const IamfBufferInfo* info,
int64_t timestamp,
std::string* error_message) const;
bool ConfigureDecoder(const IamfBufferInfo* info,
int64_t timestamp,
std::string* error_message);

OutputCB output_cb_;
ErrorCB error_cb_;

// IAMF_DecoderHandle decoder_ = nullptr;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

The decoder_ member variable is used in the implementation file but is commented out in the header. This will cause a compilation error. Please uncomment it.

Suggested change
// IAMF_DecoderHandle decoder_ = nullptr;
IAMF_DecoderHandle decoder_ = nullptr;

std::vector<uint8_t> config_obus;
size_t config_obus_size;
std::vector<uint8_t> data;
const scoped_refptr<InputBuffer> input_buffer;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

The input_buffer member is declared as const but is not initialized in a constructor, and the struct does not have one. This will cause a compilation error. To fix this, you can remove the const qualifier, which will allow it to be assigned a value after the IamfBufferInfo object is created.

Suggested change
const scoped_refptr<InputBuffer> input_buffer;
scoped_refptr<InputBuffer> input_buffer;

return false;
}

int next_obu_pos = reader->pos() + obu_size;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The variable next_obu_pos is calculated from reader->pos() which returns a size_t and obu_size which is a uint32_t. Storing the result in a signed int can lead to data loss, truncation, or signed overflow issues on 64-bit platforms or with large buffers. It should be declared as size_t to match the type of reader->pos() and to correctly handle large buffer sizes.

Suggested change
int next_obu_pos = reader->pos() + obu_size;
size_t next_obu_pos = reader->pos() + obu_size;

Comment on lines +138 to +139
IAMF_decoder_decode(decoder_, info.data.data(), info.data.size(), nullptr,
const_cast<uint8_t*>(decoded_audio->data()));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The const_cast here is unnecessary. decoded_audio is a non-const scoped_refptr<DecodedAudio>, so decoded_audio->data() resolves to the non-const overload uint8_t* DecodedAudio::data(). Using const_cast when not needed can be confusing and may hide actual const-correctness issues.

Suggested change
IAMF_decoder_decode(decoder_, info.data.data(), info.data.size(), nullptr,
const_cast<uint8_t*>(decoded_audio->data()));
IAMF_decoder_decode(decoder_, info.data.data(), info.data.size(), nullptr,
decoded_audio->data());

Comment on lines +627 to +629
info->config_obus.assign(reader.buf(), reader.buf() + info->config_obus_size);
info->data.assign(reader.buf() + info->config_obus_size,
reader.buf() + reader.size());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

These assign calls copy data from the input buffer into new vectors. This is inefficient as it involves memory allocation and copying. Consider modifying IamfBufferInfo to use a span-like type (e.g., base::span<const uint8_t>) instead of std::vector<uint8_t> for config_obus and data. This would allow you to create views into the original input_buffer's data without copying. The IamfBufferInfo::input_buffer member can be used to keep the underlying data alive.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a libiamf-based IAMF decoder for Linux. The changes look good overall, adding the necessary build configurations, decoder implementation, and support checks. However, I've found several critical issues that will prevent the code from compiling or cause runtime errors, including a potential security vulnerability due to an out-of-bounds read. Please see my detailed comments for suggestions on how to address these issues.

#include "starboard/shared/starboard/player/decoded_audio_internal.h"
#include "starboard/shared/starboard/player/filter/audio_decoder_internal.h"
#include "starboard/shared/starboard/player/job_queue.h"
// #include "third_party/libiamf/source/code/include/IAMF_decoder.h"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

The header IAMF_decoder.h that defines IAMF_DecoderHandle is commented out. This will cause compilation errors as the type IAMF_DecoderHandle (used on line 66) will be undefined. This line should be uncommented.

Suggested change
// #include "third_party/libiamf/source/code/include/IAMF_decoder.h"
#include "third_party/libiamf/source/code/include/IAMF_decoder.h"

OutputCB output_cb_;
ErrorCB error_cb_;

// IAMF_DecoderHandle decoder_ = nullptr;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

The decoder handle member decoder_ is commented out, which will lead to compilation errors in iamf_audio_decoder.cc where it is frequently used. This should be uncommented to fix the build.

Suggested change
// IAMF_DecoderHandle decoder_ = nullptr;
IAMF_DecoderHandle decoder_ = nullptr;

Comment on lines +186 to +200
str->clear();
str->resize(max_bytes_to_read);

int bytes_read = ::starboard::strlcpy(
str->data(), reinterpret_cast<const char*>(buf), max_bytes_to_read);
if (bytes_read == max_bytes_to_read) {
// Ensure that the string is null terminated.
if (buf[bytes_read] != '\0') {
return -1;
}
}
str->resize(bytes_read);

// Account for null terminator.
return ++bytes_read;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

The current implementation of ReadStringInternal has a potential out-of-bounds read. If the source string has a length equal to max_bytes_to_read, the check buf[bytes_read] accesses memory outside the intended bounds, which is a security risk. A safer and cleaner implementation would be to first locate the null terminator within the given buffer limit before copying the string.

    const void* null_terminator = memchr(buf, '\0', max_bytes_to_read);
    if (!null_terminator) {
      // Not null-terminated within max_bytes_to_read.
      return -1;
    }
    int len = static_cast<const uint8_t*>(null_terminator) - buf;
    str->assign(reinterpret_cast<const char*>(buf), len);
    return len + 1;


int samples_decoded =
IAMF_decoder_decode(decoder_, info.data.data(), info.data.size(), nullptr,
const_cast<uint8_t*>(decoded_audio->data()));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Using const_cast to get a writable buffer from decoded_audio->data() is unsafe and not best practice. The DecodedAudio class provides a writable_data() method for this purpose, which should be used instead to ensure type safety and clarity.

Suggested change
const_cast<uint8_t*>(decoded_audio->data()));
decoded_audio->writable_data());

Comment on lines +573 to +574
std::optional<uint32_t> binaural_audio_element_id;
std::optional<uint32_t> surround_audio_element_id;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The variables binaural_audio_element_id and surround_audio_element_id are local to ParseDescriptorOBU. Since ParseDescriptorOBU is called in a loop within ParseInputBuffer, any parsed audio element information is lost between calls. This can lead to incorrect mix presentation selection if the AudioElement OBU and MixPresentation OBU are not parsed in the same call.

To fix this, these variables should be declared in ParseInputBuffer and passed as pointers to ParseDescriptorOBU to preserve their state across multiple OBU parsings within a single input buffer.

std::vector<uint8_t> config_obus;
size_t config_obus_size;
std::vector<uint8_t> data;
const scoped_refptr<InputBuffer> input_buffer;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The member input_buffer is a const scoped_refptr and is not initialized, which will cause a compilation error. It also appears to be unused. This line should be removed.

Copy link
Contributor

@kjyoun kjyoun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and you may need fix the build failure.

2025-11-04T00:28:04.4024028Z    54 | int GetMaxChannelCount() {
2025-11-04T00:28:04.4024429Z       |     ^~~~~~~~~~~~~~~~~~
2025-11-04T00:28:04.4024778Z 1 error generated.

#if ENABLE_IAMF_DECODE
if (audio_stream_info.codec == kSbMediaAudioCodecIamf) {
// IAMF mixes audio signals to the highest available speaker layout.
// TODO: Set the number of channels below Starboard.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: bug number?

@@ -0,0 +1,327 @@
// Copyright 2024 The Cobalt Authors. All Rights Reserved.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: 2025?


void IamfAudioDecoder::Initialize(const OutputCB& output_cb,
const ErrorCB& error_cb) {
SB_DCHECK(BelongsToCurrentThread());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SB_CHECK on new code

decoder_is_configured_ = true;
}

scoped_refptr<DecodedAudio> decoded_audio = new DecodedAudio(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can't we use unique_ptr instead?


decoded_audio

Reference-counted objects make it difficult to understand ownership and destruction order, especially when multiple threads are involved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants