Skip to content

feat: add realtime robot conversation tools - #579

Draft
meganetaaan wants to merge 1 commit into
developfrom
codex/realtime-conversation-tools
Draft

feat: add realtime robot conversation tools#579
meganetaaan wants to merge 1 commit into
developfrom
codex/realtime-conversation-tools

Conversation

@meganetaaan

Copy link
Copy Markdown
Collaborator

Summary

  • Add built-in robot tools for Realtime Conversation: set_emotion, play_motion, optional set_led, play_melody, and get_robot_status.
  • Reuse a single modern AudioOut through a four-track software mixer so conversation audio and melody/tone playback can coexist.
  • Suspend only microphone input during melody playback while preserving the Realtime connection and output path.

What Changed

  • Add recursive tool parameter schemas and safe argument validation.
  • Expose the latest successful IMU sample for robot status.
  • Migrate Speaker to the modern streaming AudioOut API and route supported platforms through the shared mixer.
  • Integrate the shared robot tools into the chat_audioio MOD.
  • Add unit, Moddable, and architecture coverage.
  • Release impact: minor
  • Changeset: .changeset/realtime-robot-tools.md

Verification

  • cd firmware && npm run format
  • cd firmware && npm run lint
  • cd firmware && npx biome ci . --error-on-warnings
  • cd firmware && npm run test:unit (48/48)
  • cd firmware && npm run test:moddable (33/33 manifests)
  • cd firmware && npm run check:architecture
  • cd firmware && npm run check:legacy-names
  • cd firmware && npm run check:manifest
  • cd firmware && npm run build:m5stackchan_cores3
  • cd firmware && npm run build:stackchan_rt
  • cd firmware && npm run mod:m5stackchan_cores3 -- mods/examples/chat_audioio/manifest.json -t build
  • cd firmware && npm run mod:stackchan_rt -- mods/examples/chat_audioio/manifest.json -t build
  • Hardware test (deferred because the connected device is currently in use by another task)

Affected Areas

  • firmware
  • web
  • schematics
  • case
  • docs
  • ci/github-actions

Breaking Changes

  • none
  • yes, described below

Related Issues

  • None

@coderabbitai

coderabbitai Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 39850403-fca4-41ad-920a-2f414ca21559

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/realtime-conversation-tools

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant