Skip to content

emu with a q stuff#1266

Open
mantrakp04 wants to merge 32 commits intodevfrom
emu-with-a-q
Open

emu with a q stuff#1266
mantrakp04 wants to merge 32 commits intodevfrom
emu-with-a-q

Conversation

@mantrakp04
Copy link
Copy Markdown
Collaborator

@mantrakp04 mantrakp04 commented Mar 18, 2026

commit 5d43722575b826a8ed8dbb6b828f48eae4bca02c
Author: mantrakp04 mantrakp@gmail.com
Date: Wed Mar 18 12:27:01 2026 -0700

Add QEMU emulator snapshot functionality and reset command

- Introduced a new `emulator-qemu:reset` command in package.json to clear snapshots and force a fresh boot of the emulator.
- Enhanced the `run-emulator.sh` script to support saving and restoring snapshots, significantly reducing restart time from ~62s to ~4s.
- Implemented logic to check for existing snapshots and restore them during startup, improving the emulator's efficiency.
- Updated documentation in CLAUDE-KNOWLEDGE.md to explain the new snapshot restore process and its benefits.

These changes enhance the QEMU emulator's performance and usability for developers, providing a more efficient workflow during development.

commit 3877445bdd83cb8690da18c8520bf260d2795172
Author: mantrakp04 mantrakp@gmail.com
Date: Wed Mar 18 11:55:18 2026 -0700

Enhance QEMU emulator performance and configuration management

- Added optimizations to the QEMU emulator's app container startup process, reducing startup time from ~92s to ~62s by using qcow2 backing files and setting the working directory to /app.
- Updated the build-image.sh script to conditionally wait for background processes, improving robustness.
- Modified the run-emulator.sh script to create the disk image using qcow2 format instead of copying, enhancing efficiency.
- Adjusted the cloud-init user-data to set STACK_RUNTIME_WORK_DIR to /app, streamlining file operations during container initialization.
- Improved the entrypoint script to avoid unnecessary file copying when the working directory is set to /app.

These changes significantly enhance the performance and usability of the QEMU emulator for developers.

commit e0b86d3f1d5c08e46d0d343bc632e2a8c5777845
Author: mantrakp04 mantrakp@gmail.com
Date: Wed Mar 18 11:07:55 2026 -0700

Refactor local emulator configuration management and enhance Docker setup

- Removed redundant comments and improved code clarity in the local emulator's route handling.
- Streamlined the Dockerfile and docker-compose.yaml for better readability and maintenance.
- Updated entrypoint and initialization scripts to enhance service startup processes.
- Introduced a new common script for QEMU emulator to centralize architecture detection and firmware handling.
- Enhanced error handling in the host file bridge for improved robustness.
- Removed obsolete country code utilities to clean up the codebase.

These changes significantly improve the local emulator's configuration management and overall setup experience for developers.

commit 4fb0f93c6cc4f749a14acf0228c261e180875609
Author: mantrakp04 mantrakp@gmail.com
Date: Wed Mar 18 10:24:53 2026 -0700

Implement local emulator file bridge for enhanced configuration management

- Introduced a new host file bridge to facilitate reading and writing configuration files between the local emulator and the host system.
- Refactored the local-emulator module to utilize the file bridge for file operations, improving error handling and response validation.
- Added tests to ensure the file bridge functionality works as expected, including handling of non-existent files and writing configurations.
- Updated the run-emulator script to start the file bridge automatically, ensuring seamless integration during emulator startup.
- Enhanced documentation to reflect the new file bridge capabilities and usage instructions.

These changes significantly improve the local emulator's ability to manage configuration files, enhancing the development experience.

commit 3d18a7ce5bbf00a62a40a3f48f27856e79ecc62f
Author: mantrakp04 mantrakp@gmail.com
Date: Tue Mar 17 22:36:46 2026 -0700

Refactor QEMU local emulator setup and enhance app bundle handling

- Introduced a new script for packaging Docker images into a compressed app bundle, improving the emulator's deployment process.
- Updated build-image.sh to create a runtime configuration ISO, ensuring better management of environment settings.
- Enhanced cloud-init user-data scripts for both dev-server and deps guests, streamlining service setup and configuration.
- Improved the run-emulator.sh script to facilitate better handling of runtime configurations and dependencies.
- Adjusted the .gitignore to include .DS_Store and removed obsolete entries, cleaning up the repository.

These changes significantly enhance the local emulator's functionality and reliability for developers.

commit 8a35fb1ce79898d73e2259e256c11b6fd9b0a584
Author: mantrakp04 mantrakp@gmail.com
Date: Tue Mar 17 21:52:24 2026 -0700

Enhance local emulator functionality and configuration

- Updated package.json to improve the start-emulator command, providing clearer dashboard and backend URLs.
- Added a new wait-until-emulator-is-ready command to ensure the emulator is fully operational before proceeding.
- Refactored the local-emulator project route to streamline file existence checks and default config creation.
- Enhanced user guidance in the dashboard for local Stack config file handling.
- Updated tests to reflect changes in config file handling, ensuring non-existent files are created with default settings.
- Improved Docker configurations for the local emulator, including new environment variables and service dependencies.

These changes significantly enhance the local development experience and emulator reliability.

commit 3910ed4bc40bbb37340c1c316c24c2826ba372bd
Author: mantrakp04 mantrakp@gmail.com
Date: Tue Mar 17 19:59:36 2026 -0700

Remove unused stash-0.patch file to clean up the repository.

commit 74146d974458037a7a9590120a524629a1a6a162
Author: mantrakp04 mantrakp@gmail.com
Date: Tue Mar 17 19:58:46 2026 -0700

Enhance QEMU local emulator with app bundle support and runtime configuration

- Introduced a new script to package the backend and dashboard assets into a standalone app bundle for the QEMU emulator.
- Updated the build-image.sh script to create an ISO containing the app bundle, ensuring the guest image includes the full runtime.
- Modified cloud-init user-data to handle the new app bundle and runtime configuration, improving the setup process for local development.
- Enhanced the run-emulator.sh script to prepare and mount the runtime configuration ISO, facilitating better environment management for the emulator.
- Updated the user-data to include necessary environment variables for the stack application, ensuring seamless integration during startup.

These changes significantly improve the local emulator's functionality and ease of use for developers.

commit 9e865a1cf524398bc58f00e0836278775c4ae936
Author: mantrakp04 mantrakp@gmail.com
Date: Tue Mar 17 16:50:45 2026 -0700

Enhance local emulator setup with new services and configurations

- Added Docker support for a local emulator, integrating PostgreSQL, Redis, Inbucket, Svix, ClickHouse, MinIO, and QStash.
- Introduced new scripts for managing the emulator lifecycle, including build and run commands.
- Implemented cloud-init provisioning for automatic service setup on first boot.
- Updated package.json with new commands for emulator management and added dotenv-cli for environment variable management.
- Added tests for OAuth authorization flow to return JSON responses.
- Included configuration files for ClickHouse and user management.

This commit significantly improves the local development experience by providing a comprehensive emulator environment.

Summary by CodeRabbit

  • New Features

    • Added comprehensive local emulator support with Docker and QEMU options
    • Added emulator CLI commands: pull, run, stop, reset, status, build, and list-releases
    • Added emulator status monitoring page in demo application
    • Auto-generation of default configuration files for emulator projects instead of errors
  • Bug Fixes

    • Fixed missing config file handling to gracefully create defaults
  • Tests

    • Added emulator configuration and CLI tests
    • Added end-to-end emulator endpoint coverage
  • Chores

    • Added emulator management npm scripts and Docker compose setup

commit 5d43722575b826a8ed8dbb6b828f48eae4bca02c
Author: mantrakp04 <mantrakp@gmail.com>
Date:   Wed Mar 18 12:27:01 2026 -0700

    Add QEMU emulator snapshot functionality and reset command

    - Introduced a new `emulator-qemu:reset` command in package.json to clear snapshots and force a fresh boot of the emulator.
    - Enhanced the `run-emulator.sh` script to support saving and restoring snapshots, significantly reducing restart time from ~62s to ~4s.
    - Implemented logic to check for existing snapshots and restore them during startup, improving the emulator's efficiency.
    - Updated documentation in CLAUDE-KNOWLEDGE.md to explain the new snapshot restore process and its benefits.

    These changes enhance the QEMU emulator's performance and usability for developers, providing a more efficient workflow during development.

commit 3877445bdd83cb8690da18c8520bf260d2795172
Author: mantrakp04 <mantrakp@gmail.com>
Date:   Wed Mar 18 11:55:18 2026 -0700

    Enhance QEMU emulator performance and configuration management

    - Added optimizations to the QEMU emulator's app container startup process, reducing startup time from ~92s to ~62s by using qcow2 backing files and setting the working directory to /app.
    - Updated the build-image.sh script to conditionally wait for background processes, improving robustness.
    - Modified the run-emulator.sh script to create the disk image using qcow2 format instead of copying, enhancing efficiency.
    - Adjusted the cloud-init user-data to set STACK_RUNTIME_WORK_DIR to /app, streamlining file operations during container initialization.
    - Improved the entrypoint script to avoid unnecessary file copying when the working directory is set to /app.

    These changes significantly enhance the performance and usability of the QEMU emulator for developers.

commit e0b86d3f1d5c08e46d0d343bc632e2a8c5777845
Author: mantrakp04 <mantrakp@gmail.com>
Date:   Wed Mar 18 11:07:55 2026 -0700

    Refactor local emulator configuration management and enhance Docker setup

    - Removed redundant comments and improved code clarity in the local emulator's route handling.
    - Streamlined the Dockerfile and docker-compose.yaml for better readability and maintenance.
    - Updated entrypoint and initialization scripts to enhance service startup processes.
    - Introduced a new common script for QEMU emulator to centralize architecture detection and firmware handling.
    - Enhanced error handling in the host file bridge for improved robustness.
    - Removed obsolete country code utilities to clean up the codebase.

    These changes significantly improve the local emulator's configuration management and overall setup experience for developers.

commit 4fb0f93c6cc4f749a14acf0228c261e180875609
Author: mantrakp04 <mantrakp@gmail.com>
Date:   Wed Mar 18 10:24:53 2026 -0700

    Implement local emulator file bridge for enhanced configuration management

    - Introduced a new host file bridge to facilitate reading and writing configuration files between the local emulator and the host system.
    - Refactored the local-emulator module to utilize the file bridge for file operations, improving error handling and response validation.
    - Added tests to ensure the file bridge functionality works as expected, including handling of non-existent files and writing configurations.
    - Updated the run-emulator script to start the file bridge automatically, ensuring seamless integration during emulator startup.
    - Enhanced documentation to reflect the new file bridge capabilities and usage instructions.

    These changes significantly improve the local emulator's ability to manage configuration files, enhancing the development experience.

commit 3d18a7ce5bbf00a62a40a3f48f27856e79ecc62f
Author: mantrakp04 <mantrakp@gmail.com>
Date:   Tue Mar 17 22:36:46 2026 -0700

    Refactor QEMU local emulator setup and enhance app bundle handling

    - Introduced a new script for packaging Docker images into a compressed app bundle, improving the emulator's deployment process.
    - Updated build-image.sh to create a runtime configuration ISO, ensuring better management of environment settings.
    - Enhanced cloud-init user-data scripts for both dev-server and deps guests, streamlining service setup and configuration.
    - Improved the run-emulator.sh script to facilitate better handling of runtime configurations and dependencies.
    - Adjusted the .gitignore to include .DS_Store and removed obsolete entries, cleaning up the repository.

    These changes significantly enhance the local emulator's functionality and reliability for developers.

commit 8a35fb1ce79898d73e2259e256c11b6fd9b0a584
Author: mantrakp04 <mantrakp@gmail.com>
Date:   Tue Mar 17 21:52:24 2026 -0700

    Enhance local emulator functionality and configuration

    - Updated package.json to improve the start-emulator command, providing clearer dashboard and backend URLs.
    - Added a new wait-until-emulator-is-ready command to ensure the emulator is fully operational before proceeding.
    - Refactored the local-emulator project route to streamline file existence checks and default config creation.
    - Enhanced user guidance in the dashboard for local Stack config file handling.
    - Updated tests to reflect changes in config file handling, ensuring non-existent files are created with default settings.
    - Improved Docker configurations for the local emulator, including new environment variables and service dependencies.

    These changes significantly enhance the local development experience and emulator reliability.

commit 3910ed4bc40bbb37340c1c316c24c2826ba372bd
Author: mantrakp04 <mantrakp@gmail.com>
Date:   Tue Mar 17 19:59:36 2026 -0700

    Remove unused stash-0.patch file to clean up the repository.

commit 74146d974458037a7a9590120a524629a1a6a162
Author: mantrakp04 <mantrakp@gmail.com>
Date:   Tue Mar 17 19:58:46 2026 -0700

    Enhance QEMU local emulator with app bundle support and runtime configuration

    - Introduced a new script to package the backend and dashboard assets into a standalone app bundle for the QEMU emulator.
    - Updated the build-image.sh script to create an ISO containing the app bundle, ensuring the guest image includes the full runtime.
    - Modified cloud-init user-data to handle the new app bundle and runtime configuration, improving the setup process for local development.
    - Enhanced the run-emulator.sh script to prepare and mount the runtime configuration ISO, facilitating better environment management for the emulator.
    - Updated the user-data to include necessary environment variables for the stack application, ensuring seamless integration during startup.

    These changes significantly improve the local emulator's functionality and ease of use for developers.

commit 9e865a1cf524398bc58f00e0836278775c4ae936
Author: mantrakp04 <mantrakp@gmail.com>
Date:   Tue Mar 17 16:50:45 2026 -0700

    Enhance local emulator setup with new services and configurations

    - Added Docker support for a local emulator, integrating PostgreSQL, Redis, Inbucket, Svix, ClickHouse, MinIO, and QStash.
    - Introduced new scripts for managing the emulator lifecycle, including build and run commands.
    - Implemented cloud-init provisioning for automatic service setup on first boot.
    - Updated package.json with new commands for emulator management and added dotenv-cli for environment variable management.
    - Added tests for OAuth authorization flow to return JSON responses.
    - Included configuration files for ClickHouse and user management.

    This commit significantly improves the local development experience by providing a comprehensive emulator environment.
@vercel
Copy link
Copy Markdown

vercel bot commented Mar 18, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
stack-auth-hosted-components Ready Ready Preview, Comment Mar 27, 2026 9:41pm
stack-backend Ready Ready Preview, Comment Mar 27, 2026 9:41pm
stack-dashboard Ready Ready Preview, Comment Mar 27, 2026 9:41pm
stack-demo Ready Ready Preview, Comment Mar 27, 2026 9:41pm
stack-docs Ready Ready Preview, Comment Mar 27, 2026 9:41pm

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Mar 18, 2026

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

This PR introduces a comprehensive local emulator feature enabling developers to run Stack Auth locally via Docker and QEMU VMs. It includes GitHub Actions workflows for building/publishing QEMU images, backend file-based configuration handling, Docker containerization with orchestrated services, CLI commands for emulator management, and end-to-end tests validating the implementation.

Changes

Cohort / File(s) Summary
GitHub Actions Workflow
.github/workflows/qemu-emulator-build.yaml
New workflow building QEMU emulator VM images for arm64/amd64, uploading as artifacts, and publishing to GitHub Releases with "latest" prerelease management.
Backend Local Emulator Core
apps/backend/src/lib/local-emulator.ts, apps/backend/src/lib/local-emulator.test.ts
Added host-mount path resolution, environment-based config override, ENOENT/empty-file handling, and module validation; new test suite covers all control flow paths including missing files, env-based content, and host-mount scenarios.
Backend Configuration Logic
apps/backend/src/lib/config.tsx
Modified setBranchConfigOverride to write local-emulator branch configs to files before attempting DB upsert; removed early return gate and added Vitest tests asserting file writes and rejection on write failures.
Backend API & Database
apps/backend/src/app/api/latest/internal/local-emulator/project/route.tsx, apps/backend/prisma/seed.ts
Removed explicit filesystem validation in route handler; delegated missing-file handling to helper functions. Added team_admin permission grant to local emulator admin user during seed.
Dashboard UI
apps/dashboard/src/app/(main)/(protected)/(outside-dashboard)/projects/page-client.tsx
Updated guidance text to indicate auto-generation of config with defaults and renamed button from "Open project" to "Open or create project".
CLI Emulator Command
packages/stack-cli/src/commands/emulator.ts, packages/stack-cli/src/index.ts
New registerEmulatorCommand exports pull/run/stop/reset/build/list-releases subcommands for managing QEMU emulator VMs; integrated into CLI program initialization.
Docker Build & Compose
docker/local-emulator/Dockerfile, docker/local-emulator/docker-compose.yaml, docker/local-emulator/base.env, docker/local-emulator/*.conf
Multi-stage Dockerfile bundles Node.js backend/dashboard, PostgreSQL, Redis, ClickHouse, MinIO, Inbucket, Svix, and QStash. Docker Compose exposes services via port-prefix convention, persists data via named volumes, includes healthchecks for PostgreSQL and HTTP endpoints. Configuration files provide ClickHouse and environment variable setup.
QEMU VM Provisioning
docker/local-emulator/qemu/build-image.sh, docker/local-emulator/qemu/run-emulator.sh, docker/local-emulator/qemu/common.sh, docker/local-emulator/qemu/cloud-init/*, docker/local-emulator/qemu/.gitignore, docker/local-emulator/qemu/images/.gitignore
build-image.sh downloads Debian QCOW2 base, builds emulator Docker image, creates cloud-init ISOs, launches QEMU with polling for provision timeout. run-emulator.sh manages VM lifecycle (start/stop/reset/status) with port validation, overlay QCOW2, runtime config ISO, port forwarding, and endpoint health polling. common.sh provides host detection, QEMU binary mapping, firmware lookup, and ISO creation helpers. Cloud-init files bootstrap the VM with Docker, systemd services, and helper scripts.
Initialization & Startup Scripts
docker/local-emulator/entrypoint.sh, docker/local-emulator/init-services.sh, docker/local-emulator/start-app.sh, docker/local-emulator/supervisord.conf
Entrypoint initializes PostgreSQL data directory and starts supervisord. Init-services waits for MinIO/ClickHouse readiness, configures buckets/databases. Start-app waits for dependencies and launch markers. Supervisord manages multiple daemons (PostgreSQL, Redis, Inbucket, ClickHouse, MinIO, Svix, QStash) and the main Stack app.
Demo/Example Pages
examples/demo/src/app/api/emulator-status/route.ts, examples/demo/src/app/emulator-status/page.tsx
New route handler queries emulator service health (HTTP/TCP) with latency measurement. New page fetches and displays service status with auto-refresh polling and manual controls.
End-to-End Tests
apps/e2e/tests/backend/endpoints/api/v1/internal/local-emulator-project.test.ts, apps/e2e/tests/general/cli.test.ts
Updated local-emulator project test to expect HTTP 200 (config creation) instead of 400 error. Added CLI test suite validating stack-cli emulator subcommands (pull/run/stop/reset/build/list-releases/help output).
Configuration & Utilities
package.json, docker/server/entrypoint.sh, examples/demo/src/components/header.tsx
Added npm scripts for emulator Docker Compose, QEMU build/run/manage/bench, readiness checks, and fml/dev:tui/kms convenience scripts. Updated turbo dependency. Modified server entrypoint to use configurable work directory and skip redundant copies. Removed inline comment from header placeholder.

Sequence Diagram(s)

sequenceDiagram
    actor User
    participant CLI as Stack CLI
    participant GitHub as GitHub Releases
    participant QEMUBuild as QEMU Build Script
    participant Docker as Docker/QEMU
    participant CloudInit as Cloud-Init
    participant Emulator as Emulator Container
    participant Dashboard as Dashboard

    User->>CLI: stack-cli emulator pull<br/>(arch, repo, branch, tag)
    CLI->>GitHub: gh release download<br/>stack-emulator-{arch}.qcow2
    GitHub-->>CLI: .qcow2 artifact
    CLI->>CLI: Save to qemu/images/

    User->>CLI: stack-cli emulator run
    CLI->>CLI: Locate qcow2<br/>(auto-pull if missing)
    CLI->>Docker: run-emulator.sh start<br/>(EMULATOR_ARCH)
    Docker->>Docker: Validate ports free<br/>Create runtime.env ISO<br/>Create overlay QCOW2
    Docker->>QEMU: Launch QEMU with:<br/>overlay disk, ISO,<br/>port forwarding
    QEMU->>CloudInit: Boot VM
    CloudInit->>CloudInit: Install Docker<br/>Load bundle.tar.gz<br/>Mount host FS
    CloudInit->>Emulator: docker run<br/>stack-local-emulator
    Emulator->>Emulator: Start PostgreSQL,<br/>Redis, ClickHouse,<br/>MinIO, services
    QEMU->>QEMU: Poll for<br/>STACK_CLOUD_INIT_DONE
    Docker->>Docker: Poll service ports<br/>HTTP endpoints
    Docker-->>CLI: Ready ✓
    CLI-->>User: Emulator running

    User->>Dashboard: Access http://127.0.0.1:8101
    Dashboard->>Emulator: API requests
    Emulator-->>Dashboard: Responses
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

  • PR #1233: Implements the same local-emulator feature with overlapping code modifications across backend, docker, CLI, and tests.
  • PR #1016: Modifies apps/backend/prisma/seed.ts and team-permission logic related to this PR's seed changes.

Suggested reviewers

  • N2D4

Poem

🐰 A QEMU machine spins with grace,
Cloud-init scripts prepare the space,
Docker services dance in a row,
CLI commands help us go,
Local emulator steals the show! 🚀

🚥 Pre-merge checks | ❌ 3

❌ Failed checks (1 warning, 2 inconclusive)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 2.86% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Title check ❓ Inconclusive The PR title 'emu with a q stuff' is vague and generic, using non-descriptive terms that fail to convey the actual scope of changes. Replace with a clear, specific title summarizing the main change, such as 'Add QEMU emulator with snapshot support and Docker integration' or 'Implement local QEMU emulator with Docker and configuration management'.
Description check ❓ Inconclusive The PR description contains commit history showing incremental development work, but lacks a proper structured description following the template with clear sections about what was changed and why. Add a structured summary section at the top explaining the main objectives, key changes (snapshots, Docker setup, file bridge), and benefits before or instead of relying solely on commit messages.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch emu-with-a-q

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

- Unified the QEMU emulator setup by consolidating the previously separate VM roles (deps and dev-server) into a single VM, simplifying the architecture and reducing complexity.
- Updated the `build-image.sh` script to adjust resource allocations and streamline image building processes.
- Enhanced the `prepare-image-bundle.sh` script to support multiple Docker images, improving the packaging of emulator components.
- Refactored cloud-init configurations to use a single definition for the emulator, improving maintainability and clarity.
- Updated documentation in CLAUDE-KNOWLEDGE.md to reflect changes in the emulator's architecture and cloud-init usage.

These changes significantly enhance the QEMU emulator's usability and performance for developers.
…file bridge

- Removed the `readConfigFileContentIfExists` and `writeConfigToFile` functions from the local emulator module, simplifying file handling.
- Updated the `readConfigFromFile` function to utilize environment variables for configuration content, enhancing flexibility.
- Refactored tests to align with the new configuration management approach, ensuring accurate validation of file reading and handling.
- Consolidated Docker environment variables into a new `base.env` file for improved organization and maintainability.
- Removed the host file bridge script and related logic, streamlining the emulator's architecture and reducing complexity.

These changes significantly enhance the local emulator's usability and maintainability for developers.
- Updated the local emulator Dockerfile to include a multi-stage build process, improving dependency management and reducing image size.
- Added Node.js build stages and integrated pnpm for package management, enhancing build efficiency.
- Implemented a migration-pruner stage to remove unnecessary node_modules, optimizing runtime performance.
- Refined the server Dockerfile to utilize pruned node_modules for runtime, ensuring a leaner image.
- Enhanced entrypoint and initialization scripts for better service startup processes.
- Updated exposed ports to include backend and dashboard services, improving accessibility.

These changes significantly improve the Docker setup for both local emulator and server, enhancing performance and maintainability.
- Updated the `build-image.sh` script to simplify the Docker image list, focusing on the main emulator image.
- Enhanced the image compression process by adding progress logging during the conversion to qcow2 format.
- Improved the `run-emulator.sh` script to provide real-time feedback while saving snapshots, enhancing user experience.

These changes streamline the build process and improve the usability of the emulator's snapshot functionality.
@mantrakp04 mantrakp04 changed the title Squashed commit of the following: emu with a q stuff Mar 19, 2026
- Renamed and restructured the Docker Compose configuration to unify the local emulator services into a single `stack-local` service, simplifying the architecture.
- Updated the Dockerfile to include a multi-stage build process for the local emulator, integrating the backend and dashboard into a single image.
- Introduced a new `start-app.sh` script to manage service initialization and health checks, ensuring all dependencies are ready before starting the application.
- Enhanced the cloud-init user-data script to streamline the setup process and improve service management.
- Removed obsolete service definitions and scripts, reducing complexity and improving maintainability.

These changes significantly enhance the local emulator's usability and performance for developers.
- Introduced a new function to generate a fingerprint of the base QEMU image, allowing for detection of changes since the last snapshot.
- Updated snapshot handling logic to discard stale snapshots if the base image has changed, improving reliability and efficiency.
- Enhanced the `run-emulator.sh` script to save the fingerprint after a snapshot is created, ensuring accurate tracking of image state.

These changes enhance the snapshot management process in the QEMU emulator, providing better consistency and performance for developers.
- Updated `stack.config.ts` to enable various application features including authentication, emails, teams, RBAC, API keys, webhooks, and launch checklist.
- Modified `seed.ts` to grant team permissions for the local emulator admin user, improving access control.
- Adjusted `run-emulator.sh` to allow auto-detection of the configuration file, enhancing usability.
- Updated `.gitignore` to include the new `run/` directory, ensuring cleaner repository management.

These changes improve the local emulator's configuration management and user experience for developers.
- Updated `stack.config.ts` to enable password sign-in for authentication.
- Refactored `config.tsx` to include functionality for writing configuration changes back to the local emulator's config file.
- Introduced `writeConfigToFile` and `getLocalEmulatorFilePath` functions in `local-emulator.ts` to improve file handling and path resolution.
- Enhanced `run-emulator.sh` to support mounting the host filesystem, facilitating better integration with the local environment.

These changes improve the local emulator's configuration management and usability for developers.
mantrakp04 and others added 3 commits March 19, 2026 11:29
- Introduced a new GitHub Actions workflow for building and publishing QEMU emulator images, triggered on pushes to main and dev branches, as well as pull requests.
- Added a new `emulator` command in the Stack CLI to manage QEMU emulator operations, including pulling images, starting, stopping, and checking status.
- Implemented functionality for downloading emulator images from GitHub Releases and managing snapshots, enhancing the local development experience.

These changes significantly improve the automation and usability of the QEMU emulator for developers.
- Modified `stack.config.ts` to adjust authentication settings, enabling OTP and passkey sign-ins while disabling password sign-in.
- Updated the GitHub Actions workflow for the QEMU emulator to improve the start and stop commands, enhancing clarity in the process.
- Refactored the CLI commands for the emulator, removing snapshot options and streamlining the pull command for image downloads.
- Added new tests for the emulator CLI to validate help and argument parsing, ensuring a better developer experience.

These changes improve the configuration management and usability of the QEMU emulator for developers.
…mands

- Deleted `stack.config.ts` and `CLAUDE-KNOWLEDGE.md` as they are no longer needed.
- Refactored the `local-emulator.ts` to improve error handling and type definitions.
- Updated the `header.tsx` component to remove the Emulator Status link, simplifying the navigation.
- Enhanced the `emulator.ts` CLI commands for better readability and maintainability.

These changes improve the overall structure and usability of the emulator and related components.
…tus reporting

- Modified `build-image.sh` and `run-emulator.sh` to check for write permissions on `/dev/kvm` instead of existence, enhancing compatibility with various environments.
- Added status tracking for services in `run-emulator.sh`, allowing for better error handling and reporting during emulator status checks.

These changes enhance the reliability and usability of the QEMU emulator setup.
- Updated `local-emulator.ts` to explicitly cast the module evaluation result to `Record<string, unknown>`, improving type safety.
- Enhanced `emulator.ts` architecture detection with clearer case structures for better readability and maintainability.

These changes improve code clarity and type management in the local emulator setup.
- Added functionality to write an empty configuration file if none exists when processing a project in `route.tsx`.
- Updated `readConfigFromFile` in `local-emulator.ts` to support reading configuration from an environment variable, improving flexibility in configuration management.
- Ensured the directory structure is created when writing configuration files, enhancing file handling robustness.

These changes improve the local emulator's configuration management and usability for developers.
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ Duplicate comments (1)
.github/workflows/qemu-emulator-build.yaml (1)

116-157: ⚠️ Potential issue | 🟡 Minor

Release creation is not idempotent for re-runs.

The tag format emulator-${BRANCH}-${DATE}-${SHORT_SHA} is deterministic for a given commit within a single day. Re-running the workflow (e.g., after a transient failure) for the same commit will cause gh release create to fail with HTTP 422 ("already_exists").

Consider adding ${{ github.run_id }} or ${{ github.run_attempt }} to make the tag unique per run, or use create-or-edit semantics similar to the "latest" tag handling below.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/qemu-emulator-build.yaml around lines 116 - 157, The
release tag generation (RELEASE_TAG) is deterministic and can clash on workflow
re-runs; update the TAG computation used for env.RELEASE_TAG (and the value
echoed into GITHUB_ENV) to include a run-unique token such as `${{ github.run_id
}}` or `${{ github.run_attempt }}` (e.g.,
TAG="emulator-${BRANCH}-${DATE}-${SHORT_SHA}-${{ github.run_id }}"), or
alternatively change the gh invocation around the env.RELEASE_TAG to perform
create-or-edit semantics (use `gh release view "${{ env.RELEASE_TAG }}"
>/dev/null 2>&1 && gh release edit ... || gh release create ...`) so `gh release
create` won’t fail with already_exists; ensure you update the places that set
and reference RELEASE_TAG and the `gh release create` command accordingly.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Duplicate comments:
In @.github/workflows/qemu-emulator-build.yaml:
- Around line 116-157: The release tag generation (RELEASE_TAG) is deterministic
and can clash on workflow re-runs; update the TAG computation used for
env.RELEASE_TAG (and the value echoed into GITHUB_ENV) to include a run-unique
token such as `${{ github.run_id }}` or `${{ github.run_attempt }}` (e.g.,
TAG="emulator-${BRANCH}-${DATE}-${SHORT_SHA}-${{ github.run_id }}"), or
alternatively change the gh invocation around the env.RELEASE_TAG to perform
create-or-edit semantics (use `gh release view "${{ env.RELEASE_TAG }}"
>/dev/null 2>&1 && gh release edit ... || gh release create ...`) so `gh release
create` won’t fail with already_exists; ensure you update the places that set
and reference RELEASE_TAG and the `gh release create` command accordingly.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: fbfd4ed5-e9b2-4815-ad16-f499bc23bb4e

📥 Commits

Reviewing files that changed from the base of the PR and between 42bc125 and 4be8a82.

📒 Files selected for processing (1)
  • .github/workflows/qemu-emulator-build.yaml

- Updated the GitHub Actions workflow to streamline the QEMU emulator image build process, including improved handling of existing releases and enhanced error reporting.
- Refactored local emulator configuration management to support reading from host-mounted paths, ensuring robust file handling and error reporting for configuration reads and writes.
- Added tests to validate new configuration handling features, improving reliability for developers working with local emulator setups.

These changes enhance the overall performance and usability of the QEMU emulator and local emulator configuration management.
- Updated the EMULATOR_READY_TIMEOUT variable from 600 to 3200 seconds in the GitHub Actions workflow for the QEMU emulator build process.
- This change aims to provide additional time for the emulator to become ready, enhancing reliability during the build and testing phases.

These adjustments improve the overall performance and stability of the QEMU emulator setup.
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

♻️ Duplicate comments (2)
docker/local-emulator/qemu/build-image.sh (2)

229-233: ⚠️ Potential issue | 🟡 Minor

Potential race condition reading pidfile immediately after daemonize.

After QEMU daemonizes, the pidfile may not be written yet when cat "$pidfile" executes. This could cause the script to fail or read an incomplete PID.

🛡️ Proposed fix: Add a brief wait for pidfile
     -daemonize \
     -pidfile "$pidfile"

+  # Wait briefly for QEMU to write the pidfile
+  local wait_count=0
+  while [ ! -s "$pidfile" ] && [ "$wait_count" -lt 20 ]; do
+    sleep 0.5
+    wait_count=$((wait_count + 1))
+  done
+  if [ ! -s "$pidfile" ]; then
+    err "QEMU failed to write pidfile"
+    exit 1
+  fi
+
   pid="$(cat "$pidfile")"
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@docker/local-emulator/qemu/build-image.sh` around lines 229 - 233, After
calling QEMU with the daemonize option the script reads pidfile immediately
which can race; modify the code around the daemonize invocation and the lines
that set pid="$(cat "$pidfile")" and elapsed=0 to wait for the pidfile to appear
and be non-empty with a short timeout: implement a loop checking [ -s "$pidfile"
] (use sleep 0.1 or similar between attempts), incrementing elapsed and failing
with a clear error if the timeout is reached, then read pid from the pidfile
once present; reference the existing pidfile variable, the pid assignment, and
elapsed control logic when locating where to insert the wait.

199-200: ⚠️ Potential issue | 🟠 Major

Don't swallow disk resize failures.

If qemu-img resize fails, the script continues with the original disk size, causing confusing failures during provisioning. Let the error propagate immediately.

💽 Proposed fix
   cp "$base_img" "$tmp_img"
-  qemu-img resize "$tmp_img" "$DISK_SIZE" >/dev/null 2>&1 || true
+  qemu-img resize "$tmp_img" "$DISK_SIZE" >/dev/null
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@docker/local-emulator/qemu/build-image.sh` around lines 199 - 200, The
qemu-img resize call currently swallows failures with "|| true" causing the
script to continue using the wrong disk size; remove the "|| true" (and keep any
intended output redirection) so that the qemu-img resize "$tmp_img" "$DISK_SIZE"
command can fail fast and propagate its non-zero exit (or alternatively enable
errexit for the script), referencing the qemu-img resize invocation and the TMP
image variable "$tmp_img"/"$DISK_SIZE" so the error surfaces immediately instead
of being ignored.
🧹 Nitpick comments (1)
docker/local-emulator/qemu/run-emulator.sh (1)

368-387: Consider using bash builtins for timing.

The cmd_bench function shells out to Python for timestamps. For a bash script, consider using date +%s.%N or the $SECONDS builtin to avoid the Python dependency.

♻️ Proposed simplification
 cmd_bench() {
-  local start_time end_time
+  local start_time end_time elapsed
   cmd_stop >/dev/null 2>&1 || true
-  start_time="$(python3 - <<'PY'
-import time
-print(time.time())
-PY
-)"
+  start_time="$(date +%s.%N)"
   cmd_start
-  end_time="$(python3 - <<'PY'
-import time
-print(time.time())
-PY
-)"
-  python3 - <<PY
-start_time = float("${start_time}")
-end_time = float("${end_time}")
-print(f"Startup time: {end_time - start_time:.1f}s")
-PY
+  end_time="$(date +%s.%N)"
+  elapsed="$(echo "$end_time - $start_time" | bc)"
+  printf "Startup time: %.1fs\n" "$elapsed"
 }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@docker/local-emulator/qemu/run-emulator.sh` around lines 368 - 387, The
cmd_bench function currently shells out to Python to capture timestamps; replace
the Python calls by using bash timing builtins to remove the Python
dependency—use either SECONDS (or $SECONDS + nanoseconds if sub-second precision
needed) or capture high-resolution timestamps with date +%s.%N into start_time
and end_time, then compute the delta in bash and print the result; update
references to start_time and end_time in cmd_bench and keep cmd_start/cmd_stop
invocation behavior unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@apps/backend/src/lib/local-emulator.ts`:
- Around line 98-103: The guard that extracts config after jiti.evalModule
currently calls throwErr for a missing export which produces a non-StatusError;
change that to throw a StatusError with StatusError.BadRequest (same as the
isValidConfig branch) so both missing-export and invalid-config paths return a
BadRequest; locate the block around jiti.evalModule, the variables mod and
config, and replace the throwErr(...) call with throwing new
StatusError(StatusError.BadRequest, `Invalid config in ${filePath}. The file
must export a 'config' object.`) to match the existing error handling.
- Around line 84-88: The code treats an empty
STACK_LOCAL_EMULATOR_CONFIG_CONTENT as unset because it uses a truthy check;
change the conditional that branches on envContent (around getEnvVariable,
resolvedPath, and Buffer.from(...)) to explicitly detect presence vs
absence—e.g., replace the `envContent ?` check with an explicit check like
`envContent !== ""` (or `envContent != null` if you change the getEnvVariable
default to undefined) so an explicit empty string is considered an override
rather than falling back to resolveConfigFilePath/readFile.

In `@packages/stack-cli/src/commands/emulator.ts`:
- Around line 143-150: The action handler currently uses opts.arch directly
causing invalid values to flow into pullImage; add validation for opts.arch
before calling detectArch()/pullImage by checking it against the EmulatorArch
set (create an isValidEmulatorArch(arch: string): arch is EmulatorArch helper or
similar) and if invalid, throw or call process.exit with a clear error message
like "Invalid --arch: <value>; expected one of: ...". Update the .action
callback to use validated opt (or fallback to detectArch()) and only call
pullImage(arch, ...) when validation passes.

---

Duplicate comments:
In `@docker/local-emulator/qemu/build-image.sh`:
- Around line 229-233: After calling QEMU with the daemonize option the script
reads pidfile immediately which can race; modify the code around the daemonize
invocation and the lines that set pid="$(cat "$pidfile")" and elapsed=0 to wait
for the pidfile to appear and be non-empty with a short timeout: implement a
loop checking [ -s "$pidfile" ] (use sleep 0.1 or similar between attempts),
incrementing elapsed and failing with a clear error if the timeout is reached,
then read pid from the pidfile once present; reference the existing pidfile
variable, the pid assignment, and elapsed control logic when locating where to
insert the wait.
- Around line 199-200: The qemu-img resize call currently swallows failures with
"|| true" causing the script to continue using the wrong disk size; remove the
"|| true" (and keep any intended output redirection) so that the qemu-img resize
"$tmp_img" "$DISK_SIZE" command can fail fast and propagate its non-zero exit
(or alternatively enable errexit for the script), referencing the qemu-img
resize invocation and the TMP image variable "$tmp_img"/"$DISK_SIZE" so the
error surfaces immediately instead of being ignored.

---

Nitpick comments:
In `@docker/local-emulator/qemu/run-emulator.sh`:
- Around line 368-387: The cmd_bench function currently shells out to Python to
capture timestamps; replace the Python calls by using bash timing builtins to
remove the Python dependency—use either SECONDS (or $SECONDS + nanoseconds if
sub-second precision needed) or capture high-resolution timestamps with date
+%s.%N into start_time and end_time, then compute the delta in bash and print
the result; update references to start_time and end_time in cmd_bench and keep
cmd_start/cmd_stop invocation behavior unchanged.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: f8e87e41-3a1f-4aec-beef-b85c78e321dc

📥 Commits

Reviewing files that changed from the base of the PR and between 4be8a82 and d5d2a09.

📒 Files selected for processing (7)
  • .github/workflows/qemu-emulator-build.yaml
  • apps/backend/src/lib/local-emulator.test.ts
  • apps/backend/src/lib/local-emulator.ts
  • docker/local-emulator/qemu/build-image.sh
  • docker/local-emulator/qemu/cloud-init/emulator/user-data
  • docker/local-emulator/qemu/run-emulator.sh
  • packages/stack-cli/src/commands/emulator.ts
✅ Files skipped from review due to trivial changes (1)
  • docker/local-emulator/qemu/cloud-init/emulator/user-data

…ation updates

- Introduced a new script to generate the local emulator environment file (`.env.development`) based on existing backend and dashboard configurations, ensuring consistency and ease of use.
- Updated `docker-compose.yaml` to reference the new environment file, improving configuration management for the local emulator.
- Modified emulator startup scripts to include environment generation, streamlining the process for developers.
- Added a new environment file for local development, containing necessary credentials and settings for the emulator.

These changes improve the local emulator's usability and configuration management, facilitating a smoother development experience.
- Improved the `readConfigFromFile` function in `local-emulator.ts` to provide clearer error handling for invalid configurations, ensuring robust validation of the exported config object.
- Added a new test in `cli.test.ts` to verify that the emulator CLI correctly rejects invalid architecture values, enhancing test coverage for the emulator command.
- Simplified the startup time logging in `run-emulator.sh` by removing unnecessary Python calls, improving script efficiency.

These changes enhance the reliability and usability of the local emulator and its CLI interface.
- Modified the local emulator scripts to replace references from `.env.development` to `base.env`, ensuring consistency in environment variable sourcing.
- Updated the QEMU runtime path in the documentation to clarify the cause of a regression in the local emulator's startup time.
- Enhanced the `.dockerignore` file to exclude new emulator-related files, improving build efficiency.

These changes streamline the local emulator setup and improve clarity in documentation regarding environment configurations.
Copy link
Copy Markdown
Contributor

@N2D4 N2D4 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is really cool, i love the github releases approach and how it's integrated into the CLI. left some comments

Copy link
Copy Markdown
Contributor

@N2D4 N2D4 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is really cool, i love the github releases approach and how it's integrated into the CLI. left some comments

- Introduced file existence checks in the local emulator project route to ensure the specified config file is present before proceeding with project creation.
- Updated the `setBranchConfigOverride` function to streamline writing configurations directly to the file when the local emulator is enabled, improving reliability.
- Refactored the `readConfigFromFile` function to handle empty file scenarios by writing a default config, enhancing user experience.
- Adjusted related tests to reflect changes in error handling and file creation logic, ensuring robust validation of local emulator functionality.

These updates improve the local emulator's configuration management and enhance the overall developer experience.
- Updated local emulator scripts to standardize command names and improve clarity, changing references from `generate-local-emulator-env` to `emulator:generate-env` and similar adjustments for other commands.
- Introduced a new configuration file (`stack.config.ts`) to centralize authentication and team settings, enhancing the management of user permissions and onboarding processes.
- Removed the outdated `.env.development` file to streamline environment handling, ensuring that the local emulator relies on generated configurations.
- Adjusted documentation and example usage to reflect the new command structure and configuration file, improving developer experience.

These changes enhance the usability and maintainability of the local emulator setup, providing a clearer structure for configuration and command usage.
- Updated the local emulator scripts to streamline command usage, replacing the previous docker-compose setup with a QEMU-based approach for improved performance and simplicity.
- Removed the outdated `docker-compose.yaml` file and adjusted related scripts to utilize QEMU for running the emulator, enhancing clarity and maintainability.
- Enhanced the GitHub Actions workflow to include a smoke test for the emulator, ensuring that services are healthy after startup.
- Updated environment variable handling in the emulator scripts to reflect new port configurations, improving service accessibility.
- Adjusted documentation to reflect the new emulator commands and usage patterns, providing clearer guidance for developers.

These changes improve the usability and reliability of the local emulator setup, facilitating a smoother development experience.
- Updated the GitHub Actions workflow to include specific paths for the QEMU emulator build, ensuring that changes in the local emulator directory trigger the workflow.
- Added a new README.md file for the QEMU local emulator, detailing its architecture, usage, and scripts for building and running the emulator, improving clarity for developers.
- Documented the build process and optimizations taken, providing comprehensive guidance for users on how to utilize the local emulator effectively.

These changes improve the usability and maintainability of the QEMU emulator setup, facilitating a smoother development experience.
@mantrakp04
Copy link
Copy Markdown
Collaborator Author

@greptile-ai review

- Replaced the 'run' command expectation with 'start' in the emulator help output.
- Removed the test for 'emulator build' help as it is no longer relevant.
- Updated error message for invalid architecture in the 'emulator pull' test to improve clarity.

These changes enhance the accuracy and relevance of the CLI tests for the emulator functionality.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants