Skip to content

UN-2609 [FEAT] Added dumb-init wrapper to forward SIGTERM to Python processes in tool containers #1466

Merged
vishnuszipstack merged 9 commits intomainfrom
UN-2609-Graceful-shutdown-for-tool-script-entrypoint
Aug 5, 2025
Merged

UN-2609 [FEAT] Added dumb-init wrapper to forward SIGTERM to Python processes in tool containers #1466
vishnuszipstack merged 9 commits intomainfrom
UN-2609-Graceful-shutdown-for-tool-script-entrypoint

Conversation

@kirtimanmishrazipstack
Copy link
Contributor

@kirtimanmishrazipstack kirtimanmishrazipstack commented Jul 31, 2025

What

  • From runner, we tools are invoked using shell command, handled that invocation using dumb-init.

Why

  • Tools are not receiving SIGTERM in staging cluster without this dumb-init

How

Can this PR break any existing features. If yes, please list possible items. If no, please explain why. (PS: Admins do not merge the PR without this section filled)

  • N/A

Database Migrations

  • N/A

Env Config

  • N/A

Relevant Docs

Related Issues or PRs

Dependencies Versions

Notes on Testing

  • Created a docker container with and without dumb-init from shell
  • All test cases are on folder runner/tests/signal-handling-demo. I will remove this folder once PR is approved.

Screenshots

  • SIGTERM Without dumb-init
Screenshot from 2025-07-31 14-37-40 - SIGTERM with `dumb-init` Screenshot from 2025-07-31 14-38-11

Checklist

I have read and understood the Contribution Guidelines.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Jul 31, 2025

Summary by CodeRabbit

  • Chores
    • Added dumb-init to container images to improve process management.
    • Updated container startup to use dumb-init for better signal handling and process reaping.

Walkthrough

The changes introduce the dumb-init process supervisor into several Docker images by adding it to their installation steps. Additionally, the container startup command in the runner is updated to prepend dumb-init, ensuring it acts as the init system for spawned processes. No other logic or interface changes are present.

Changes

Cohort / File(s) Change Summary
Dockerfile: Add dumb-init dependency
tools/classifier/Dockerfile, tools/structure/Dockerfile, tools/text_extractor/Dockerfile
Updated the apt-get install commands to include the dumb-init package in each Dockerfile.
Runner: Use dumb-init in container command
runner/src/unstract/runner/runner.py
Modified the container startup command to prepend dumb-init before /bin/sh -c container_command.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant Runner
    participant Container

    User->>Runner: Request to run container with command
    Runner->>Container: Start with ["dumb-init", "/bin/sh", "-c", container_command]
    Container->>dumb-init: Launch process
    dumb-init->>/bin/sh: Execute container_command
Loading

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~7 minutes

Note

⚡️ Unit Test Generation is now available in beta!

Learn more here, or try it out under "Finishing Touches" below.


📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
Cache: Disabled due to Reviews > Disable Cache setting
Knowledge Base: Disabled due to Reviews > Disable Knowledge Base setting

📥 Commits

Reviewing files that changed from the base of the PR and between 49a4352 and 0c02584.

📒 Files selected for processing (2)
  • tools/classifier/Dockerfile (1 hunks)
  • tools/text_extractor/Dockerfile (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (2)
  • tools/classifier/Dockerfile
  • tools/text_extractor/Dockerfile
✨ Finishing Touches
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch UN-2609-Graceful-shutdown-for-tool-script-entrypoint

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Explain this complex logic.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai explain this code block.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and explain its main purpose.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai generate unit tests to generate unit tests for this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@kirtimanmishrazipstack kirtimanmishrazipstack marked this pull request as draft July 31, 2025 06:39
@kirtimanmishrazipstack kirtimanmishrazipstack requested review from a team, harini-venkataraman and johnyrahul and removed request for a team July 31, 2025 10:02
@kirtimanmishrazipstack kirtimanmishrazipstack marked this pull request as ready for review July 31, 2025 10:02
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (3)
runner/tests/signal-handling-demo/README.md (2)

16-16: Fix markdown formatting: Remove trailing punctuation from heading.

-# Results:
+# Results

20-42: Add language specification to fenced code blocks.

The code blocks should specify the language for better syntax highlighting and markdown compliance.

-```
+```console

Apply this change to both code blocks (lines 20-42 and 47-71) that contain console output.

Also applies to: 47-71

runner/tests/signal-handling-demo/shell-signal-test.py (1)

24-62: Excellent signal handler implementation with one minor inconsistency.

The signal handler properly captures both SIGTERM and SIGINT, provides detailed logging, and simulates realistic cleanup work. The use of sys.stdout.flush() ensures immediate output visibility.

However, there's a minor inconsistency:

-            time.sleep(10)  # Each cleanup task takes 1 second
+            time.sleep(10)  # Each cleanup task takes 10 seconds
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
Cache: Disabled due to Reviews > Disable Cache setting
Knowledge Base: Disabled due to Reviews > Disable Knowledge Base setting

📥 Commits

Reviewing files that changed from the base of the PR and between 450340f and 49a4352.

📒 Files selected for processing (4)
  • .gitignore (1 hunks)
  • runner/tests/signal-handling-demo/Dockerfile (1 hunks)
  • runner/tests/signal-handling-demo/README.md (1 hunks)
  • runner/tests/signal-handling-demo/shell-signal-test.py (1 hunks)
✅ Files skipped from review due to trivial changes (2)
  • .gitignore
  • runner/tests/signal-handling-demo/Dockerfile
🧰 Additional context used
🪛 markdownlint-cli2 (0.17.2)
runner/tests/signal-handling-demo/README.md

16-16: Trailing punctuation in heading
Punctuation: ':'

(MD026, no-trailing-punctuation)


20-20: Fenced code blocks should have a language specified

(MD040, fenced-code-language)


47-47: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: build
🔇 Additional comments (5)
runner/tests/signal-handling-demo/README.md (2)

1-74: Excellent documentation for signal handling testing.

This README provides comprehensive manual testing instructions that clearly demonstrate the signal forwarding problem and solution. The detailed output logs effectively show the difference between broken (shell wrapper) and working (dumb-init) scenarios.


4-14: Docker commands look correct for testing signal handling.

The build and run commands properly demonstrate both scenarios - broken shell wrapper and working dumb-init wrapper.

runner/tests/signal-handling-demo/shell-signal-test.py (3)

19-23: Clean class initialization.

The initialization properly sets up the necessary instance variables for tracking program state and timing.


63-123: Well-structured main program loop with comprehensive logging.

The implementation includes:

  • Proper signal handler registration
  • Clear startup instructions and PID display
  • Safety timeout mechanism (60 seconds max)
  • Detailed status logging every 2 seconds
  • Graceful handling of both signals and KeyboardInterrupt
  • Appropriate exit handling

1-17: Excellent overall script structure and documentation.

The script follows Python best practices with:

  • Proper shebang and comprehensive docstring
  • Clean imports and module structure
  • Clear main entry point with if __name__ == "__main__"

This is a well-designed testing tool for demonstrating signal handling behavior.

Also applies to: 125-128

Copy link
Contributor

@hari-kuriakose hari-kuriakose left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@github-actions
Copy link
Contributor

github-actions bot commented Aug 5, 2025

filepath function $$\textcolor{#23d18b}{\tt{passed}}$$ SUBTOTAL
$$\textcolor{#23d18b}{\tt{runner/src/unstract/runner/clients/test\_docker.py}}$$ $$\textcolor{#23d18b}{\tt{test\_logs}}$$ $$\textcolor{#23d18b}{\tt{1}}$$ $$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{runner/src/unstract/runner/clients/test\_docker.py}}$$ $$\textcolor{#23d18b}{\tt{test\_cleanup}}$$ $$\textcolor{#23d18b}{\tt{1}}$$ $$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{runner/src/unstract/runner/clients/test\_docker.py}}$$ $$\textcolor{#23d18b}{\tt{test\_cleanup\_skip}}$$ $$\textcolor{#23d18b}{\tt{1}}$$ $$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{runner/src/unstract/runner/clients/test\_docker.py}}$$ $$\textcolor{#23d18b}{\tt{test\_client\_init}}$$ $$\textcolor{#23d18b}{\tt{1}}$$ $$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{runner/src/unstract/runner/clients/test\_docker.py}}$$ $$\textcolor{#23d18b}{\tt{test\_get\_image\_exists}}$$ $$\textcolor{#23d18b}{\tt{1}}$$ $$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{runner/src/unstract/runner/clients/test\_docker.py}}$$ $$\textcolor{#23d18b}{\tt{test\_get\_image}}$$ $$\textcolor{#23d18b}{\tt{1}}$$ $$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{runner/src/unstract/runner/clients/test\_docker.py}}$$ $$\textcolor{#23d18b}{\tt{test\_get\_container\_run\_config}}$$ $$\textcolor{#23d18b}{\tt{1}}$$ $$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{runner/src/unstract/runner/clients/test\_docker.py}}$$ $$\textcolor{#23d18b}{\tt{test\_get\_container\_run\_config\_without\_mount}}$$ $$\textcolor{#23d18b}{\tt{1}}$$ $$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{runner/src/unstract/runner/clients/test\_docker.py}}$$ $$\textcolor{#23d18b}{\tt{test\_run\_container}}$$ $$\textcolor{#23d18b}{\tt{1}}$$ $$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{runner/src/unstract/runner/clients/test\_docker.py}}$$ $$\textcolor{#23d18b}{\tt{test\_get\_image\_for\_sidecar}}$$ $$\textcolor{#23d18b}{\tt{1}}$$ $$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{runner/src/unstract/runner/clients/test\_docker.py}}$$ $$\textcolor{#23d18b}{\tt{test\_sidecar\_container}}$$ $$\textcolor{#23d18b}{\tt{1}}$$ $$\textcolor{#23d18b}{\tt{1}}$$
$$\textcolor{#23d18b}{\tt{TOTAL}}$$ $$\textcolor{#23d18b}{\tt{11}}$$ $$\textcolor{#23d18b}{\tt{11}}$$

@sonarqubecloud
Copy link

sonarqubecloud bot commented Aug 5, 2025

@vishnuszipstack vishnuszipstack merged commit b482210 into main Aug 5, 2025
7 checks passed
@vishnuszipstack vishnuszipstack deleted the UN-2609-Graceful-shutdown-for-tool-script-entrypoint branch August 5, 2025 04:29
pk-zipstack pushed a commit that referenced this pull request Aug 20, 2025
…rocesses in tool containers (#1466)

* added tool invocation with dumb-init

* adding demo folder (to be removed)

* adding demo folder to runner/tests

* adding demo folder to runner/tests

* removing test folder

* small change

---------

Co-authored-by: vishnuszipstack <117254672+vishnuszipstack@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants