Skip to content

seth/llmguard - #60335

Draft
snickell wants to merge 17 commits into
seth/pycall-llmguard-demofrom
seth/llmguard
Draft

seth/llmguard#60335
snickell wants to merge 17 commits into
seth/pycall-llmguard-demofrom
seth/llmguard

Conversation

@snickell

@snickell snickell commented Aug 11, 2024

Copy link
Copy Markdown
Contributor

Preview: NOT READY WILL CRASH

This is a broken+will-crash demo of how to integrate the LLMGuard pip with ActiveJob and Python.run(&block). Continuation of #60221.

I'm using this as a concrete example to shape the new Python.run API developed in seth/pretty-in-py #60300

Pathes to Release

Its quite possible this PR will be merged exactly as-is. The blocker is crashes in the two approaches to fulfilling the fantasy API this PR uses: Python.run(). Here's where it will currently crash.

Path 1: use the Python.run() API shim.

  1. This PR uses the Python.run API provided by seth/pretty-in-py: seth/pretty-in-py: Python.run API #60300
  2. However that API crashes in delayed_job, indications are the fix is a major Ruby version upgrade, seth/ruby-3.3: Upgrade to Ruby 3.3.4 #60329

Path 2: skip the API shim and move straight to the real thing.

  1. Skip the Python.run() API shim phase and build the real thing on top of PyCall thread a library I made to address our base issues: https://github.com/snickell/pycall_thread
  2. However, it crashes Ruby on exit, which means our tests can't pass. The fix is this low-level C issue to PyCall itself, seth/finalize: Call Py_FinalizeEx() when process exits red-data-tools/pycall.rb#187


def perform(prompt:)
results = Python.run do
pyimport "pycdo.aichat.safety", as: :safety

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently this will CRASH HERE. LLMGuard plays pretty rough. This PR is blocking on me fixing this underlying code.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant