UN-2954: Enable line confidence extraction in LLMWhisperer V2 by Deepak-Kesavan · Pull Request #204 · Zipstack/unstract-sdk

Deepak-Kesavan · 2025-11-04T21:57:57Z

What

Enable line-level confidence score extraction in LLMWhisperer V2 adapter for the SDK.

Why

To support displaying average confidence scores of highlighted lines in Prompt Studio, we need to request and receive line confidence data from LLMWhisperer API.

How

Added INCLUDE_LINE_CONFIDENCE constant to WhispererConfig class
Modified get_whisperer_params() to include include_line_confidence parameter when highlighting is enabled
Set include_line_confidence to True when enable_highlight is True
Bumped llmwhisperer-client dependency to version 2.5.0 to support the new parameter
Bumped SDK version to v0.79.0

Relevant Docs

LLMWhisperer V2 API documentation

Related Issues or PRs

Issue: UN-2954
Related PR in main repo (frontend + backend changes)

Dependencies Versions / Env Variables

llmwhisperer-client==2.5.0 (bumped from >=2.2.1)

Notes on Testing

When highlighting is enabled, the LLMWhisperer API will now return confidence scores as the 5th element in highlight coordinate arrays
Coordinate format: [page, x, y, width, confidence]
Frontend changes in the main repo handle extraction and display of these confidence values

Screenshots

N/A - Backend changes only

Checklist

I have read and understood the Contribution Guidelines.

coderabbitai · 2025-11-04T21:58:06Z

Summary by CodeRabbit

Chores
- Updated minimum Python version requirement to 3.12+.
- Updated llmwhisperer-client dependency to version 2.5.0 or later.
- Bumped SDK version to 0.79.0.

Walkthrough

Bumps Python requirement to >=3.12,<3.13, updates llmwhisperer-client to >=2.5.0, increments SDK version to v0.79.0, and adds WhispererConfig.INCLUDE_LINE_CONFIDENCE with mapping from extra_params.enable_highlight.

Changes

Cohort / File(s)	Summary
Project metadata `pyproject.toml`	Updated Python requirement to `>=3.12,<3.13` and raised `llmwhisperer-client` constraint to `>=2.5.0`.
SDK version `src/unstract/sdk/__init__.py`	Updated module `__version__` from `v0.78.1` to `v0.79.0`.
Whisperer config & mapping `src/unstract/sdk/adapters/x2text/llm_whisperer_v2/src/constants.py`, `src/unstract/sdk/adapters/x2text/llm_whisperer_v2/src/helper.py`	Added `INCLUDE_LINE_CONFIDENCE = "include_line_confidence"` to `WhispererConfig`; `get_whisperer_params` now sets `WhispererConfig.INCLUDE_LINE_CONFIDENCE` from `extra_params.enable_highlight` (in addition to existing mappings).

Sequence Diagram(s)

sequenceDiagram
  participant Caller as Caller
  participant Helper as get_whisperer_params()
  participant Config as WhispererConfig

  Note over Caller,Helper `#DDEBF7`: Build Whisperer parameters
  Caller->>Helper: provide extra_params (contains enable_highlight)
  Helper-->>Config: set ADD_LINE_NOS (existing)
  Helper-->>Config: set INCLUDE_LINE_CONFIDENCE = extra_params.enable_highlight
  Note right of Config `#E6F4EA`: New config key added

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Check pyproject.toml constraint syntax and compatibility.
Verify any consumers of WhispererConfig handle the new INCLUDE_LINE_CONFIDENCE key.
Confirm SDK version bump is consistent with release process.

Pre-merge checks and finishing touches

✅ Passed checks (2 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and specifically describes the main change: enabling line confidence extraction in LLMWhisperer V2.
Description check	✅ Passed	The PR description follows the template with all major sections completed including What, Why, How, Dependencies, and Testing notes.

✨ Finishing touches

📝 Generate docstrings

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment
Commit unit tests in branch UN-2954-display-average-confidence-of-highlighted-line-in-prompt-studio

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

Cache: Disabled due to Reviews > Disable Cache setting

Knowledge base: Disabled due to Reviews -> Disable Knowledge Base setting

📥 Commits

Reviewing files that changed from the base of the PR and between 44ec893 and b428a80.

📒 Files selected for processing (1)

src/unstract/sdk/__init__.py (1 hunks)

🚧 Files skipped from review as they are similar to previous changes (1)

src/unstract/sdk/init.py

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

coderabbitai

Actionable comments posted: 0

🧹 Nitpick comments (1)

pyproject.toml (1)

72-72: Consider a more flexible version constraint.

The exact version pin (==2.5.0) prevents automatic security patches and bug fixes. Consider using a compatible release specifier like ~=2.5.0 or a version range like >=2.5.0,<3.0.0 to allow patch updates while maintaining compatibility.

However, if exact pinning is intentional for SDK stability and reproducibility, this is acceptable.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

Cache: Disabled due to Reviews > Disable Cache setting

Knowledge base: Disabled due to Reviews -> Disable Knowledge Base setting

📥 Commits

Reviewing files that changed from the base of the PR and between b85c150 and d2f8ad7.

📒 Files selected for processing (4)

pyproject.toml (1 hunks)
src/unstract/sdk/__init__.py (1 hunks)
src/unstract/sdk/adapters/x2text/llm_whisperer_v2/src/constants.py (1 hunks)
src/unstract/sdk/adapters/x2text/llm_whisperer_v2/src/helper.py (1 hunks)

🔇 Additional comments (3)

src/unstract/sdk/__init__.py (1)

1-1: LGTM!

The version bump to v0.78.2 is appropriate for this feature addition.

src/unstract/sdk/adapters/x2text/llm_whisperer_v2/src/helper.py (1)

203-204: LGTM!

The addition of INCLUDE_LINE_CONFIDENCE parameter is correctly mapped from extra_params.enable_highlight, mirroring the pattern used for ADD_LINE_NOS. This ensures that line confidence data is included when highlighting is enabled, as described in the PR objectives.

src/unstract/sdk/adapters/x2text/llm_whisperer_v2/src/constants.py (1)

75-75: LGTM!

The new constant INCLUDE_LINE_CONFIDENCE follows the established naming convention and is appropriately placed alongside related configuration keys.

- Add include_line_confidence parameter to LLMWhisperer V2 whisper requests - Enable confidence extraction when highlighting is enabled - Bump llmwhisperer-client to version 2.5.0 - Update requires-python to >=3.12 (required by llmwhisperer-client 2.5.0) - Bump SDK version to v0.78.2 - Update uv.lock with resolved dependencies

src/unstract/sdk/adapters/x2text/llm_whisperer_v2/src/helper.py

src/unstract/sdk/__init__.py

Co-authored-by: Chandrasekharan M <117059509+chandrasekharan-zipstack@users.noreply.github.com> Signed-off-by: Deepak K <89829542+Deepak-Kesavan@users.noreply.github.com>

Deepak-Kesavan requested review from jagadeeswaran-zipstack, johnyrahul and vishnuszipstack November 4, 2025 21:59

Deepak-Kesavan self-assigned this Nov 4, 2025

coderabbitai bot reviewed Nov 4, 2025

View reviewed changes

Deepak-Kesavan force-pushed the UN-2954-display-average-confidence-of-highlighted-line-in-prompt-studio branch from d2f8ad7 to 44ec893 Compare November 4, 2025 22:10

Deepak-Kesavan mentioned this pull request Nov 4, 2025

UN-2954: Display average confidence of highlighted lines in Prompt Studio Zipstack/unstract#1646

Merged

johnyrahul reviewed Nov 5, 2025

View reviewed changes

src/unstract/sdk/adapters/x2text/llm_whisperer_v2/src/helper.py Show resolved Hide resolved

johnyrahul approved these changes Nov 5, 2025

View reviewed changes

chandrasekharan-zipstack reviewed Nov 5, 2025

View reviewed changes

src/unstract/sdk/__init__.py Outdated Show resolved Hide resolved

Update src/unstract/sdk/__init__.py

b428a80

Co-authored-by: Chandrasekharan M <117059509+chandrasekharan-zipstack@users.noreply.github.com> Signed-off-by: Deepak K <89829542+Deepak-Kesavan@users.noreply.github.com>

vishnuszipstack approved these changes Nov 5, 2025

View reviewed changes

Deepak-Kesavan merged commit 609dae5 into main Nov 5, 2025
2 checks passed

Deepak-Kesavan deleted the UN-2954-display-average-confidence-of-highlighted-line-in-prompt-studio branch November 5, 2025 08:21

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

UN-2954: Enable line confidence extraction in LLMWhisperer V2#204

UN-2954: Enable line confidence extraction in LLMWhisperer V2#204
Deepak-Kesavan merged 2 commits intomainfrom
UN-2954-display-average-confidence-of-highlighted-line-in-prompt-studio

Deepak-Kesavan commented Nov 4, 2025 •

edited

Loading

Uh oh!

coderabbitai bot commented Nov 4, 2025 •

edited

Loading

Uh oh!

coderabbitai bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

Deepak-Kesavan commented Nov 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Why

How

Relevant Docs

Related Issues or PRs

Dependencies Versions / Env Variables

Notes on Testing

Screenshots

Checklist

Uh oh!

coderabbitai bot commented Nov 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Walkthrough

Changes

Sequence Diagram(s)

Estimated code review effort

Pre-merge checks and finishing touches

Uh oh!

coderabbitai bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Deepak-Kesavan commented Nov 4, 2025 •

edited

Loading

coderabbitai bot commented Nov 4, 2025 •

edited

Loading