…g set (#1785)

Fix two interacting bugs that prevented TTL from propagating to HITL
queue records:

1. WorkflowUtil.get_hitl_ttl_seconds was an OSS stub that always returned
   None. Now delegates to get_hitl_ttl_seconds_by_workflow via try/except
   import, falling back to None in OSS environments.

2. _push_to_queue_for_api_deployment never fetched TTL. Now mirrors the
   connector path by calling WorkflowUtil.get_hitl_ttl_seconds and passing
   ttl_seconds through to _create_queue_result and
   _enqueue_to_packet_or_regular_queue.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

…1806)

* refactor: Add dynamic plugin loading for enterprise components

## What

- Add dynamic plugin loading support to OSS codebase
- Enable enterprise components to be loaded at runtime without modifying tracked files

## Why

- Enterprise code was overwriting git-tracked OSS files causing dirty git state
- Need clean separation between OSS and enterprise codebases
- OSS should work independently without enterprise components

## How

- `unstract_migrations.py`: Uses try/except ImportError to load from `pluggable_apps.migrations_ext`
- `api_hub_usage_utils.py`: Uses try/except ImportError to load from `plugins.verticals_usage`
- `utils.py`: Uses try/except ImportError to load from `pluggable_apps.manual_review_v2` and `plugins.workflow_manager.workflow_v2.rule_engine`
- `backend.Dockerfile`: Conditional install of `requirements.txt` if present

## Can this PR break any existing features. If yes, please list possible items. If no, please explain why.

- No. The changes add optional plugin loading that gracefully falls back to default behavior when plugins are not present. Existing OSS functionality is preserved.

## Database Migrations

- None

## Env Config

- None

## Relevant Docs

- None

## Related Issues or PRs

- None

## Dependencies Versions

- None

## Notes on Testing

- OSS build: Verify app starts and works without enterprise plugins
- Enterprise build: Verify plugins are loaded and function correctly

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* refactor: Use get_plugin() for API Hub usage utilities

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Refactor random sampling logic in utils.py

Removed redundant import of random and exception handling for manual_review_v2.

Signed-off-by: Hari John Kuriakose <hari@zipstack.com>

* fix: Add Traefik port labels and clean up service ignore list

- Add explicit loadbalancer port labels for backend (8000) and frontend (3000)
  services in docker-compose to ensure proper Traefik routing
- Rename spawned_services to ignored_services for clarity
- Extend ignored_services list to include tool-classifier, tool-text_extractor,
  and worker-unified services that don't need environment setup

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: Update frontend Docker config for nginx serving

Update Traefik port label to 80 to match nginx and fix Dockerfile to use
BUILD_CONTEXT_PATH for the runtime config script in both dev and prod stages.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: Use ARG instead of ENV for BUILD_CONTEXT_PATH in frontend Dockerfile

Convert BUILD_CONTEXT_PATH from environment variable to build argument
for proper Docker multi-stage build support. ARGs must be declared
globally and re-declared in each stage that needs them.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* feat: Add HubSpot integration plugin for contact event tracking

- Add new integrations plugin category under backend/plugins/integrations/
- Create HubSpot plugin with event-based contact updates
- Track user milestone events: project creation, document upload,
  prompt run, tool export, and API deployment
- Plugin validates is_first_for_org flag and first org member status
- Remove unused hubspot_signup_api() stub from authentication_service
- Update subscription_helper to use plugin pattern for form submissions

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* [FIX] Fix HITL review screen showing "Never expires" despite TTL being set (#1785)

Fix two interacting bugs that prevented TTL from propagating to HITL
queue records:

1. WorkflowUtil.get_hitl_ttl_seconds was an OSS stub that always returned
   None. Now delegates to get_hitl_ttl_seconds_by_workflow via try/except
   import, falling back to None in OSS environments.

2. _push_to_queue_for_api_deployment never fetched TTL. Now mirrors the
   connector path by calling WorkflowUtil.get_hitl_ttl_seconds and passing
   ttl_seconds through to _create_queue_result and
   _enqueue_to_packet_or_regular_queue.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: Add dynamic plugin loading for enterprise components (#1736)

* refactor: Add dynamic plugin loading for enterprise components

- Add dynamic plugin loading support to OSS codebase
- Enable enterprise components to be loaded at runtime without modifying tracked files

- Enterprise code was overwriting git-tracked OSS files causing dirty git state
- Need clean separation between OSS and enterprise codebases
- OSS should work independently without enterprise components

- `unstract_migrations.py`: Uses try/except ImportError to load from `pluggable_apps.migrations_ext`
- `api_hub_usage_utils.py`: Uses try/except ImportError to load from `plugins.verticals_usage`
- `utils.py`: Uses try/except ImportError to load from `pluggable_apps.manual_review_v2` and `plugins.workflow_manager.workflow_v2.rule_engine`
- `backend.Dockerfile`: Conditional install of `requirements.txt` if present

- No. The changes add optional plugin loading that gracefully falls back to default behavior when plugins are not present. Existing OSS functionality is preserved.

- None

- None

- None

- None

- None

- OSS build: Verify app starts and works without enterprise plugins
- Enterprise build: Verify plugins are loaded and function correctly

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* refactor: Use get_plugin() for API Hub usage utilities

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Refactor random sampling logic in utils.py

Removed redundant import of random and exception handling for manual_review_v2.

Signed-off-by: Hari John Kuriakose <hari@zipstack.com>

* fix: Add Traefik port labels and clean up service ignore list

- Add explicit loadbalancer port labels for backend (8000) and frontend (3000)
  services in docker-compose to ensure proper Traefik routing
- Rename spawned_services to ignored_services for clarity
- Extend ignored_services list to include tool-classifier, tool-text_extractor,
  and worker-unified services that don't need environment setup

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: Update frontend Docker config for nginx serving

Update Traefik port label to 80 to match nginx and fix Dockerfile to use
BUILD_CONTEXT_PATH for the runtime config script in both dev and prod stages.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: Use ARG instead of ENV for BUILD_CONTEXT_PATH in frontend Dockerfile

Convert BUILD_CONTEXT_PATH from environment variable to build argument
for proper Docker multi-stage build support. ARGs must be declared
globally and re-declared in each stage that needs them.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: typo in ignored services var name

* fix: handle script execution via entrypoint

---------

Signed-off-by: Hari John Kuriakose <hari@zipstack.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* feat: Add auth error code for forbidden emails (#1789)

* feat: add auth error code and frontend error display

* fix: run frontend dev server on port 80 and add signup handler

- Set PORT=80 env var in frontend Dockerfile development stage
- Change EXPOSE from 3000 to 80 to match production nginx
- Add handleSignup function and pass to LoginForm component

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix: add ARG declaration to Dockerfile stages using BUILD_CONTEXT_PATH

SonarQube flagged that ARG must be declared in each Docker build stage
where it is used. Added the missing ARG BUILD_CONTEXT_PATH to both
development and builder stages.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>

* [MISC] Improve dev experience by adding a compose debug override (#1765)

* [MISC] Improve Docker dev experience: separate debugpy, optimize memory, add V2 workers support

- Move debugpy to optional compose.debug.yaml for cleaner default dev setup
- Update compose.override.yaml with memory-optimized settings (1 worker, 2 threads)
- Add V2 workers configuration with build definitions
- Move V1 workers to optional workers-v1 profile
- Use modern uv run python -Xfrozen_modules=off pattern for services
- Updated README with compose.debug.yaml usage instructions

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* [MISC] Add Docker Compose version requirement and scheduler comment

- Add Docker Compose 2.24.4+ requirement note for !override directive
- Add comment explaining why worker-log-history-scheduler-v2 has no watch

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* [MISC] Fix debug ports table in README

- Fix port order: runner=5679, platform=5680, prompt=5681
- Remove x2text-service (not in compose.debug.yaml)
- Add V2 workers debug ports (5682-5688)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* [MISC] Simplify sample.compose.override.yaml

Remove db image override and V1 worker command overrides from
sample file. Users should reference compose.override.yaml for
actual dev setup.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* [FIX] Optimize queries made by worker and retry config of worker base client (#1798)

* Fix production queryset performance and retry amplification

Resolves 39.8s GET /internal/v1/file-execution/<uuid>/ latency by:
- Removing 7 debug COUNT(*) full table scans from get_queryset()
- Adding get_object() O(1) PK lookup override in ViewSet
- Adding @with_cache decorators to pipeline fetch methods
- Adding pipeline_data_key() to prevent cache key collisions
- Fixing urllib3 retry amplification: clear status_forcelist, let app-level retries handle status codes
- Use config defaults instead of hardcoded retry values

Root cause: Count queries on every request + retry storm (urllib3 × app-level).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Address PR #1798 review comments

- Chain exception context in get_object() (Ruff B904)
- Fix cache key collision: get_workflow_definition() now uses
  CacheType.WORKFLOW_DEFINITION instead of CacheType.WORKFLOW to avoid
  type mismatch with get_workflow() sharing the same cache key
- Include check_active in get_pipeline_data() cache key to prevent
  active-status bypass when check_active=False is cached first
- Refactor status() and update_hash() to use self.get_object() instead
  of duplicating manual queryset + org filtering; add except APIException
  pass-through so NotFound propagates as 404 not 500

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* UN-2971 [FEAT] Pass selectedProduct to login/signup API for OAuth product scope (#1803)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Signed-off-by: Hari John Kuriakose <hari@zipstack.com>
Co-authored-by: Claude <noreply@anthropic.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix: Scope HubSpot milestone count checks to current organization

PromptStudioOutputManager and DocumentManager lack
DefaultOrganizationManagerMixin, so .objects.count() was counting
across ALL organizations. Filter through the tool FK to CustomTool
(which is org-scoped) to get correct per-org counts.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* [REFACTOR] Extract HubSpot notification logic into shared utility

Move all _notify_hubspot_* methods from views into a shared
utils/hubspot_notify.py module with a single notify_hubspot_event()
function, reducing duplication across prompt_studio views and
api_deployment_views.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Signed-off-by: Hari John Kuriakose <hari@zipstack.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: vishnuszipstack <117254672+vishnuszipstack@users.noreply.github.com>
Co-authored-by: Chandrasekharan M <117059509+chandrasekharan-zipstack@users.noreply.github.com>
Co-authored-by: vishnuszipstack <vishnu@zipstack.com>
Co-authored-by: Gayathri <142381512+gaya3-zipstack@users.noreply.github.com>