Skip to content

Functions skip variables subquery#11036

Merged
fogelito merged 2 commits into1.8.xfrom
functions-skip-subqueiry
Dec 29, 2025
Merged

Functions skip variables subquery#11036
fogelito merged 2 commits into1.8.xfrom
functions-skip-subqueiry

Conversation

@fogelito
Copy link
Copy Markdown
Contributor

What does this PR do?

(Provide a description of what this PR does and why it's needed.)

Test Plan

(Write your test plan here. If you changed any code, please provide us with clear instructions on how you verified your changes work. Screenshots may also be helpful.)

Related PRs and Issues

  • (Related PR or issue)

Checklist

  • Have you read the Contributing Guidelines on issues?
  • If the PR includes a change to an API's metadata (desc, label, params, etc.), does it also include updated API specs and example docs?

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Dec 29, 2025

📝 Walkthrough

Walkthrough

Batch processing for event-driven function lookups was increased from 30 to 100. Query ordering was changed from ascending by function name to ascending by the internal sequence field. The listing query now selects only $id and events. Each function document is re-fetched individually inside the processing loop (per-function caching) before further processing.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Pre-merge checks and finishing touches

❌ Failed checks (2 warnings)
Check name Status Explanation Resolution
Description check ⚠️ Warning The PR description is a blank template with no meaningful content related to the changeset. All key sections (What does this PR do, Test Plan, Related PRs) are empty. Fill in the description sections with details about why the subquery optimization was needed, what performance improvements are expected, and how the changes were tested.
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (1 passed)
Check name Status Explanation
Title check ✅ Passed The title 'Functions skip variables subquery' is directly related to the main change: avoiding a variables subquery by fetching only necessary fields and caching full documents.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch functions-skip-subqueiry

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions
Copy link
Copy Markdown

github-actions bot commented Dec 29, 2025

Security Scan Results for PR

Docker Image Scan Results

Package Version Vulnerability Severity
libpng 1.6.51-r0 CVE-2025-66293 HIGH
libpng-dev 1.6.51-r0 CVE-2025-66293 HIGH

Source Code Scan Results

🎉 No vulnerabilities found!

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (1)
src/Appwrite/Platform/Workers/Functions.php (1)

151-154: Clarify the comment about caching.

The comment "get variables subqueries cached" is misleading. This code fetches the full function document (including variables) from the database—it's not necessarily leveraging a cache.

Consider updating the comment to be more accurate:

🔎 Suggested comment clarification
-                    /**
-                     * get variables subqueries cached
-                     */
+                    /**
+                     * Fetch full function document including variables for execution
+                     */
                     $function = $dbForProject->getDocument('functions', $function->getId());
📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between a0514d2 and 774e3af.

📒 Files selected for processing (1)
  • src/Appwrite/Platform/Workers/Functions.php
🧰 Additional context used
🧠 Learnings (2)
📚 Learning: 2025-11-12T04:07:12.576Z
Learnt from: abnegate
Repo: appwrite/appwrite PR: 10800
File: src/Appwrite/Platform/Modules/Databases/Http/Databases/Collections/Documents/Bulk/Delete.php:153-154
Timestamp: 2025-11-12T04:07:12.576Z
Learning: In Appwrite's event queue system, `queueForRealtime`, `queueForFunctions`, and `queueForWebhooks` are copied from `queueForEvents`. Therefore, when resetting event queues in transaction staging paths, only `$queueForEvents->reset()` needs to be called in most cases. The other queues only require explicit reset when they are triggered in a loop.

Applied to files:

  • src/Appwrite/Platform/Workers/Functions.php
📚 Learning: 2025-10-23T08:06:38.889Z
Learnt from: abnegate
Repo: appwrite/appwrite PR: 10546
File: src/Appwrite/Platform/Workers/Migrations.php:144-148
Timestamp: 2025-10-23T08:06:38.889Z
Learning: In the Appwrite codebase, migration workers receive already-validated data from queued jobs. Query validation using Query::parseQueries() happens at the API endpoint level (with try-catch for QueryException) before jobs are queued, so workers in src/Appwrite/Platform/Workers/Migrations.php don't need redundant validation.

Applied to files:

  • src/Appwrite/Platform/Workers/Functions.php
🧬 Code graph analysis (1)
src/Appwrite/Platform/Workers/Functions.php (1)
src/Appwrite/Databases/TransactionState.php (1)
  • getDocument (42-83)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (5)
  • GitHub Check: Setup & Build Appwrite Image
  • GitHub Check: Linter
  • GitHub Check: Setup & Build Appwrite Image
  • GitHub Check: CodeQL
  • GitHub Check: scan
🔇 Additional comments (3)
src/Appwrite/Platform/Workers/Functions.php (3)

125-126: LGTM! Batch limit increase is appropriate.

Increasing the batch limit from 30 to 100 will reduce the number of database round-trips when processing many functions, improving overall throughput.


130-130: Selective field fetch optimization is well-implemented.

Fetching only $id and events fields here avoids loading variable subqueries for functions that won't match the event filter. The full document is retrieved later (line 154) only for functions that will actually execute.


133-133: The ordering by $sequence is intentional and aligns with function execution order handling documented in the repository history. Using $sequence (internal document creation order) ensures deterministic execution when multiple functions subscribe to the same events, which is more appropriate than name-based ordering.

@github-actions
Copy link
Copy Markdown

github-actions bot commented Dec 29, 2025

✨ Benchmark results

  • Requests per second: 1,144
  • Requests with 200 status code: 205,971
  • P99 latency: 0.169033196

⚡ Benchmark Comparison

Metric This PR Latest version
RPS 1,144 1,203
200 205,971 216,556
P99 0.169033196 0.169839972

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

♻️ Duplicate comments (1)
src/Appwrite/Platform/Workers/Functions.php (1)

152-156: Critical: N+1 query problem from per-function re-fetch.

Re-fetching each function individually inside the loop creates a classic N+1 query problem. With the increased batch size of 100, this could result in 100 additional database queries per batch iteration, significantly degrading performance.

🔎 Recommended solutions

Option 1: Remove selective fetch and fetch all data initially

                $functions = $dbForProject->find('functions', [
-                    Query::select(['$id', 'events']), // Skip variables subqueries
                    Query::contains('events', $events),
                    Query::limit($limit),
                    Query::offset($offset),
                    Query::orderAsc('$sequence'),
                ]);

                // ... inside loop ...

                foreach ($functions as $function) {
                    if ($isResourceBlocked($project, RESOURCE_TYPE_FUNCTIONS, $function->getId())) {
                        Console::log('Function ' . $function->getId() . ' is blocked, skipping execution.');
                        continue;
                    }

-                    /**
-                     * get variables subqueries cached
-                     */
-                    $function = $dbForProject->getDocument('functions', $function->getId());
-
                    Console::success('Iterating function: ' . $function->getAttribute('name'));

Option 2: Batch re-fetch after filtering (if selective fetch is necessary)

+                // Collect function IDs after filtering
+                $functionIds = [];
                foreach ($functions as $function) {
                    if (!array_intersect($events, $function->getAttribute('events', []))) {
                        continue;
                    }

                    if ($isResourceBlocked($project, RESOURCE_TYPE_FUNCTIONS, $function->getId())) {
                        Console::log('Function ' . $function->getId() . ' is blocked, skipping execution.');
                        continue;
                    }
+                    $functionIds[] = $function->getId();
+                }
+
+                // Batch fetch all needed functions at once
+                if (!empty($functionIds)) {
+                    $fullFunctions = $dbForProject->find('functions', [
+                        Query::equal('$id', $functionIds),
+                        Query::limit(count($functionIds))
+                    ]);
+
+                    foreach ($fullFunctions as $function) {
                        Console::success('Iterating function: ' . $function->getAttribute('name'));
                        // ... rest of execution logic

Option 1 is preferred unless there's strong evidence that the selective fetch provides significant performance benefits that outweigh the N+1 query cost.

🧹 Nitpick comments (1)
src/Appwrite/Platform/Workers/Functions.php (1)

143-145: Remove redundant event intersection check.

The array_intersect check appears redundant since line 131 already filters functions using Query::contains('events', $events). If the database query is working correctly, all returned functions should already have at least one matching event.

🔎 Proposed fix to remove redundant check
                foreach ($functions as $function) {
-                    if (!array_intersect($events, $function->getAttribute('events', []))) {
-                        continue;
-                    }
-
                    if ($isResourceBlocked($project, RESOURCE_TYPE_FUNCTIONS, $function->getId())) {

If this check is intentionally defensive (e.g., to handle edge cases where the database query might not filter correctly), please add a comment explaining why it's necessary.

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 774e3af and 417bb22.

📒 Files selected for processing (1)
  • src/Appwrite/Platform/Workers/Functions.php
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: 2025-08-11T10:32:38.142Z
Learnt from: abnegate
Repo: appwrite/appwrite PR: 10297
File: app/controllers/api/users.php:649-657
Timestamp: 2025-08-11T10:32:38.142Z
Learning: In Appwrite's utopia-php/database (version 0.77.* and later), the database layer internally handles filter separation for count operations. When using `$dbForProject->count()` within a `skipFilters` closure, there's no need to manually separate filter queries using `Query::groupByType($queries)['filters']` as the database handles this automatically.

Applied to files:

  • src/Appwrite/Platform/Workers/Functions.php
🧬 Code graph analysis (1)
src/Appwrite/Platform/Workers/Functions.php (2)
src/Appwrite/Platform/Modules/Projects/Http/Projects/XList.php (1)
  • find (163-197)
src/Appwrite/Databases/TransactionState.php (1)
  • getDocument (42-83)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
  • GitHub Check: Setup & Build Appwrite Image
  • GitHub Check: Setup & Build Appwrite Image
  • GitHub Check: scan
🔇 Additional comments (2)
src/Appwrite/Platform/Workers/Functions.php (2)

134-134: Good improvement: ordering by $sequence.

Ordering by $sequence provides a more deterministic and stable sort order compared to alphabetical ordering by name, especially since function names can be changed by users.


131-131: No action needed. Query::contains('events', $events) correctly matches functions where the events array contains any of the values in the $events array, which is the intended behavior for this change.

@fogelito fogelito merged commit 8a52975 into 1.8.x Dec 29, 2025
109 of 112 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants