Skip to content

Truncate logs in function worker#9773

Merged
Meldiron merged 7 commits intoappwrite:1.7.xfrom
samikshaaagarwal:samiksha/9510
Aug 21, 2025
Merged

Truncate logs in function worker#9773
Meldiron merged 7 commits intoappwrite:1.7.xfrom
samikshaaagarwal:samiksha/9510

Conversation

@samikshaaagarwal
Copy link
Copy Markdown

@samikshaaagarwal samikshaaagarwal commented May 16, 2025

Fixes #9510

Checklist

  • Have you read the Contributing Guidelines on issues?
  • If the PR includes a change to an API's metadata (desc, label, params, etc.), does it also include updated API specs and example docs?

Testing Method

Following curls were utilised:

  1. Via /executions api with async = true

Curl derived from console

curl 'http://localhost/v1/functions/6825588000018d78a2be/executions' \
  -H 'Accept: */*' \
  -H 'Accept-Language: en-GB,en-US;q=0.9,en;q=0.8' \
  -H 'Connection: keep-alive' \
  -b 'a_session_console_legacy=TOKEN' \
  -H 'DNT: 1' \
  -H 'Origin: http://localhost' \
  -H 'Referer: http://localhost/console/project-6821d9471ced049f3ed2/functions/function-6825588000018d78a2be/executions/execute-function' \
  -H 'Sec-Fetch-Dest: empty' \
  -H 'Sec-Fetch-Mode: cors' \
  -H 'Sec-Fetch-Site: same-origin' \
  -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/136.0.0.0 Safari/537.36' \
  -H 'X-Appwrite-Mode: admin' \
  -H 'X-Appwrite-Project: 6821d9471ced049f3ed2' \
  -H 'X-Appwrite-Response-Format: 1.6.0' \
  -H 'X-Fallback-Cookies: {"a_session_console":"TOKEN"}' \
  -H 'content-type: application/json' \
  -H 'sec-ch-ua: "Not.A/Brand";v="99", "Chromium";v="136"' \
  -H 'sec-ch-ua-mobile: ?0' \
  -H 'sec-ch-ua-platform: "macOS"' \
  -H 'x-sdk-language: web' \
  -H 'x-sdk-name: Console' \
  -H 'x-sdk-platform: console' \
  -H 'x-sdk-version: 1.1.0' \
  --data-raw '{"body":"","async":true,"path":"/","method":"GET","headers":{"null":""}}'
  1. Via /executions api with async = false
Same as above with async = false
  1. Via domains
curl 'http://682558800a0d98f1150c.functions.localhost' \
  -H 'Accept: */*' \
  -H 'Accept-Language: en-GB,en-US;q=0.9,en;q=0.8' \
  -H 'Connection: keep-alive' \
  -X POST \
  --data-raw '{"body":"","async":false,"path":"/","method":"GET","headers":{"null":""}}' \
  --output output.txt \
  --silent --show-error \
  2>> output.txt

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented May 16, 2025

📝 Walkthrough

Walkthrough

Adds max length constants for function logs and errors. Updates the executions collection schema to use these constants for logs and errors field size. Implements truncation logic in controllers, HTTP module, and worker to cap logs/errors at the defined limits, prefixing a warning and keeping the tail content. Applies truncation before persisting execution documents in multiple code paths. Introduces an end-to-end test and a test function resource that generate oversized logs/errors to validate truncation behavior.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Assessment against linked issues

Objective Addressed Explanation
Handle executions where logs/errors exceed 1,000,000 chars by truncating with a warning so execution completes instead of staying "processing" (#9510)

Assessment against linked issues: Out-of-scope changes

(none)

Suggested reviewers

  • Meldiron

Tip

🔌 Remote MCP (Model Context Protocol) integration is now available!

Pro plan users can now connect to remote MCP servers from the Integrations page. Connect with popular remote MCPs such as Notion and Linear to add more context to your reviews and chats.

✨ Finishing Touches
  • 📝 Generate Docstrings
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

CodeRabbit Commands (Invoked using PR/Issue comments)

Type @coderabbitai help to get the list of available commands.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Status, Documentation and Community

  • Visit our Status Page to check the current availability of CodeRabbit.
  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@github-actions
Copy link
Copy Markdown

github-actions bot commented May 16, 2025

Security Scan Results for PR

Docker Image Scan Results

Package Version Vulnerability Severity
binutils 2.42-r0 CVE-2025-0840 HIGH
git 2.45.3-r0 CVE-2025-46334 HIGH
git 2.45.3-r0 CVE-2025-48384 HIGH
git 2.45.3-r0 CVE-2025-48385 HIGH
git-init-template 2.45.3-r0 CVE-2025-46334 HIGH
git-init-template 2.45.3-r0 CVE-2025-48384 HIGH
git-init-template 2.45.3-r0 CVE-2025-48385 HIGH
icu 74.2-r0 CVE-2025-5222 HIGH
icu-data-en 74.2-r0 CVE-2025-5222 HIGH
icu-dev 74.2-r0 CVE-2025-5222 HIGH
icu-libs 74.2-r0 CVE-2025-5222 HIGH
libecpg 16.8-r0 CVE-2025-8714 HIGH
libecpg 16.8-r0 CVE-2025-8715 HIGH
libecpg-dev 16.8-r0 CVE-2025-8714 HIGH
libecpg-dev 16.8-r0 CVE-2025-8715 HIGH
libexpat 2.6.4-r0 CVE-2024-8176 HIGH
libpq 16.8-r0 CVE-2025-8714 HIGH
libpq 16.8-r0 CVE-2025-8715 HIGH
libpq-dev 16.8-r0 CVE-2025-8714 HIGH
libpq-dev 16.8-r0 CVE-2025-8715 HIGH
libxml2 2.12.7-r0 CVE-2024-56171 HIGH
libxml2 2.12.7-r0 CVE-2025-24928 HIGH
libxml2 2.12.7-r0 CVE-2025-27113 HIGH
libxml2 2.12.7-r0 CVE-2025-32414 HIGH
libxml2 2.12.7-r0 CVE-2025-32415 HIGH
postgresql16-dev 16.8-r0 CVE-2025-8714 HIGH
postgresql16-dev 16.8-r0 CVE-2025-8715 HIGH
pyc 3.12.9-r0 CVE-2024-12718 HIGH
pyc 3.12.9-r0 CVE-2025-4138 HIGH
pyc 3.12.9-r0 CVE-2025-4330 HIGH
pyc 3.12.9-r0 CVE-2025-4517 HIGH
python3 3.12.9-r0 CVE-2024-12718 HIGH
python3 3.12.9-r0 CVE-2025-4138 HIGH
python3 3.12.9-r0 CVE-2025-4330 HIGH
python3 3.12.9-r0 CVE-2025-4517 HIGH
python3-pyc 3.12.9-r0 CVE-2024-12718 HIGH
python3-pyc 3.12.9-r0 CVE-2025-4138 HIGH
python3-pyc 3.12.9-r0 CVE-2025-4330 HIGH
python3-pyc 3.12.9-r0 CVE-2025-4517 HIGH
python3-pycache-pyc0 3.12.9-r0 CVE-2024-12718 HIGH
python3-pycache-pyc0 3.12.9-r0 CVE-2025-4138 HIGH
python3-pycache-pyc0 3.12.9-r0 CVE-2025-4330 HIGH
python3-pycache-pyc0 3.12.9-r0 CVE-2025-4517 HIGH
sqlite-libs 3.45.3-r1 CVE-2025-29087 HIGH
xz 5.6.2-r0 CVE-2025-31115 HIGH
xz-libs 5.6.2-r0 CVE-2025-31115 HIGH
golang.org/x/crypto v0.31.0 CVE-2025-22869 HIGH
golang.org/x/oauth2 v0.24.0 CVE-2025-22868 HIGH
stdlib 1.22.10 CVE-2025-47907 HIGH

Source Code Scan Results

🎉 No vulnerabilities found!

@samikshaaagarwal samikshaaagarwal changed the base branch from main to 1.6.x May 19, 2025 15:29
@samikshaaagarwal
Copy link
Copy Markdown
Author

samikshaaagarwal commented May 19, 2025

While there is duplicity in the introduced code, should pick up the chore of refactoring and consolidating all execution updation code at one place separately. The whole code of updating function execution document should be stored at a single place.

CC @stnguyen90

@samikshaaagarwal samikshaaagarwal marked this pull request as ready for review May 19, 2025 15:31
@samikshaaagarwal
Copy link
Copy Markdown
Author

@stnguyen90 can you please take a look at this before the branch goes very stale. Thanks.

@vermakhushboo
Copy link
Copy Markdown
Contributor

@stnguyen90 can you please take a look at this before the branch goes very stale. Thanks.

@samikshaaagarwal could you change the base to 1.7.x?

@vermakhushboo vermakhushboo mentioned this pull request Aug 20, 2025
2 tasks
@vermakhushboo vermakhushboo changed the base branch from 1.6.x to 1.6.x-log-truncation August 20, 2025 12:49
@vermakhushboo vermakhushboo changed the base branch from 1.6.x-log-truncation to 1.7.x August 20, 2025 13:25
@pkg-pr-new
Copy link
Copy Markdown

pkg-pr-new bot commented Aug 20, 2025

Open in StackBlitz

npm i https://pkg.pr.new/appwrite/appwrite/@appwrite.io/console@9773

commit: 1bd6b64

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
src/Appwrite/Platform/Modules/Functions/Http/Executions/Create.php (2)

137-141: Probable bug: literal string used in empty() prevents intended check

empty('headers') is always false; the condition devolves to “if not an array, JSON-decode”, even when the param is actually empty. Use the variable-variable as intended and provide a safe fallback.

-        foreach ($assocParams as $assocParam) {
-            if (!empty('headers') && !is_array($$assocParam)) {
-                $$assocParam = \json_decode($$assocParam, true);
-            }
-        }
+        foreach ($assocParams as $assocParam) {
+            if (!empty($$assocParam) && !\is_array($$assocParam)) {
+                $$assocParam = \json_decode($$assocParam, true) ?? [];
+            }
+        }

143-148: Booleans parsing: typo and fragile conversion

  • Typo in loop variable ($booleamParam) hurts readability.
  • Current parsing skips "0" because empty('0') is true, and only handles "true"/"false". Prefer FILTER_VALIDATE_BOOLEAN.
-        foreach ($booleanParams as $booleamParam) {
-            if (!empty($$booleamParam) && !is_bool($$booleamParam)) {
-                $$booleamParam = $$booleamParam === "true" ? true : false;
-            }
-        }
+        foreach ($booleanParams as $booleanParam) {
+            if (isset($$booleanParam) && !\is_bool($$booleanParam)) {
+                $$booleanParam = \filter_var($$booleanParam, FILTER_VALIDATE_BOOLEAN, FILTER_NULL_ON_FAILURE) ?? false;
+            }
+        }
♻️ Duplicate comments (1)
app/controllers/general.php (1)

629-638: Apply the same guard for errors truncation

Mirrors the logs fix; keeps both paths consistent.

-            if (\is_string($errors) && \strlen($errors) > $maxErrorLength) {
-                $warningMessage = "[WARNING] Errors truncated. The output exceeded {$maxErrorLength} characters.\n";
-                $warningLength = \strlen($warningMessage);
-                $maxContentLength = $maxErrorLength - $warningLength;
-                $errors = $warningMessage . \substr($errors, -$maxContentLength);
-            }
+            if (\is_string($errors) && \strlen($errors) > $maxErrorLength) {
+                $warningMessage = "[WARNING] Errors truncated. The output exceeded {$maxErrorLength} characters.\n";
+                $warningLength = \strlen($warningMessage);
+                $maxContentLength = max(0, $maxErrorLength - $warningLength);
+                $errors = $warningMessage . ($maxContentLength > 0 ? \substr($errors, -$maxContentLength) : '');
+            }
🧹 Nitpick comments (15)
app/init/constants.php (1)

146-147: Good centralization; minor consistency/naming nits and potential reuse

Introducing limits as constants is the right move to keep DB schema and code in sync. Two small follow-ups:

  • Use numeric separators (consistent with the rest of this file) for readability.
  • Consider whether build logs should also reference a constant to avoid the remaining 1000000 magic number in the deployments collection (see projects.php buildLogs). If the same limit is desired, reuse; if not, introduce APP_BUILD_LOG_LENGTH_LIMIT.

Apply this small consistency tweak:

-const APP_FUNCTION_LOG_LENGTH_LIMIT = 1000000;
-const APP_FUNCTION_ERROR_LENGTH_LIMIT = 1000000;
+const APP_FUNCTION_LOG_LENGTH_LIMIT = 1_000_000;
+const APP_FUNCTION_ERROR_LENGTH_LIMIT = 1_000_000;
tests/resources/functions/error-truncation/index.js (1)

1-7: Make test resilient to future limit changes by reading the limit from env

Solid test resource. To decouple from the hardcoded 1,000,000 and keep the test robust if the limit changes, read the limit from an env var (falling back to the current value). This also mirrors the pattern used in other tests.

Apply:

-module.exports = async(context) => {
-    // Create a string that is 1000001 characters long (exceeds the 1000000 limit)
-    const longString = 'z' + 'a'.repeat(1000000);
+module.exports = async (context) => {
+    // Create a string that is (LIMIT + 1) characters long to force truncation
+    const LIMIT = Number(process.env.APP_FUNCTION_ERROR_LENGTH_LIMIT) || 1_000_000;
+    const longString = 'z' + 'a'.repeat(LIMIT);
app/config/collections/projects.php (1)

1709-1716: Avoid remaining magic number for build logs (optional)

deployments.buildLogs.size still uses a raw 1000000. If the intention is to keep it equal to function logs, consider reusing APP_FUNCTION_LOG_LENGTH_LIMIT. If you want an independent knob, introduce APP_BUILD_LOG_LENGTH_LIMIT in constants and use it here. This removes a lingering magic number and eases future changes.

Potential change (if reusing the same limit):

'size' => APP_FUNCTION_LOG_LENGTH_LIMIT,

Or, with a dedicated constant:

  • Add in app/init/constants.php:
    const APP_BUILD_LOG_LENGTH_LIMIT = 1_000_000;
  • Use here:
    'size' => APP_BUILD_LOG_LENGTH_LIMIT,
tests/resources/functions/log-truncation/index.js (1)

1-13: Harness looks good; align with env-configurable limit (optional)

This correctly forces truncation and exercises “truncate from the beginning” by dropping the initial sentinel char. For future-proofing, mirror the error test by reading the limit from env with a fallback.

-module.exports = async(context) => {
-    // Create a string that is 1000001 characters long (exceeds the 1000000 limit)
-    const longString = 'z' + 'a'.repeat(1000000);
+module.exports = async (context) => {
+    // Create a string that is (LIMIT + 1) characters long to force truncation
+    const LIMIT = Number(process.env.APP_FUNCTION_LOG_LENGTH_LIMIT) || 1_000_000;
+    const longString = 'z' + 'a'.repeat(LIMIT);
src/Appwrite/Platform/Modules/Functions/Http/Executions/Create.php (2)

426-446: Deduplicate truncation logic and guard edge cases with a tiny helper

The logs/errors truncation blocks are nearly identical. A local helper avoids duplication and centralizes edge cases (non-strings, pathological limits). The behavior remains identical: keep the tail, prepend a warning, fit within the limit.

Apply:

-            $maxLogLength = APP_FUNCTION_LOG_LENGTH_LIMIT;
-            $logs = $executionResponse['logs'] ?? '';
-
-            if (\is_string($logs) && \strlen($logs) > $maxLogLength) {
-                $warningMessage = "[WARNING] Logs truncated. The output exceeded {$maxLogLength} characters.\n";
-                $warningLength = \strlen($warningMessage);
-                $maxContentLength = $maxLogLength - $warningLength;
-                $logs = $warningMessage . \substr($logs, -$maxContentLength);
-            }
-
-            // Truncate errors if they exceed the limit
-            $maxErrorLength = APP_FUNCTION_ERROR_LENGTH_LIMIT;
-            $errors = $executionResponse['errors'] ?? '';
-
-            if (\is_string($errors) && \strlen($errors) > $maxErrorLength) {
-                $warningMessage = "[WARNING] Errors truncated. The output exceeded {$maxErrorLength} characters.\n";
-                $warningLength = \strlen($warningMessage);
-                $maxContentLength = $maxErrorLength - $warningLength;
-                $errors = $warningMessage . \substr($errors, -$maxContentLength);
-            }
+            $truncate = static function ($text, int $limit, string $label): string {
+                if (!\is_string($text)) {
+                    return '';
+                }
+                if (\strlen($text) <= $limit) {
+                    return $text;
+                }
+                $warning = "[WARNING] {$label} truncated. The output exceeded {$limit} characters.\n";
+                $maxContentLength = max(0, $limit - \strlen($warning));
+                return $warning . \substr($text, -$maxContentLength);
+            };
+
+            $logs = $truncate($executionResponse['logs'] ?? '', APP_FUNCTION_LOG_LENGTH_LIMIT, 'Logs');
+            $errors = $truncate($executionResponse['errors'] ?? '', APP_FUNCTION_ERROR_LENGTH_LIMIT, 'Errors');
 
             /** Update execution status */
             $status = $executionResponse['statusCode'] >= 500 ? 'failed' : 'completed';
             $execution->setAttribute('status', $status);
             $execution->setAttribute('responseStatusCode', $executionResponse['statusCode']);
             $execution->setAttribute('responseHeaders', $headersFiltered);
-            $execution->setAttribute('logs', $logs);
-            $execution->setAttribute('errors', $errors);
+            $execution->setAttribute('logs', $logs);
+            $execution->setAttribute('errors', $errors);

Optionally, if you expect multi-byte content in logs/errors and want to avoid splitting code points, consider mb_strcut with mb_strlen in the helper.

Also applies to: 452-454


83-85: Nit: class name casing to match imported symbol

You import Utopia\Validator\WhiteList but instantiate new Whitelist(...). PHP class names are case-insensitive, so this works, but matching the import’s case improves clarity.

src/Appwrite/Platform/Workers/Functions.php (3)

553-561: Harden truncation logic against edge-case limits

Great addition. One small guard avoids pathological configs (e.g., if a future limit is set shorter than the warning itself), which today would create a negative length and potentially bloat the final string. Clamp content length to >= 0.

Apply this diff:

-            if (\is_string($logs) && \strlen($logs) > $maxLogLength) {
-                $warningMessage = "[WARNING] Logs truncated. The output exceeded {$maxLogLength} characters.\n";
-                $warningLength = \strlen($warningMessage);
-                $maxContentLength = $maxLogLength - $warningLength;
-                $logs = $warningMessage . \substr($logs, -$maxContentLength);
-            }
+            if (\is_string($logs) && \strlen($logs) > $maxLogLength) {
+                $warningMessage = "[WARNING] Logs truncated. The output exceeded {$maxLogLength} characters.\n";
+                $warningLength = \strlen($warningMessage);
+                $maxContentLength = max(0, $maxLogLength - $warningLength);
+                $logs = $warningMessage . ($maxContentLength > 0 ? \substr($logs, -$maxContentLength) : '');
+            }

564-572: Mirror the guard for errors truncation

Same reasoning as logs: clamp to avoid negative lengths if limits change.

-            if (\is_string($errors) && \strlen($errors) > $maxErrorLength) {
-                $warningMessage = "[WARNING] Errors truncated. The output exceeded {$maxErrorLength} characters.\n";
-                $warningLength = \strlen($warningMessage);
-                $maxContentLength = $maxErrorLength - $warningLength;
-                $errors = $warningMessage . \substr($errors, -$maxContentLength);
-            }
+            if (\is_string($errors) && \strlen($errors) > $maxErrorLength) {
+                $warningMessage = "[WARNING] Errors truncated. The output exceeded {$maxErrorLength} characters.\n";
+                $warningLength = \strlen($warningMessage);
+                $maxContentLength = max(0, $maxErrorLength - $warningLength);
+                $errors = $warningMessage . ($maxContentLength > 0 ? \substr($errors, -$maxContentLength) : '');
+            }

553-573: Reduce duplication: extract a shared truncation helper

The same truncation logic now exists here and in app/controllers/general.php (and likely Create.php). Consider a single reusable helper, e.g., Appwrite\Functions\OutputTruncator::truncate(string $text, int $limit, string $label): string, to keep behavior consistent and minimize drift.

Example outline (outside this file):

namespace Appwrite\Functions;
final class OutputTruncator {
    public static function truncate(?string $text, int $limit, string $what): string {
        $text ??= '';
        if (!\is_string($text) || \strlen($text) <= $limit) {
            return (string)$text;
        }
        $warning = "[WARNING] {$what} truncated. The output exceeded {$limit} characters.\n";
        $remain = max(0, $limit - \strlen($warning));
        return $warning . ($remain > 0 ? \substr($text, -$remain) : '');
    }
}

Then here:

$logs = \Appwrite\Functions\OutputTruncator::truncate($executionResponse['logs'] ?? '', APP_FUNCTION_LOG_LENGTH_LIMIT, 'Logs');
$errors = \Appwrite\Functions\OutputTruncator::truncate($executionResponse['errors'] ?? '', APP_FUNCTION_ERROR_LENGTH_LIMIT, 'Errors');
app/controllers/general.php (3)

618-627: Add length clamp to avoid negative substr length for logs

Same small guard as in the worker ensures robust behavior if limits are tuned down in future.

-            if (\is_string($logs) && \strlen($logs) > $maxLogLength) {
-                $warningMessage = "[WARNING] Logs truncated. The output exceeded {$maxLogLength} characters.\n";
-                $warningLength = \strlen($warningMessage);
-                $maxContentLength = $maxLogLength - $warningLength;
-                $logs = $warningMessage . \substr($logs, -$maxContentLength);
-            }
+            if (\is_string($logs) && \strlen($logs) > $maxLogLength) {
+                $warningMessage = "[WARNING] Logs truncated. The output exceeded {$maxLogLength} characters.\n";
+                $warningLength = \strlen($warningMessage);
+                $maxContentLength = max(0, $maxLogLength - $warningLength);
+                $logs = $warningMessage . ($maxContentLength > 0 ? \substr($logs, -$maxContentLength) : '');
+            }

618-639: DRY the truncation across entry points

This file duplicates the exact logic now present in src/Appwrite/Platform/Workers/Functions.php (and likely in Modules/Functions/Http/Executions/Create.php). Extract a single utility (e.g., Appwrite\Functions\OutputTruncator::truncate) and use it here and elsewhere to avoid divergence.

If you want, I can draft a small shared helper + usage patches for all three call sites.


618-639: Optional: Expose truncation flags on execution metadata

I ran the suggested grep across PHP, TypeScript, and JavaScript files to see how clients consume the logs and errors fields. The output shows only test assertions and simple display/transport of these fields—no code appears to parse or rely on the warning text itself. This means adding boolean flags should be safe and non-breaking:

• tests/unit/Utopia/Response/Filters/V16Test.php & V17Test.php assert the presence and shape of logs/errors but don’t inspect content
• GraphQL client tests (e2e/Services/GraphQL/…Test.php) only verify the absence of an errors key and that logs is returned as an array or string

Recommendations:

  • In the execution model/controller (app/controllers/general.php), set two new boolean attributes—logsTruncated and errorsTruncated—to true when truncation occurs (omit/leave false otherwise).
  • Update the GraphQL schema (and any REST serializers) to expose these flags alongside the existing logs and errors fields.
  • Add unit/e2e tests to assert these flags are correctly set when truncation logic is exercised.

This will allow UI/API consumers to display a truncation indicator programmatically, without needing to parse the warning prefix.

tests/e2e/Services/Functions/FunctionsCustomServerTest.php (3)

2286-2317: Solid E2E for log truncation; consider asserting exact length

The assertions are good. As a small improvement, assert exact final length equals APP_FUNCTION_LOG_LENGTH_LIMIT (not just <=) to catch accidental under-truncation.

-        $this->assertLessThanOrEqual(APP_FUNCTION_LOG_LENGTH_LIMIT, strlen($logs));
+        $this->assertEquals(APP_FUNCTION_LOG_LENGTH_LIMIT, strlen($logs));

Optional: craft the log payload to be a distinct head/tail (e.g., repeated 'A' then 'Z') and assert that only the tail survives to prove tail-truncation semantics.


2319-2350: Good E2E for error truncation; mirror the exact-length assertion

Same suggestion as logs: enforce exact size to guard against regressions, and optionally assert tail semantics.

-        $this->assertLessThanOrEqual(APP_FUNCTION_ERROR_LENGTH_LIMIT, strlen($errors));
+        $this->assertEquals(APP_FUNCTION_ERROR_LENGTH_LIMIT, strlen($errors));

2286-2350: Add E2E tests for Router and Worker invocation

We’ve confirmed that truncation is implemented in all three entry points:

  • Worker path: src/Appwrite/Platform/Workers/Functions.php
  • HTTP sync (Executions Create): src/Appwrite/Platform/Modules/Functions/Http/Executions/Create.php
  • Router/domain path: app/controllers/general.php

Current tests only cover the HTTP sync path. Please add:

  • An E2E test that invokes the function via the custom router/domain endpoint (the general controller) and asserts that both logs and errors are truncated to APP_FUNCTION_LOG_LENGTH_LIMIT/APP_FUNCTION_ERROR_LENGTH_LIMIT with the [WARNING] … truncated prefix.
  • An E2E test using an async execution (omitting or setting async to true) to exercise the worker path and verify identical truncation behavior end-to-end.
📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled by default for public repositories
  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 6dc0275 and bc47ac7.

📒 Files selected for processing (8)
  • app/config/collections/projects.php (2 hunks)
  • app/controllers/general.php (1 hunks)
  • app/init/constants.php (1 hunks)
  • src/Appwrite/Platform/Modules/Functions/Http/Executions/Create.php (1 hunks)
  • src/Appwrite/Platform/Workers/Functions.php (1 hunks)
  • tests/e2e/Services/Functions/FunctionsCustomServerTest.php (1 hunks)
  • tests/resources/functions/error-truncation/index.js (1 hunks)
  • tests/resources/functions/log-truncation/index.js (1 hunks)
🧰 Additional context used
🧬 Code graph analysis (3)
tests/resources/functions/error-truncation/index.js (1)
tests/resources/functions/log-truncation/index.js (1)
  • longString (3-3)
tests/resources/functions/log-truncation/index.js (1)
tests/resources/functions/error-truncation/index.js (1)
  • longString (3-3)
tests/e2e/Services/Functions/FunctionsCustomServerTest.php (1)
tests/e2e/Services/Proxy/ProxyBase.php (3)
  • setupFunction (221-264)
  • packageFunction (283-298)
  • cleanupFunction (163-171)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
  • GitHub Check: Setup & Build Appwrite Image
  • GitHub Check: Setup & Build Appwrite Image
  • GitHub Check: Linter
  • GitHub Check: scan
🔇 Additional comments (6)
app/config/collections/projects.php (2)

1917-1937: DB limits now match code-level truncation — good alignment

Switching to APP_FUNCTION_ERROR_LENGTH_LIMIT and APP_FUNCTION_LOG_LENGTH_LIMIT keeps persistence limits and runtime truncation consistent.


1917-1937: Additional literal size => 1000000 occurrences found

I ran a repo-wide scan and uncovered further hardcoded 1,000,000-byte limits outside of the snippet under review. Please verify whether these should also be migrated to use APP_FUNCTION_ERROR_LENGTH_LIMIT / APP_FUNCTION_LOG_LENGTH_LIMIT (or a dedicated constant) or otherwise documented:

  • app/config/collections/databases.php
    attributes field (size => 1000000 at line 68)
    indexes field (size => 1000000 at line 77)

  • app/config/collections/platform.php
    – text field (size => 1000000 at line 951)
    – text field (size => 1000000 at line 1037)

  • app/config/collections/projects.php
    – original logs/errors definitions (size => 1000000 at line 1712)

Let me know if these need to be updated for consistency or if they intentionally differ.

src/Appwrite/Platform/Modules/Functions/Http/Executions/Create.php (2)

426-446: Align with product decision: truncate from the beginning, warning at the start — LGTM

Behavior matches the decision to drop oldest content and prepend a warning. Tests relying on sentinel removal ('z' missing) should pass.


484-491: No header filtering needed for synchronous executions

The unfiltered rebuild at src/Appwrite/Platform/Modules/Functions/Http/Executions/Create.php 484–491 lives inside the synchronous execution path, which—per the Execution response model (“All headers are returned if execution is created as synchronous”)—intentionally returns every header. The earlier FUNCTION_ALLOWLIST_HEADERS_RESPONSE filtering at lines 419–423 applies only to the asynchronous persistence branch, so there’s no leak in the intended flow.

src/Appwrite/Platform/Workers/Functions.php (1)

579-581: LGTM: using sanitized outputs

Persisting the sanitized $logs and $errors back into the execution document looks correct and aligns with the collection size limits.

app/controllers/general.php (1)

640-648: LGTM: persist sanitized outputs and preserve status semantics

Setting status, responseStatusCode, headers, and duration alongside the sanitized logs/errors is consistent with existing execution semantics.

$this->cleanupFunction($functionId);
}

public function testErrorTruncation(): void
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's merge these tests into one, so we dont need to do 2 deployments. they slow down CI/CD by around 10 seconds

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (4)
tests/resources/functions/log-error-truncation/index.js (2)

2-3: Minor readability: use numeric separators and named constants.

Makes intent and scale obvious without changing behavior.

Apply this diff:

-module.exports = async(context) => {
-    // Create a string that is 1000001 characters long (exceeds the 1000000 limit)
-    const longString = 'z' + 'a'.repeat(1000000);
+module.exports = async (context) => {
+    // Create a string that is 1_000_001 characters long (exceeds the 1_000_000 limit)
+    const HEAD_SENTINEL = 'z';
+    const FILL_COUNT = 1_000_000;
+    const longString = HEAD_SENTINEL + 'a'.repeat(FILL_COUNT);

2-6: Optional: strengthen tail-preservation assertion by adding a distinct tail marker.

If you want the e2e test to assert the tail is preserved (not just that the head isn't), append a short unique tail token and assert it survives truncation. This would require adjusting the PHP assertions accordingly.

Example change (Node fixture):

-    const HEAD_SENTINEL = 'z';
-    const FILL_COUNT = 1_000_000;
-    const longString = HEAD_SENTINEL + 'a'.repeat(FILL_COUNT);
+    const HEAD_SENTINEL = 'z';
+    const TAIL_SENTINEL = 'TAIL';
+    const TOTAL = 1_000_001;
+    const FILL_COUNT = TOTAL - HEAD_SENTINEL.length - TAIL_SENTINEL.length;
+    const longString = HEAD_SENTINEL + 'a'.repeat(FILL_COUNT) + TAIL_SENTINEL;

And then in the PHP test, assert that logs/errors end with 'TAIL'.

tests/e2e/Services/Functions/FunctionsCustomServerTest.php (2)

2305-2307: Add execution status assertion for completeness.

We assert 201/200 but not the execution status. Add a quick check to ensure the execution actually completed.

Apply this diff:

         $this->assertEquals(201, $execution['headers']['status-code']);
         $this->assertEquals(200, $execution['body']['responseStatusCode']);
+        $this->assertEquals('completed', $execution['body']['status']);

2285-2325: Also validate the async (worker) path without extra deployment.

The truncation logic is exercised via the synchronous path here. Adding an async execution in the same test verifies the worker code path too and guards against regressions that cause “processing” hangs for large logs, matching the original bug report.

Apply this diff near the end of the test (before cleanupFunction):

-        $this->cleanupFunction($functionId);
+        // Also verify async path (worker) truncation with no extra deployment
+        $execution = $this->createExecution($functionId, [
+            'async' => true
+        ]);
+        $this->assertEquals(202, $execution['headers']['status-code']);
+        $this->assertNotEmpty($execution['body']['$id']);
+        $executionId = $execution['body']['$id'] ?? '';
+
+        $this->assertEventually(function () use ($functionId, $executionId) {
+            $execution = $this->getExecution($functionId, $executionId);
+            $this->assertEquals(200, $execution['headers']['status-code']);
+            $this->assertEquals('completed', $execution['body']['status']);
+            $this->assertLessThanOrEqual(APP_FUNCTION_LOG_LENGTH_LIMIT, strlen($execution['body']['logs']));
+            $this->assertStringStartsWith('[WARNING] Logs truncated', $execution['body']['logs']);
+            $this->assertStringNotContainsString('z', $execution['body']['logs']);
+            $this->assertStringContainsString('a', $execution['body']['logs']);
+            $this->assertLessThanOrEqual(APP_FUNCTION_ERROR_LENGTH_LIMIT, strlen($execution['body']['errors']));
+            $this->assertStringStartsWith('[WARNING] Errors truncated', $execution['body']['errors']);
+            $this->assertStringNotContainsString('z', $execution['body']['errors']);
+            $this->assertStringContainsString('a', $execution['body']['errors']);
+        }, 10000, 500);
+
+        $this->cleanupFunction($functionId);
📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled by default for public repositories
  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between bc47ac7 and 10eed78.

📒 Files selected for processing (2)
  • tests/e2e/Services/Functions/FunctionsCustomServerTest.php (1 hunks)
  • tests/resources/functions/log-error-truncation/index.js (1 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
tests/e2e/Services/Functions/FunctionsCustomServerTest.php (1)
tests/e2e/Services/Proxy/ProxyBase.php (3)
  • setupFunction (221-264)
  • packageFunction (283-298)
  • cleanupFunction (163-171)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
  • GitHub Check: Setup & Build Appwrite Image
  • GitHub Check: Linter
  • GitHub Check: scan
🔇 Additional comments (3)
tests/resources/functions/log-error-truncation/index.js (1)

1-14: Fixture exercises head-truncation behavior correctly — LGTM

The sentinel 'z' at the head plus a 1,000,000-char tail creates a deterministic over-limit payload. Using both context.log and context.error validates logs and errors paths. Return JSON keeps execution green.

tests/e2e/Services/Functions/FunctionsCustomServerTest.php (2)

2285-2326: Good end-to-end coverage of truncation behavior.

Covers the over-limit case, asserts prefix warning, and checks head-sentinel is gone. Cleanup included.


2310-2311: Constants confirmed as defined and available at runtime

Both APP_FUNCTION_LOG_LENGTH_LIMIT and APP_FUNCTION_ERROR_LENGTH_LIMIT are declared in app/init/constants.php (lines 146–147), so they’ll be present when the E2E suite bootstraps the application. No further action needed.

@Meldiron Meldiron merged commit 36049cf into appwrite:1.7.x Aug 21, 2025
39 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

🐛 Bug Report: Execution stuck at processing

3 participants