Skip to content

fix: support backup and restore of large projects - #491

Merged
tristan-mouchet merged 1 commit into
developfrom
fix/backup-restore-large-projects
Aug 7, 2026
Merged

fix: support backup and restore of large projects#491
tristan-mouchet merged 1 commit into
developfrom
fix/backup-restore-large-projects

Conversation

@tristan-mouchet

Copy link
Copy Markdown
Collaborator

Summary

Backing up or restoring a project with enough inlined assets failed with Invalid string length. The export was built as a single JSON.stringify string (and rebuilt via JSON.parse on restore), which exceeds V8's ~512MB max string length once asset files are embedded as base64. This fixes both directions so large projects can be backed up and restored, including password-encrypted backups.

Changes

  • Serialize exports as concatenated Buffer chunks (serializeExportToBuffer), stringifying the manifest, each table row, and each asset file individually — a Buffer isn't bound by the string cap. Output is byte-identical to JSON.stringify.
  • Parse imports by byte-scanning the decompressed buffer (parseExportFromBuffer), JSON.parse-ing the manifest, each row, and each file from small slices, so no large string is ever built. Safe because JSON structural characters are ASCII and don't collide with UTF-8 content.
  • Encryption/decryption is unchanged and buffer-only, so encrypted backups benefit from both fixes.
  • Existing .ycode files remain compatible (standard JSON, key-order/whitespace agnostic); invalid files still surface Invalid backup file.

Reported in #490.

Test plan

  • Back up a small project, restore it, confirm identical result
  • Back up a project with many/large assets (previously errored), confirm the .ycode downloads without Invalid string length
  • Restore that large backup and verify pages, collections, and assets are intact
  • Repeat backup + restore with a password and confirm decryption works
  • Restore a backup created by a previous version (backward compatibility)
  • Attempt to restore a corrupted/invalid file and confirm the Invalid backup file error

Building the whole export as a single JSON string via JSON.stringify
(and rebuilding it on import) hit V8's ~512MB string cap once assets
are inlined as base64, throwing "Invalid string length".

Serialize the export as concatenated Buffer chunks and parse the
import by byte-scanning the decompressed buffer, parsing the manifest,
each row, and each asset file from small slices. Output stays
byte-identical to JSON.stringify, so existing backups still restore.
@tristan-mouchet tristan-mouchet self-assigned this Aug 7, 2026
@tristan-mouchet tristan-mouchet added the Bug Something isn't working label Aug 7, 2026
@tristan-mouchet
tristan-mouchet merged commit 747139b into develop Aug 7, 2026
2 of 3 checks passed
@tristan-mouchet tristan-mouchet linked an issue Aug 7, 2026 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Backup error

1 participant