Skip to content

Idea: New v3 opt-in serialization format to save space #366

@TysonAndre

Description

@TysonAndre
  • Use version \x00\x00\x00\x03 when this is stable
  • Avoid wasting space emitting key representations when array_is_list, i.e. when array keys are consecutive integers starting from 0 (have a fallback to start emitting unexpected keys with different values, if needed)
  • Allow more than one reference group to the same php value to be distinctly unserialized (e.g. $x = new stdClass; $y = $x; igbinary_serialize([&$x, &$x, &$y, &$y]);)
    (TODO: I forget the exact issue. Was that fixed already for the general case, or was the issue mixes of references and non-references)
  • Shorter representations for 1-character interned strings
  • Shorter representations for private properties of serialized class \x00$classname\x00$propname (and protected properties \x00*\x00$propname) (keys for parent classes and traits would be serialized normally)
  • Don't bother creating string ids for empty strings (EDIT: that's already part of the v2 design?) and 1-character strings that would already be interned in modern php versions (support reusing per-request in RINIT in igbinary itself if practical for php versions without ZSTR_CHAR guaranteed to exist)

Opt-in can be done on per-serializer settings, to allow any migrations (e.g. redis, memcached) to be done separately.

The unserializer should recognize v3 from the header

Related to phpredis/phpredis#2194

Updates (October 19):

Implementation is broken/incomplete and the v3 serialization format will be changed, found on the fork branch igbinary-v3-format

  • Shorter representations for very small non-negative integers (e.g. 0..7 can be X+(0..7)) is an idea I've seen in several other binary protocol formats, useful for ['key' => 1, 'x' => 0] where integers are used instead of booleans, or just from most integers being small (e.g. counts) when averaged out over common use cases
  • Still haven't implemented for serializing lists
  • Still haven't finished adding a way to represent creating a new reference group to the same php object for the general case
  • Haven't benchmarked this yet

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions