-
-
Notifications
You must be signed in to change notification settings - Fork 9.8k
Description
Symfony version(s) affected
7.x
Description
I'd like to use the JsonStreamer component to work with an API that returns a large dataset in json (~200k items) where the array of items is a nested property. Attempting to do so results is very high memory usage. After some digging, it also appears to happen even if it's a flat array.
I've created a minimal repo to reproduce the issue for both a 1D array, as well as a nested array: https://github.com/natewiebe13/symfony-json-streamer-memory
In my tests, I'm getting about 200MB of memory usage if I load the entire JSON structure into memory and iterating. If I use halaxa/json-machine it uses about 25MB of memory, and using the JsonStreamer component, it uses 2.1GB
At this point, I'm not sure if my usage is incorrect, or if there's something that can be improved in the component/documentation.
How to reproduce
Reproducer: https://github.com/natewiebe13/symfony-json-streamer-memory
Possible Solution
No response
Additional Context
No response