I have a log file that is updated about 60 times per second with big lines, about 160 KB per line. I increased read chunk size from 4096 to 40960 (10 times, file tail.py, method _run_pass), beaver read new lines faster, but not enough. The lag between last log entry and line processing by beaver increases - 10 seconds delay per minute of working.
I've tested with tranport type = stdout, format: raw

Increasing number_of_consumer_processes to 2 or 4 doesn't help.
Increasing reading chunk size to 500 KB doesn't help.

It is not a disk issue (SSDin this server and some stat about IO operations in Grafana looks good).

Why beaver reading lines from log can be so slow?
Any hints how speed it up?
Can increasing discover_interval help?