Does this not have identical behaviour to itertools.batched?

(sidenote: now that we're on Python 3.13 we can finally use this utility which makes me so happy 🤓)

cool, i did not know this existed. i added this before 3.13 :-) i think it's worth keeping until the only thing we support is 3.13+ or until we use this code exclusively for the runtime

Could we not use the original Batcher utlity to collect the batches instead of re-implementing the logic? There are quite a few edge-cases that the original batcher handles that could be useful here IMO

i.e

    batcher: Batcher[T]

    def add(self, item: T):
        with self._flush_lock:
            if self._closed:
                raise ValueError("Batcher is stopped, can no longer add items")
             
             # This call triggers the batcher's policy, meaning we should flush.
             if self.batcher.add(item): 
                self._flush_lock.notify_all()

my main reason was to not entangle the two implementations. the batching logic is ultimately relatively simple and this way it's clearer when reading the code what exactly happens.

suggestion: I think it'd be useful to know whether a call has triggered a batch, or perhaps some signal indicating that a batch has been sent. Else, how would I know whether the function has been called?

vals = []
buffer = AsyncBatcher(lambda x: vals.append(x), max_batch_size=2, max_flush_interval=1000)

for data in stream_generator():
   if not (is_triggered := buffer.add(data)):
      continue

    # The BatchHandler should have been called
    do_stuff(vals)

some signal indicating that a batch has been sent

the invocation of the handler is the signal. this is fundamentally the difference to Batcher. AsyncBatcher calls the handler for you.

in your case, i would just pass do_stuff as a handler, as it seems that's what you want. no need to collect values first.

AsyncBatcher(do_stuff, max_batch_size=2, max_flush_interval=1000)

If the handler fails, that means we lose the entire batch of data. Is this desirable?

i suppose an exception handler would be a nice mechanism to give more flexibility on what should happen with batches that fail. this is not so easy to generalize in a way that is flexible, and would introduce a lot of additional complexity that there's currently no use case for. the assumption therefore is currently that fault tolerance is the handler's responsibility.