We have a custom BIO registered via BIO_meth_set_write_ex() to integrate openssl with seastar's async IO. When a write on the BIO fails with EPIPE, we return 0 from our bwrite callback, following the BIO_meth_new documentation:

The parameters for the function have the same meaning as for BIO_write_ex() and it must return values as described for BIO_write_ex().

BIO_read documents BIO_write_ex() as returning 1 on success and 0 on failure:

BIO_write_ex() returns 1 if no error was encountered writing data, 0 otherwise. Requesting to write 0 bytes is not considered an error.
BIO_write() returns -2 if the "write" operation is not implemented by the BIO or -1 on other errors. Otherwise it returns the number of bytes written. This may be 0 if the BIO b is NULL or dlen <= 0.

When our bwrite returns 0 without setting BIO_should_retry, we observe that SSL_write_ex incorrectly reports success. Here is a concrete timeline:

1. TLS connection established, client writes application data
2. Peer closes the connection
3. SSL_write_ex encrypts plaintext into a TLS record and calls our BIO to send it
4. Our bwrite calls sendmsg, gets EPIPE, calls ERR_raise_data(ERR_LIB_SYS, EPIPE), returns 0
5. SSL_write_ex returns 1 with bytes_written > 0
6. Errors from step 4 are left on the per-thread error queue

Tracing through the code (3.5.5), bio_write_intern (crypto/bio/bio_lib.c) passes our 0 return through unchanged, which means that a call to BIO_write in this scenario returns 0:

Then in tls_retry_write_records (ssl/record/methods/tls_common.c:1934), this scenario lands on the success case on L1887:

BIO_write returning 0 passes the if (i >= 0) check and is classified as success.

The read side appears to have had a similar issue, fixed in commit be42447 ("Fix the converters between the old and new BIO_read functions to handle end-of-file state properly").

Questions:

1. Should our bwrite callback return 0 (per the documented BIO_write_ex contract) or -1 on a non-retryable write error?
2. If 0 is correct, should tls_retry_write_records/bio_write_intern be updated to handle this case as a failure?