I mentioned this in another comment that I believe got resolved somewhere... this really does not belong in encoding.c++. Any improvement we make should be usable by the entire runtime. Let's move the optimization into jsg::JsString so that all of the APIs benefit.

I'll do this last. I want to make sure we don't regress with your recommendations first.

This is still creating the intermediate on-heap allocation that can trigger GC. Let's have the intermediate allocation (the one that writeOneByte writes into) be an off-heap (kj::SmallArray<kj::byte, 4096> buffer(length)) allocation. In both cases we end up copying into the on-heap destination buffer but we avoid the wasted additional on-heap allocation in the second case.

With this, on happy path (length === utf8_length), we do a memcpy here. I'm not sure if this is better. @erikcorry what do you think?

if (utf8_length == length) {
      // ASCII fast path: copy to on-heap BackingStore once
      backingStore = js.allocBackingStore(length, jsg::Lock::AllocOption::UNINITIALIZED);
      memcpy(backingStore->Data(), buffer.begin(), length);
      auto array = v8::Uint8Array::New(v8::ArrayBuffer::New(js.v8Isolate, backingStore), 0, length);
      return jsg::JsUint8Array(array);
    }
    ```

It's not about the memcpy... on line 601 you allocate a v8::BackingStore that receives the results of the WriteOneByte, then here in this branch you create a second one that receives the results of the latin1-to-utf8 encoding. What I'm saying here is that you should only create one v8::BackingStore that represents the final result. The WriteOneByte should write into an off-v8-heap allocation (e.g. a kj::Array). This way you greatly lessen the risk of triggering a GC in the middle.

Given that ValueView forces flattening, which we need to avoid... let's instead use WriteV2 to write to an off-heap allocation (e.g. kj::SmallArray<uint16_t, 4096> buffer(length)) to avoid the additional GC pressure caused by flattening. In this case, it's perfectly fine to give up some CPU for the copy in order to avoid the higher cost of flattening.

FWIW, while this may not create JS objects it can and will trigger GC.. so let's try to do this (allocating the on-heap result buffer) once per call. All other allocations should be off-heap.

You might be able to simplify things further by combining this and the utf8LengthFromInvalidUtf16 function into one. Return the ut8 length, but also replace the invalid code units with the replacement char in-place, then apply the encoding which, if the scan was correct, should never have to worry about the unpaired surrogates. It would save an additional linear scan.

This won't work because we need to have a separate copy in order to update it. The original input is a const char*. we can't fix and calculate length on 1 linear scan.

The original input does not need to be const tho, that's under our control. If you modify it like this then you can simplify the next transcoding step since you'll know the unpaired surrogates have already been replaced and you don't need to handle them again:

  size_t utf8LengthFromInvalidUtf16(kj::ArrayPtr<char16_t> input) {
    size_t utf8Length = 0;
    bool pendingSurrogate = false;

    for (size_t i = 0; i < input.size(); i++) {
      char16_t c = input[i];

      if (pendingSurrogate) {
        if (isTrailSurrogate(c)) {
          // Valid surrogate pair = 4 bytes in UTF-8
          utf8Length += 4;
          pendingSurrogate = false;
        } else {
          // Unpaired lead surrogate at position i-1 - replace it
          input[i - 1] = 0xFFFD;
          utf8Length += 3;  // U+FFFD is always 3 bytes in UTF-8

          // Now process current character (which is not a trail surrogate)
          if (isLeadSurrogate(c)) {
            // Current char becomes the new pending lead surrogate
            pendingSurrogate = true;
          } else {
            // Regular character
            utf8Length += utf8BytesForCodeUnit(c);
            pendingSurrogate = false;
          }
        }
      } else if (isLeadSurrogate(c)) {
        pendingSurrogate = true;
      } else if (isTrailSurrogate(c)) {
        // Unpaired trail surrogate - replace it
        input[i] = 0xFFFD;
        utf8Length += 3;  // U+FFFD is always 3 bytes in UTF-8
      } else {
        utf8Length += utf8BytesForCodeUnit(c);
      }
    }

    if (pendingSurrogate) {
      // Trailing unpaired lead surrogate - replace it
      input[input.size() - 1] = 0xFFFD;
      utf8Length += 3;  // U+FFFD is always 3 bytes in UTF-8
    }

    return utf8Length;
  }

Nit: isOneByte is super cheap. The KJ_WARN_UNUSED_RESULT is probably unnecessary.

I'm really not a big fan of adding the init_mode option here. My preference would be to stay on the safer side and always have these allocations be zero-initialized as it makes it significantly less likely that we'll mess up and introduce a critical memory vuln. If we add this we need to be extra careful about how we use this.

Simply because I'm paranoid, since you're adding the uninitialized allocation, we should probably have an assert in here that verifies that writeOneByte completely overwrote the backing buffer.

to be safe, a KJ_DASSERT that verifies that utf8_length > length would be good here.

Perhaps link to simdutf/simdutf#688 in comment?

(Not an actual suggestion)
Just noting that this whole logic should be identical to the following:

Not sure which is more performant (or if that is significant at all)

Or perhaps some form of:

size_t utf8LengthFromInvalidUtf16(kj::ArrayPtr<const char16_t> input) {
  size_t utf8Length = 0;
  bool pendingSurrogate = false;

  for (size_t i = 0; i < input.size(); i++) {
    char16_t c = input[i];

    if (c < 0xD800 || c > 0xDFFF) {
      utf8Length += utf8BytesForCodeUnit(c);
      pendingSurrogate = false;
    } else if (c < 0xDC00) {
      // Lead surrogate
      utf8Length += 3;
      pendingSurrogate = true;
    } else {
      // Trail surrogate
      utf8Length += pendingSurrogate ? 1 : 3;
      pendingSurrogate = false;
    }
  }

  return utf8Length;
}