haswell: faster UTF-32 to UTF-16 - #738
Conversation
|
|
||
| while (end - buf >= std::ptrdiff_t(8 + safety_margin)) { | ||
| const __m256i in = _mm256_loadu_si256((__m256i *)buf); | ||
| while (end - buf >= std::ptrdiff_t(16 + safety_margin)) { |
There was a problem hiding this comment.
Sorry for the basic question - trying to understand why we need (16 + 12) * 4 which is 112 bytes of input available for this loop.
As per my understanding the loop reads 32 bytes (8 words) and this could expand to 16 utf16 halfwords. Considering that the second half of vector can write out of bounds upto 8 bytes (4 halfwords), wouldn't it be enough for safety_margin to be 4 to guarantee further 8 bytes of write later by scalar code? So wouldn't it be sufficient for the loop check to be something like end - buf >= std::ptrdiff_t(12)?
There was a problem hiding this comment.
It's a mistake, it should still be 8 + safety_margin. I tried unrolling the loop twice, and forgot to revert this change.
There was a problem hiding this comment.
Should safety_margin still be 12 (48 bytes)? What is the reason for this large margin?
There was a problem hiding this comment.
Hm, that's a good question. I need to recheck this.
2d7bbd5 to
1913a81
Compare
Add vectorized code for surrogate pair generation. This is a port of the westmere solution.
1913a81 to
f9945c1
Compare
|
@WojciechMula Is this a PR we should merge? |
@lemire not yet, I need to check this margin value, which @tantei3 pointed out |
|
@WojciechMula |
TBH I don't remember, probably my omission. It should be vectorized too. |
3 participants
Add vectorized code for surrogate pair generation. This is a port of the westmere solution.
Ryzen
Skylake-X
IceLake