Skip to content

optimise simdutf for short strings#926

Open
sleepingeight wants to merge 1 commit intosimdutf:masterfrom
sleepingeight:surya/sso
Open

optimise simdutf for short strings#926
sleepingeight wants to merge 1 commit intosimdutf:masterfrom
sleepingeight:surya/sso

Conversation

@sleepingeight
Copy link
Contributor

Related to #925

Started with a straightforward implementation, modifying the public API's defination in this way,

simdutf_warn_unused bool validate_utf8(const char *buf, size_t len) noexcept {
  if (len < SIMDUTF_SHORT_INPUT_THRESH) {
    return scalar::utf8::validate(buf, len);
  }
  return get_default_implementation()->validate_utf8(buf, len);
}

Which was before,

simdutf/src/implementation.cpp

Lines 1429 to 1431 in bcfbae9

simdutf_warn_unused bool validate_utf8(const char *buf, size_t len) noexcept {
return get_default_implementation()->validate_utf8(buf, len);
}

Using the benchmark utiltiy, got 0 improvements. But when benchmarked using google-benchmark for input sizes from 1 to 15, these are the results for validate_utf8 procedure -

Old:

Running ./bench_utf8
Run on (10 X 24 MHz CPU s)
CPU Caches:
  L1 Data 64 KiB
  L1 Instruction 128 KiB
  L2 Unified 4096 KiB (x10)
Load Average: 2.50, 2.81, 2.65
------------------------------------------------------------------------------------
Benchmark                          Time             CPU   Iterations UserCounters...
------------------------------------------------------------------------------------
BM_validate_utf8_small/1      840503 ns       839682 ns          755 bytes_per_second=93.0412Mi/s
BM_validate_utf8_small/2      846647 ns       842714 ns          830 bytes_per_second=185.41Mi/s
BM_validate_utf8_small/3      951668 ns       948250 ns          741 bytes_per_second=247.16Mi/s
BM_validate_utf8_small/4      944975 ns       942727 ns          743 bytes_per_second=331.473Mi/s
BM_validate_utf8_small/5      942938 ns       940954 ns          735 bytes_per_second=415.117Mi/s
BM_validate_utf8_small/6      841230 ns       839293 ns          833 bytes_per_second=558.472Mi/s
BM_validate_utf8_small/7      891388 ns       889915 ns          786 bytes_per_second=614.48Mi/s
BM_validate_utf8_small/8      663682 ns       662348 ns         1054 bytes_per_second=943.532Mi/s
BM_validate_utf8_small/9      841233 ns       839826 ns          834 bytes_per_second=837.145Mi/s
BM_validate_utf8_small/10     849720 ns       846173 ns          833 bytes_per_second=923.173Mi/s
BM_validate_utf8_small/11     957664 ns       949415 ns          731 bytes_per_second=905.053Mi/s
BM_validate_utf8_small/12     962141 ns       953108 ns          741 bytes_per_second=983.492Mi/s
BM_validate_utf8_small/13     959022 ns       948287 ns          743 bytes_per_second=1.04576Gi/s
BM_validate_utf8_small/14     848028 ns       843355 ns          833 bytes_per_second=1.26631Gi/s
BM_validate_utf8_small/15     902333 ns       893884 ns          784 bytes_per_second=1.28005Gi/s

New:

Running ./bench_utf8
Run on (10 X 24 MHz CPU s)
CPU Caches:
  L1 Data 64 KiB
  L1 Instruction 128 KiB
  L2 Unified 4096 KiB (x10)
Load Average: 1.71, 2.77, 2.64
------------------------------------------------------------------------------------
Benchmark                          Time             CPU   Iterations UserCounters...
------------------------------------------------------------------------------------
BM_validate_utf8_small/1      102941 ns       102369 ns         6844 bytes_per_second=763.171Mi/s
BM_validate_utf8_small/2      178957 ns       178164 ns         3880 bytes_per_second=876.988Mi/s
BM_validate_utf8_small/3      206253 ns       205535 ns         3313 bytes_per_second=1.11356Gi/s
BM_validate_utf8_small/4      229331 ns       228903 ns         3060 bytes_per_second=1.33316Gi/s
BM_validate_utf8_small/5      257787 ns       255509 ns         2751 bytes_per_second=1.49291Gi/s
BM_validate_utf8_small/6      290960 ns       284772 ns         2488 bytes_per_second=1.60738Gi/s
BM_validate_utf8_small/7      306497 ns       305387 ns         2265 bytes_per_second=1.74866Gi/s
BM_validate_utf8_small/8      331329 ns       330753 ns         2114 bytes_per_second=1.84518Gi/s
BM_validate_utf8_small/9      357335 ns       356322 ns         1968 bytes_per_second=1.92685Gi/s
BM_validate_utf8_small/10     381890 ns       381156 ns         1833 bytes_per_second=2.00143Gi/s
BM_validate_utf8_small/11     412080 ns       410937 ns         1722 bytes_per_second=2.042Gi/s
BM_validate_utf8_small/12     433322 ns       432472 ns         1620 bytes_per_second=2.11668Gi/s
BM_validate_utf8_small/13     458809 ns       457936 ns         1526 bytes_per_second=2.16553Gi/s
BM_validate_utf8_small/14     483932 ns       483006 ns         1448 bytes_per_second=2.21104Gi/s
BM_validate_utf8_small/15     509209 ns       508381 ns         1367 bytes_per_second=2.2507Gi/s

Seeing a greater than 2x improvement.

@sleepingeight
Copy link
Contributor Author

sleepingeight commented Jan 23, 2026

Benchmark for reference -

static std::vector<char> make_buffer() {
  constexpr size_t buffer_size = 800 * 1024;
  std::vector<char> buffer(buffer_size);

  std::mt19937 rng(12345);
  std::uniform_int_distribution<int> dist(0x20, 0x7E);
  for (size_t i = 0; i < buffer_size; i++) {
    buffer[i] = static_cast<char>(dist(rng));
  }
  return buffer;
}

static void BM_validate_utf8_small(benchmark::State& state) {
  static const std::vector<char> buffer = make_buffer();
  const size_t len = state.range(0);
  const size_t iterations = buffer.size() - len + 1;

  for (auto _ : state) {
    for (size_t i = 0; i < iterations; i++) {
      benchmark::DoNotOptimize(
          simdutf::validate_utf8(buffer.data() + i, len));
    }
  }

  // Report bytes processed per iteration
  state.SetBytesProcessed(
      int64_t(state.iterations()) * int64_t(iterations) * int64_t(len));
}

BENCHMARK(BM_validate_utf8_small)->DenseRange(1, 15);

BENCHMARK_MAIN();

cc: @lemire

#endif

#ifndef SIMDUTF_SHORT_INPUT_THRESH
#define SIMDUTF_SHORT_INPUT_THRESH 16
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how was this picked? it seems lower than both boundaries linked in #925

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When processing short inputs (e.g., strings smaller than 16 bytes),..

Sorry for that, I was considering @lemire's statement in the issue, will look into the referenced links. Thanks for pointing out.

Copy link
Contributor Author

@sleepingeight sleepingeight Jan 23, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

result2

from the above graph, it seems that with this approach we could fix the boundary around 160 to ensure guaranteed speedups.

@lemire
Copy link
Member

lemire commented Jan 23, 2026

@sleepingeight The design is clean, but I am not sure that this will work. The problem is that the function call is still not inline-able and most of our functions already branch on small inputs (often by necessity!!!) so according to my mental model, the branch you added in the source function buys you little and might even cost you a bit.

simdutf_warn_unused bool validate_utf8(const char *buf, size_t len) noexcept {
  if (len < SIMDUTF_SHORT_INPUT_THRESH) {
    return scalar::utf8::validate(buf, len);
  }
/*
 * You may think that the  get_default_implementation() is costly, but it is not. On many systems where
 * runtime dispatching is unneeded (such as ARM), this is free. On x64, it is essentially just a guarded
 * pointer access. If you run profiling data, I don't think it will come up.
 * */
  return get_default_implementation()->validate_utf8(buf, len);
}

UPDATE (see below): In at least some cases, I am wrong and the approach pursued here appears to be justified.

@lemire
Copy link
Member

lemire commented Jan 23, 2026

I created a tool to help us with this problem, please go review:

#927

My tool seems to (at least partially) validate your approach actually.

I added the following line...

  if (len < 32) {
    return scalar::utf8::validate(buf, len);
  }

On a macbook, I get...

Size       Total Time (ns)    Time/Byte (ns)       Err% Cycles/Byte     Ins/Byte        Ins/Cycle      
----------------------------------------------------------------------------------------------------------------
1          8.6                  8.6                   0  37.8            203.3             5.4            
2          9.4                  4.7                   1  20.3            109.7             5.4            
3          10.1                 3.4                   3  14.4            78.4              5.4            
4          10.4                 2.6                   2  11.1            62.8              5.6            
5          11.0                 2.2                   3  9.4             53.5              5.7            
6          11.5                 1.9                   2  8.2             47.2              5.8            
7          12.7                 1.8                   2  7.7             42.8              5.6            
8          13.2                 1.7                   3  7.0             39.4              5.6            
9          13.7                 1.5                   2  6.4             36.8              5.7            
10         14.3                 1.4                   2  6.0             34.7              5.8            
11         14.9                 1.4                   2  5.7             33.0              5.8            
12         15.5                 1.3                   3  5.4             31.6              5.8            
13         16.1                 1.2                   3  5.2             30.4              5.8            
14         16.7                 1.2                   3  5.0             29.4              5.9            
15         17.1                 1.1                   1  4.7             28.5              6.0            
16         9.0                  0.6                   0  2.4             13.3              5.5            
17         9.9                  0.6                   1  2.5             14.0              5.6            
18         10.5                 0.6                   2  2.5             14.1              5.6            
19         11.3                 0.6                   2  2.5             14.2              5.6            
20         11.8                 0.6                   2  2.5             14.3              5.7            
21         12.6                 0.6                   3  2.5             14.4              5.7            
22         13.1                 0.6                   2  2.5             14.5              5.7            
23         13.9                 0.6                   2  2.5             14.5              5.7            
24         14.3                 0.6                   2  2.5             14.6              5.8            
25         14.9                 0.6                   2  2.5             14.7              5.9            
26         15.5                 0.6                   2  2.5             14.7              5.9            
27         16.1                 0.6                   3  2.5             14.8              5.9            
28         16.8                 0.6                   3  2.5             14.8              5.9            
29         17.7                 0.6                   3  2.5             14.8              5.8            
30         18.0                 0.6                   3  2.5             14.9              6.0            
31         18.8                 0.6                   3  2.5             14.9              5.9            
32         153.9                4.8                   3  19.4            48.3              2.5            
33         151.4                4.6                   8  18.7            47.0              2.5            
34         154.9                4.6                   8  18.4            45.7              2.5            
35         153.8                4.4                   9  17.7            44.5              2.5            
36         155.9                4.3                   8  17.4            43.4              2.5            
37         153.5                4.1                   7  16.9            42.3              2.5            
38         153.4                4.0                   8  16.4            41.3              2.5       

So, contrary to what I wrote above, in at least one case, your approach helps tremendously.

This is good news because your approach is so clean and simple.

@lemire
Copy link
Member

lemire commented Jan 23, 2026

We will need to test this on different systems. :-)

@sleepingeight
Copy link
Contributor Author

sleepingeight commented Jan 24, 2026

Thank you for the benchmark. Are the results shown above for the new code or the old? Asking since I received very good speedups for small inputs on Macbook M1 Pro, also I did not get a big jump for input sizes > 32.

Benchmarks on Macbook M1 Pro (arm),
Old -

Size       Total Time (ns)    Time/Byte (ns)       Err% Cycles/Byte     Ins/Byte        Ins/Cycle      
----------------------------------------------------------------------------------------------------------------
1          11.3                 11.3                  0  39.7            183.1             4.6            
2          11.2                 5.6                   0  19.8            92.1              4.6            
3          12.4                 4.1                   0  14.5            61.7              4.2            
4          12.5                 3.1                   0  10.9            46.5              4.3            
5          12.5                 2.5                   0  8.8             37.4              4.3            
6          11.2                 1.9                   0  6.6             31.4              4.7            
7          11.8                 1.7                   0  5.9             27.0              4.5            
8          9.0                  1.1                   0  4.1             20.3              5.0            
9          11.2                 1.2                   0  4.4             20.5              4.6            
10         11.3                 1.1                   0  4.0             18.5              4.6            
11         12.5                 1.1                   0  4.0             16.9              4.3            
12         12.5                 1.0                   0  3.6             15.6              4.3            
13         12.5                 1.0                   0  3.4             14.5              4.3            
14         11.2                 0.8                   0  2.8             13.5              4.8            
15         11.9                 0.8                   0  2.8             12.7              4.6            
16         9.7                  0.6                   0  2.2             10.2              4.7            
17         12.5                 0.7                   0  2.6             10.9              4.2            
18         11.8                 0.7                   0  2.3             10.3              4.5            
19         13.1                 0.7                   0  2.4             9.9               4.1            
20         13.1                 0.7                   0  2.3             9.4               4.1            
21         13.1                 0.6                   0  2.2             9.0               4.1            
22         11.9                 0.5                   0  1.9             8.6               4.6            
23         12.5                 0.5                   0  1.9             8.3               4.4            
24         9.7                  0.4                   0  1.4             6.8               4.7            
25         12.5                 0.5                   0  1.7             7.4               4.3            
26         11.8                 0.5                   0  1.6             7.2               4.5            
27         13.1                 0.5                   0  1.7             7.0               4.1            
28         13.1                 0.5                   0  1.6             6.8               4.1            
29         13.1                 0.5                   0  1.6             6.6               4.2            
30         11.8                 0.4                   0  1.4             6.4               4.6            
31         12.5                 0.4                   0  1.4             6.2               4.4            
32         9.7                  0.3                   0  1.1             5.2               4.8            
33         12.5                 0.4                   0  1.3             5.7               4.3            
34         11.9                 0.3                   0  1.2             5.5               4.5            
35         13.1                 0.4                   0  1.3             5.4               4.1     

New -

Size       Total Time (ns)    Time/Byte (ns)       Err% Cycles/Byte     Ins/Byte        Ins/Cycle      
----------------------------------------------------------------------------------------------------------------
1          1.9                  1.9                   0  9.6             54.1              5.6            
2          2.2                  1.1                   0  5.3             30.6              5.8            
3          3.1                  1.0                   0  4.5             22.7              5.0            
4          3.5                  0.9                   0  3.7             18.8              5.1            
5          3.8                  0.8                   0  3.2             16.4              5.2            
6          4.1                  0.7                   0  2.8             14.9              5.3            
7          4.5                  0.6                   0  2.6             13.7              5.3            
8          4.8                  0.6                   0  2.4             12.9              5.3            
9          5.0                  0.6                   0  2.3             12.2              5.4            
10         5.3                  0.5                   0  2.1             11.7              5.5            
11         5.7                  0.5                   0  2.0             11.3              5.7            
12         9.1                  0.8                   2  2.7             10.9              4.0            
13         6.4                  0.5                   0  1.9             10.6              5.7            
14         6.6                  0.5                   0  1.8             10.4              5.9            
15         6.9                  0.5                   0  1.7             10.1              5.9            
16         2.2                  0.1                   0  0.7             3.4               5.1            
17         2.8                  0.2                   0  0.7             3.8               5.1            
18         3.2                  0.2                   0  0.8             4.0               5.3            
19         4.1                  0.2                   0  0.9             4.2               4.7            
20         4.4                  0.2                   0  0.9             4.3               4.8            
21         4.8                  0.2                   0  0.9             4.4               5.0            
22         5.0                  0.2                   0  0.9             4.6               5.1            
23         5.3                  0.2                   0  0.9             4.7               5.2            
24         5.7                  0.2                   0  0.9             4.8               5.3            
25         6.0                  0.2                   0  0.9             4.8               5.3            
26         6.3                  0.2                   0  0.9             4.9               5.4            
27         6.7                  0.2                   0  0.9             5.0               5.5            
28         6.9                  0.2                   0  0.9             5.1               5.5            
29         7.2                  0.2                   0  0.9             5.1               5.6            
30         7.5                  0.2                   0  0.9             5.2               5.6            
31         7.8                  0.3                   0  0.9             5.3               5.7            
32         9.7                  0.3                   0  1.1             5.2               4.8            
33         12.5                 0.4                   0  1.3             5.7               4.3            
34         11.8                 0.3                   0  1.2             5.6               4.6            
35         13.1                 0.4                   0  1.3             5.5               4.2            

Will also test this on a x86 machine asap.

@sleepingeight
Copy link
Contributor Author

Results on Intel(R) Core i9-14900HX (Haswell, x86_64) -

Old -

Size       Total Time (ns)    Time/Byte (ns)       Err% Cycles/Byte     Ins/Byte        Ins/Cycle      
----------------------------------------------------------------------------------------------------------------
1          11.2                 11.2                  0  42.3            114.2             2.7            
2          11.4                 5.7                   0  21.7            58.1              2.7            
3          11.4                 3.8                   0  14.5            38.7              2.7            
4          11.4                 2.8                   0  10.8            28.3              2.6            
5          11.4                 2.3                   0  8.7             22.6              2.6            
6          11.4                 1.9                   0  7.2             18.9              2.6            
7          11.4                 1.6                   0  6.2             16.2              2.6            
8          11.5                 1.4                   0  5.4             13.9              2.6            
9          11.4                 1.3                   0  4.8             12.4              2.6            
10         11.4                 1.1                   0  4.3             11.1              2.6            
11         11.4                 1.0                   0  3.9             10.1              2.6            
12         11.4                 0.9                   0  3.6             9.3               2.6            
13         11.4                 0.9                   0  3.3             8.6               2.6            
14         11.4                 0.8                   0  3.1             7.9               2.6            
15         11.4                 0.8                   0  2.9             7.4               2.6            
16         10.9                 0.7                   0  2.6             6.8               2.6            
17         11.1                 0.7                   0  2.5             6.4               2.6            
18         11.1                 0.6                   0  2.4             6.1               2.6            
19         11.1                 0.6                   0  2.2             5.7               2.6            
20         11.1                 0.6                   0  2.1             5.5               2.6            
21         11.1                 0.5                   0  2.0             5.2               2.6            
22         11.1                 0.5                   0  1.9             5.0               2.6            
23         11.1                 0.5                   0  1.8             4.7               2.6            
24         11.1                 0.5                   0  1.8             4.6               2.6            
25         11.1                 0.4                   0  1.7             4.4               2.6            
26         11.1                 0.4                   0  1.6             4.2               2.6            
27         11.1                 0.4                   0  1.6             4.0               2.6            
28         11.1                 0.4                   0  1.5             3.9               2.6            
29         11.1                 0.4                   0  1.5             3.8               2.6            
30         11.1                 0.4                   0  1.4             3.6               2.6            
31         11.1                 0.4                   0  1.4             3.5               2.6            
32         6.4                  0.2                   1  0.8             3.4               4.5            
33         11.7                 0.4                   0  1.3             3.3               2.5            
34         11.7                 0.3                   0  1.3             3.2               2.5            
35         11.7                 0.3                   0  1.3             3.1               2.5            

New -

Size       Total Time (ns)    Time/Byte (ns)       Err% Cycles/Byte     Ins/Byte        Ins/Cycle      
----------------------------------------------------------------------------------------------------------------
1          2.7                  2.7                   0  10.3            34.2              3.3            
2          3.2                  1.6                   0  6.2             21.1              3.4            
3          4.0                  1.3                   0  5.1             16.7              3.3            
4          5.0                  1.3                   1  4.8             14.6              3.0            
5          5.0                  1.0                   0  3.9             13.2              3.4            
6          5.6                  0.9                   0  3.5             12.4              3.5            
7          6.9                  1.0                   0  3.8             11.7              3.1            
8          7.1                  0.9                   1  3.4             11.3              3.3            
9          7.3                  0.8                   2  3.1             10.9              3.5            
10         6.6                  0.7                   0  2.5             10.6              4.2            
11         6.5                  0.6                   1  2.2             10.4              4.6            
12         7.2                  0.6                   1  2.3             10.2              4.4            
13         7.8                  0.6                   0  2.3             10.0              4.4            
14         8.6                  0.6                   0  2.3             9.9               4.2            
15         9.5                  0.6                   0  2.4             9.7               4.0            
16         2.1                  0.1                   0  0.5             1.9               3.6            
17         5.0                  0.3                   1  1.1             2.6               2.3            
18         5.5                  0.3                   0  1.2             2.9               2.5            
19         6.1                  0.3                   1  1.2             3.2               2.6            
20         6.5                  0.3                   0  1.2             3.4               2.7            
21         7.1                  0.3                   1  1.3             3.6               2.8            
22         7.4                  0.3                   0  1.3             3.8               3.0            
23         7.3                  0.3                   0  1.2             4.0               3.3            
24         7.8                  0.3                   1  1.2             4.2               3.4            
25         8.2                  0.3                   2  1.3             4.3               3.5            
26         8.6                  0.3                   2  1.3             4.5               3.6            
27         8.8                  0.3                   2  1.2             4.6               3.7            
28         9.1                  0.3                   1  1.2             4.7               3.8            
29         9.7                  0.3                   0  1.3             4.8               3.8            
30         11.1                 0.4                   2  1.4             4.9               3.5            
31         11.4                 0.4                   1  1.4             5.0               3.6            
32         6.4                  0.2                   0  0.8             3.5               4.6            
33         10.1                 0.3                   0  1.2             3.4               2.9            
34         10.1                 0.3                   0  1.1             3.3               2.9            
35         10.1                 0.3                   0  1.1             3.2               2.9            

Performing better than the old code but not as performant as on macbook m1 pro (above results).

@sleepingeight
Copy link
Contributor Author

@lemire, I've tested another approach by overloading the function get_default_implementation as below

inline const implementation *get_default_implementation(size_t len) {
  const implementation* opt[2] = {internal::get_fallback_singleton(), internal::get_single_implementation()};
  return opt[(len > 32)];
}

I've modified macros at other places so that the above overload works. So, the validate_utf8 procedure becomes

simdutf_warn_unused bool validate_utf8(const char *buf, size_t len) noexcept {
  return get_default_implementation(len)->validate_utf8(buf, len);
}

Performance of this is less than the approach above but greater than the original. Few numbers for ref -

Benchmarks on Macbook M1 Pro (arm)

Size       Total Time (ns)    Time/Byte (ns)       Err% Cycles/Byte     Ins/Byte        Ins/Cycle      
----------------------------------------------------------------------------------------------------------------
1          2.9                  2.9                   0  12.7            82.1              6.5            
2          3.2                  1.6                   0  6.8             44.6              6.5            
3          4.1                  1.4                   0  5.5             32.0              5.8            
4          4.5                  1.1                   0  4.5             25.8              5.8            
5          4.8                  1.0                   0  3.8             22.0              5.8            
6          5.1                  0.8                   0  3.3             19.5              5.9            
7          5.4                  0.8                   0  3.0             17.7              6.0            
8          5.7                  0.7                   0  2.7             16.4              6.0            
9          6.0                  0.7                   0  2.5             15.3              6.0            
10         6.3                  0.6                   0  2.4             14.5              6.1            
11         6.7                  0.6                   0  2.3             13.8              6.1            
12         7.0                  0.6                   0  2.2             13.3              6.1            

Which approach do you recommend?

@lemire
Copy link
Member

lemire commented Jan 26, 2026

@sleepingeight Your current PR is likely the right approach (although I was skeptical at first).

I've modified macros at other places so that the above overload works. So, the validate_utf8 procedure becomes

We probably don't want to do something so complicated.

Are the results shown above for the new code or the old?

Neither. I manually modified the code from our main branch and got these results.

We need to test a lot of functions on a lot of different systems.

The challenge is that we need to make sure that it works well for many systems. This will require data.

@lemire
Copy link
Member

lemire commented Feb 1, 2026

So I don't think we want to go so broad without careful analysis.

I think we need to do a case-by-case analysis using the new benchmark tools #927

@sleepingeight
Copy link
Contributor Author

Do you have any suggestion in mind on how to take this forward? I can test things on arm and haswell processors.

@lemire
Copy link
Member

lemire commented Feb 2, 2026

Do you have any suggestion in mind on how to take this forward? I can test things on arm and haswell processors.

I think that we want a careful analysis that can identify a case where our performance is less than stellar. And then we want to clearly document a benefit. In other words, we want more of a scalpel. It is just as important for us to improve the performance than to avoid regressions. So we need to be careful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants