Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions build_config/ci/gcc-clang.rb
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@
conf.enable_test
end

MRuby::Build.new('ascii-case') do |conf|
MRuby::Build.new('ascii-ctype') do |conf|
conf.toolchain

# The one build here that indexes by character and converts case by ASCII.
Expand All @@ -70,7 +70,7 @@
# mruby-regexp/test/ascii_case.rb skips its assertions there.
# Tests only, for the reason byte-string gives above.
conf.gembox 'full-core'
conf.cc.defines << 'MRB_USE_ASCII_CASE'
conf.cc.defines << 'MRB_USE_ASCII_CTYPE'

conf.enable_test
end
20 changes: 10 additions & 10 deletions doc/guides/language.md
Original file line number Diff line number Diff line change
Expand Up @@ -416,16 +416,16 @@ differently on 32-bit or NaN boxing configurations.

Key compile-time macros that affect language behavior:

| Macro | Effect |
| -------------------- | ---------------------------------- |
| `MRB_NO_FLOAT` | Remove all float support |
| `MRB_USE_FLOAT32` | Use 32-bit float instead of double |
| `MRB_UTF8_STRING` | UTF-8 strings and Unicode case |
| `MRB_USE_ASCII_CASE` | Keep UTF-8, convert case by ASCII |
| `MRB_INT32` | Force 32-bit integer |
| `MRB_INT64` | Force 64-bit integer |
| `MRB_STR_LENGTH_MAX` | Max string length (default 1MB) |
| `MRB_ARY_LENGTH_MAX` | Max array length (default 2^17) |
| Macro | Effect |
| --------------------- | ---------------------------------- |
| `MRB_NO_FLOAT` | Remove all float support |
| `MRB_USE_FLOAT32` | Use 32-bit float instead of double |
| `MRB_UTF8_STRING` | UTF-8 strings and Unicode case |
| `MRB_USE_ASCII_CTYPE` | Keep UTF-8, convert case by ASCII |
| `MRB_INT32` | Force 32-bit integer |
| `MRB_INT64` | Force 64-bit integer |
| `MRB_STR_LENGTH_MAX` | Max string length (default 1MB) |
| `MRB_ARY_LENGTH_MAX` | Max array length (default 2^17) |

See [mrbconf.md](mrbconf.md) for the complete list of configuration
macros.
17 changes: 9 additions & 8 deletions doc/guides/mrbconf.md
Original file line number Diff line number Diff line change
Expand Up @@ -226,18 +226,19 @@ end
pairs with one other. Without this macro it folds ASCII letters, and a
pattern holding a character that needs one of the Unicode foldings raises
`RegexpError` rather than answering as if the character had no case.
- `MRB_USE_ASCII_CASE` narrows the case half back to ASCII, taking the refusal
- `MRB_USE_ASCII_CTYPE` narrows the case half back to ASCII, taking the refusal
with it and leaving the indexing.
- If it isn't defined, they only support the US-ASCII encoding.

`MRB_USE_ASCII_CASE`
`MRB_USE_ASCII_CTYPE`

- Narrows case conversion back to ASCII, so `String#downcase`, `#upcase`,
`#capitalize`, `#swapcase` and `#casecmp?` answer for `'A'` to `'Z'` and hand
every other character back as it stands.
- Drops the Unicode case table the build would otherwise carry. That is what
the option is for: a target counting its bytes buys the UTF-8 indexing of
`MRB_UTF8_STRING` without the table beside it.
- Narrows the character classification of `MRB_UTF8_STRING` back to ASCII
while keeping its indexing. What is classified today is case, so
`String#downcase`, `#upcase`, `#capitalize`, `#swapcase` and `#casecmp?`
answer for `'A'` to `'Z'` and hand every other character back as it stands.
- Drops the Unicode tables the build would otherwise carry. That is what the
option is for: a target counting its bytes buys the UTF-8 indexing of
`MRB_UTF8_STRING` without the tables beside it.
- Bytes that spell no character are handed back as they stand rather than
refused with `ArgumentError`. That refusal belongs to the walk over
characters, which is the walk this narrows away: what converts instead reads
Expand Down
2 changes: 1 addition & 1 deletion doc/limitations.md
Original file line number Diff line number Diff line change
Expand Up @@ -280,7 +280,7 @@ Module refinements (`refine`, `using`) are not supported in mruby.
mruby does not have an `Encoding` class. Strings are treated as
byte sequences by default. UTF-8 aware string operations can be
enabled with the `MRB_UTF8_STRING` compile flag, which is also what
makes case conversion follow Unicode rather than ASCII; `MRB_USE_ASCII_CASE`
makes case conversion follow Unicode rather than ASCII; `MRB_USE_ASCII_CTYPE`
narrows that half back without giving up the indexing. A Unicode conversion
refuses bytes that spell no character with `ArgumentError`; one narrowed to
ASCII reads no characters and hands those bytes back untouched.
Expand Down
6 changes: 3 additions & 3 deletions include/mruby/internal.h
Original file line number Diff line number Diff line change
Expand Up @@ -400,13 +400,13 @@ enum mrb_case_mode {
which is what every build without the tables answers to every string.
`swapcase` lives in mruby-string-ext and reaches the tables through this, so
they are asked about in one place. */
#if defined(MRB_UTF8_STRING) && !defined(MRB_USE_ASCII_CASE)
#if defined(MRB_UTF8_STRING) && !defined(MRB_USE_ASCII_CTYPE)
int mrb_str_case_convert_unicode(mrb_state *mrb, mrb_value str, enum mrb_case_mode mode);
#else
#define mrb_str_case_convert_unicode(mrb, str, mode) (-1)
#endif

#if defined(MRB_UTF8_STRING) && !defined(MRB_USE_ASCII_CASE)
#if defined(MRB_UTF8_STRING) && !defined(MRB_USE_ASCII_CTYPE)
/* What case a character has, from the tables in unicase.c. A string is
converted through mrb_str_case_convert_unicode() above; these are for a
caller holding a codepoint rather than a string, which is mruby-regexp
Expand Down Expand Up @@ -464,7 +464,7 @@ void mrb_uni_case_fold_range(uint32_t lo, uint32_t hi,
void mrb_uni_case_unfold_range(uint32_t lo, uint32_t hi,
void (*add)(void *, uint32_t, uint32_t), void *user);
#endif /* HAVE_MRUBY_REGEXP_GEM */
#endif /* MRB_UTF8_STRING && !MRB_USE_ASCII_CASE */
#endif /* MRB_UTF8_STRING && !MRB_USE_ASCII_CTYPE */

/* attr accessor bodies (class.c); the VM compares function pointers against
these to run attr calls without a full method-call frame */
Expand Down
2 changes: 1 addition & 1 deletion mrbgems/mruby-regexp/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -242,7 +242,7 @@ there.

Case folding beyond ASCII is not this gem's to configure. The table is
core's, carried by any build that defines `MRB_UTF8_STRING` without
`MRB_USE_ASCII_CASE`, and is what `String#downcase` and the four case methods
`MRB_USE_ASCII_CTYPE`, and is what `String#downcase` and the four case methods
beside it read; `/i` reads the two directions it needs over that same table.
So `/i` folds what the build's own case conversion folds, and a build
converting case by ASCII has nothing for it to fold beyond ASCII either,
Expand Down
2 changes: 1 addition & 1 deletion mrbgems/mruby-regexp/include/re_internal.h
Original file line number Diff line number Diff line change
Expand Up @@ -203,7 +203,7 @@ mrb_bool mrb_re_is_word_char(uint32_t c);
therefore folds the way that build's own case conversion does and no other
way, a pattern read as bytes having no character to fold in the first
place. */
#if defined(MRB_UTF8_STRING) && !defined(MRB_USE_ASCII_CASE)
#if defined(MRB_UTF8_STRING) && !defined(MRB_USE_ASCII_CTYPE)
# define RE_UNICODE_CASE
#endif

Expand Down
2 changes: 1 addition & 1 deletion mrbgems/mruby-regexp/mrbgem.rake
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ MRuby::Gem::Specification.new('mruby-regexp') do |spec|
# characters carries, and only where it converts their case by Unicode.
spec.build_settings do
if build.has_define?('MRB_UTF8_STRING') &&
!build.has_define?('MRB_USE_ASCII_CASE')
!build.has_define?('MRB_USE_ASCII_CTYPE')
spec.test_rbfiles -= ["#{spec.dir}/test/ascii_case.rb"]
else
spec.test_rbfiles -= ["#{spec.dir}/test/unicode_case.rb"]
Expand Down
2 changes: 1 addition & 1 deletion mrbgems/mruby-regexp/test/ascii_case.rb
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Only compiled into mrbtest when the build converts case by ASCII, whether by
# MRB_USE_ASCII_CASE or by reading its strings as bytes; see the gem's
# MRB_USE_ASCII_CTYPE or by reading its strings as bytes; see the gem's
# mrbgem.rake. Where it converts by Unicode, every pattern refused here
# compiles and matches instead.
assert("Regexp - /i refuses what ASCII folding cannot answer") do
Expand Down
12 changes: 10 additions & 2 deletions mrbgems/mruby-string-ext/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -809,14 +809,22 @@ Example:
"aBcDeF".casecmp?("abcdeg") #=> false
```

On a build defining `MRB_UTF8_STRING`, folding follows Unicode, and one folding may spell a character as several. `MRB_USE_ASCII_CASE` narrows it back to ASCII:
On a build defining `MRB_UTF8_STRING`, folding follows Unicode, and one folding may spell a character as several:

```ruby
"ä".casecmp?("Ä") #=> true
"ß".casecmp?("ss") #=> true
```

A string holding bytes that spell no character is refused with `ArgumentError`, since bytes that spell nothing have no folding. One read as bytes folds ASCII alone, having no characters to fold.
There, a string holding bytes that spell no character is refused with `ArgumentError`, since bytes that spell nothing have no folding. One read as bytes folds ASCII alone, having no characters to fold.

`MRB_USE_ASCII_CTYPE` narrows the folding of such a build back to ASCII, and the refusal goes with it, since what folds no longer reads characters:

```ruby
"ä".casecmp?("Ä") #=> false
"ß".casecmp?("ss") #=> false
"\xff".casecmp?("\xff") #=> true
```

### `String#+@` (Unary Plus)

Expand Down
4 changes: 2 additions & 2 deletions src/string.c
Original file line number Diff line number Diff line change
Expand Up @@ -2192,7 +2192,7 @@ mrb_str_aset_m(mrb_state *mrb, mrb_value str)
return replace;
}

#if defined(MRB_UTF8_STRING) && !defined(MRB_USE_ASCII_CASE)
#if defined(MRB_UTF8_STRING) && !defined(MRB_USE_ASCII_CTYPE)

/* What the walk below makes of an ASCII character. Each method keeps its own
loop over a string that holds nothing but ASCII, so this is reached only for
Expand Down Expand Up @@ -2363,7 +2363,7 @@ mrb_str_case_convert_unicode(mrb_state *mrb, mrb_value str, enum mrb_case_mode m
return str_case_convert_utf8(mrb, str, mode) ? 1 : 0;
}

#endif /* MRB_UTF8_STRING && !MRB_USE_ASCII_CASE */
#endif /* MRB_UTF8_STRING && !MRB_USE_ASCII_CTYPE */

/* 15.2.10.5.8 */
/*
Expand Down
4 changes: 2 additions & 2 deletions src/unicase.c
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
#include <string.h>
#include <mruby.h>

#if defined(MRB_UTF8_STRING) && !defined(MRB_USE_ASCII_CASE)
#if defined(MRB_UTF8_STRING) && !defined(MRB_USE_ASCII_CTYPE)

#include <mruby/internal.h>
#include "unicase.h"
Expand Down Expand Up @@ -387,4 +387,4 @@ mrb_uni_case_unfold_range(uint32_t lo, uint32_t hi,

#endif /* HAVE_MRUBY_REGEXP_GEM */

#endif /* MRB_UTF8_STRING && !MRB_USE_ASCII_CASE */
#endif /* MRB_UTF8_STRING && !MRB_USE_ASCII_CTYPE */
2 changes: 1 addition & 1 deletion test/t/string.rb
Original file line number Diff line number Diff line change
Expand Up @@ -498,7 +498,7 @@ def []=(*args)

assert('String case conversion - ASCII only') do
# The other reading of case: a build that converts by ASCII, whether by
# MRB_USE_ASCII_CASE or by reading its strings as bytes, has no mapping above
# MRB_USE_ASCII_CTYPE or by reading its strings as bytes, has no mapping above
# ASCII, so a character that has one on the Unicode side stands as it was
# while the ASCII beside it still converts.
assert_equal 'Ä', 'Ä'.downcase
Expand Down
Loading