mruby-socket: reject a big integer getaddrinfo hint instead of ignoring it - #7118

Addrinfo.getaddrinfo converts its family, socktype and protocol hints only under mrb_integer_p, which asks about representation rather than class. Where mruby-bigint is built in, a hint too wide for mrb_int arrives as MRB_TT_BIGINT, the conversion is skipped, and the field keeps the default from struct addrinfo hints = {0}. The caller asked for something unrepresentable and got an unhinted lookup back.

That inverts the narrowing check added in #6960: a value that fits mrb_int but not C int raises, while a larger one passes silently. On the default 64 bit build, which carries bigint:

Addrinfo.getaddrinfo("localhost", nil, 2 ** 40)  # RangeError: getaddrinfo family out of range: 1099511627776
Addrinfo.getaddrinfo("localhost", nil, 2 ** 70)  # same results as no hint at all, afamily 2

It is not specific to MRB_INT32; any build carrying mruby-bigint has it, and MRB_INT32 only lowers the threshold to 2 ** 32.

Change

getaddrinfo_hint now takes the mrb_value and rejects a big integer before reading it, so an out of range hint is reported the same way whatever its representation. The three call sites test mrb_integer_p(v) || mrb_bigint_p(v). mrb_bigint_p is FALSE without MRB_USE_BIGINT, so a build without bigint compiles to what it had before.

flags is unaffected, because mrb_get_args already raises RangeError for a big integer passed to its i specifier. It moves to the new signature only to keep one entry point.

Values of other classes are still ignored rather than rejected, as before; narrowing that is a separate question from this one.

Test

The new assertion fails on master (KO: 1) and passes with the change. Its shift width comes from a variable because a literal wide shift is constant folded, and the fold fails where bigint is absent.

The two MRB_INT32 builds do not reach this file: the literal 1 << 40 in the assertion above the new one folds to a value the 32 bit VM cannot load, so the whole file is dropped without a failure. That is the subject of #7117, and it is left alone here so the two changes do not touch the same lines. Applying that fix locally on top of this branch gives 2147 (KO: 0, new test runs and passes) with bigint and 1785 (KO: 0, new test skips) without it.

Summary by CodeRabbit

• Bug Fixes
  • Improved validation of address resolution hints, including large integer values.
  • Values outside the supported range now raise a clear RangeError instead of being processed incorrectly.
  • Applied consistent validation across address family, socket type, protocol, and flags.