@marcandre I am seeing failures in both MRI's tests and in ruby/spec.

First I saw this in the MRI 2.6 version of the ostruct test:

  1) Failure:
TC_OpenStruct#test_accessor_defines_method [/home/travis/build/jruby/jruby/test/mri/ostruct/test_ostruct.rb:183]:
<[]> expected but was
<[:foo, :foo=]>.

Updating that to use the test from the gem repo, the failure changes to:

[17/30] TC_OpenStruct#test_method_missing = 0.01 s         
  1) Failure:
TC_OpenStruct#test_method_missing [/Users/headius/projects/jruby/test/mri/ostruct/test_ostruct.rb:175]:
Expected /test_method_missing/ to match "/Users/headius/projects/jruby/lib/ruby/stdlib/ostruct.rb:258:in `method_missing'".

CRuby does not fail this, so I will try to dig deeper. However, CRuby fails the spec in exactly the same way.

Here's JRuby:

1)
YAML.dump dumps an OpenStruct FAILED
Expected "--- !ruby/object:OpenStruct\nage: 20\nname: John\n"
 to match "--- !ruby/object:OpenStruct\ntable:\n  :age: 20\n  :name: John\n"
/home/travis/build/jruby/jruby/spec/ruby/library/yaml/dump_spec.rb:40:in `block in <main>'
org/jruby/RubyBasicObject.java:2695:in `instance_exec'
org/jruby/RubyArray.java:4568:in `all?'
org/jruby/RubyArray.java:1820:in `each'
/home/travis/build/jruby/jruby/spec/ruby/library/yaml/dump_spec.rb:5:in `<main>'
org/jruby/RubyKernel.java:1083:in `load'
org/jruby/RubyBasicObject.java:2695:in `instance_exec'
org/jruby/RubyArray.java:1820:in `each'

And CRuby:

1)
YAML.dump dumps an OpenStruct FAILED
Expected "--- !ruby/object:OpenStruct\nage: 20\nname: John\n"
 to match "--- !ruby/object:OpenStruct\ntable:\n  :age: 20\n  :name: John\n"
/Users/headius/projects/jruby/spec/ruby/library/yaml/dump_spec.rb:40:in `block (2 levels) in <top (required)>'
/Users/headius/projects/jruby/spec/ruby/library/yaml/dump_spec.rb:5:in `<top (required)>'

I do not know if the MRI suite and ruby/spec failures are related.

It occurs to me now the test_ostruct failure is expecting a specific line in the stack trace, which is likely a behavioral difference on JRuby or a bad test.

The spec failure looks like a real issue, though.

The updated test_ostruct.rb passed in CI, so only the spec failure remains.

Thanks for the heads up, I hope to check it out tomorrow

@marcandre Commenting out the additions from ruby/ostruct@683c3d6 appears to make the spec pass. The table instance variable appears to be leaking out as a struct element due to that patch.

Oh sorry, I had it reversed... apparently the spec expects table: to be there but it's not in latest ostruct, which seems more correct to me. I think this spec may be specifying old weird behavior that's been fixed.

@marcandre I have pushed a spec change that deals with the behavior change in 0.3.0. The version check isn't pretty... some alternatives to this:

• Guard based on Ruby version and assume the gem is not installed. I'm not sure when (or if) this change has gotten into a CRuby release.
• Remove the spec and add something to test_ostruct.rb, which is versioned alongside ostruct.rb. But we lose a spec.

When @brixen made the "rubysl" versions of the standard libraries, he moved the specs into those libraries... so that could be considered as a longer-term option, but we'd need a process for keeping the "complete" spec suite in sync.

Or would we? I have always considered the library specs to be outside the domain of "Ruby specs" since the language, core, and C ext specs should be sufficient to support any library. If something in library doesn't pass then there's probably a missing or broken language, core, or C ext spec.

In any case, I feel like this is more an ostruct behavior than a yaml behavior, so it probably should be moved to an ostruct spec or test. That would excuse or eliminate the extra version-checking and internal knowledge required.

FWIW, my thinking behind moving the specs into the specific gems was that it seemed like the best trade off for modularity. I know it implies some extra complexity somewhere if you want to run the "whole" suite, but like you mention, what does that really mean? You can put a boundary around the core+language specs, but not really around all the possible libraries as the standard library becomes more fluid.

Definitely not a simple problem or solution, but I opted for more modularity.

I'm going to merge this.

The spec change is not pretty but it seems appropriate since this is new behavior. I still think the spec should be moved into ostruct specs, wherever they live, and we should probably add specs for the "legacy" format being supported on 0.3.0.

Looping in @eregon for spec discussion.

I still think the spec should be moved into ostruct specs, wherever they live, and we should probably add specs for the "legacy" format being supported on 0.3.0.

One concern is when this is done, the result is often that those specs/tests are not run by Ruby implementations anymore, because it's extra trouble to run them.
Is there even a way to automatically get the tests/specs from a default gem and run them?
For tests, MRI simply copies the test files from the default gem (which adds various restrictions), and copies its minitest + test-unit harness to each default gem. It seems to work OK though.

So if we move stdlib specs to default gem repositories, we need a plan to still run the specs from Ruby implementations, and right now there is no plan and no tooling to do that easily. Feel free to open an issue on https://github.com/ruby/spec with some ideas how to do that.

And those specs are still valuable, so I don't want to just remove then.
They're often much easier to understand what goes wrong when one of them fail than MRI tests, and they explain the reasoning behind the behavior vs having to reverse-engineer what a test is trying to test and why it behaves like that. They likely improve the coverage of stdlib too.

And those specs are still valuable, so I don't want to just remove then.

Nobody said they should be removed, but their value as part of a conceptual "Ruby spec suite" is dubious. Anything a library spec can prove about a Ruby implementation would be better proven by having a language, core, or C API spec for that behavior.

The only thing that library specs (should) prove is that the library behaves according to its own spec. It should not be used as a metric for Ruby compatibility... that's what language, core, and C API specs are for.

So if we move stdlib specs to default gem repositories, we need a plan to still run the specs from Ruby implementations

Do we? Isn't it sufficient to have those libraries run their CI against the implementations in question? This is how it is for every non-stdlib library out there, and whenever those libraries fail on a given implementation we (ideally) add a "core" spec for whatever behavior was broken.

1. It is expected that new ostruct has improved YAML output, hence the spec change, .thanks

2. I'm very happy about gemification of stdlib, and I hope one day we'll go even further and have them truly separate. The current setup is far from ideal imo. Once that's done, these gems can have their own separate specs (even using rspec instead of mspec) and outside of rubysl or rubyspecs. These gems are quite mature though, so I'm not holding my breath as we probably have more pressing things to attend to.

The only thing that library specs (should) prove is that the library behaves according to its own spec. It should not be used as a metric for Ruby compatibility... that's what language, core, and C API specs are for.

stdlib (and default gems, they are still part of stdlib just with an extra gemspec) are typically more used than random gems. So tracking compatibility with them more closely directly in language implementations makes sense to me. Also stdlibs are used by almost every Ruby program so IMHO they make sense in terms of Ruby compatibility.

Also some of the stdlib/default gems have a different implementation in some Ruby implementations, e.g. tempfile and quite a few more. Ensuring these different implementations are compatible with ruby/spec (+ MRI tests) is what I think is most convenient.
For stdlib C extensions like socket, this is even more the case, I think almost every Ruby implementation has its own version.

For pure-Ruby stdlibs like ostruct, I agree it's less needed.