Tests and setup for 1.1 development #277
Conversation
* Included new JSON-LD tests in json-ld.org directory (perhaps rename this to something more W3C specific, as these are the latest tests from W3C) * Starting from `manifest.jsonld` rather than all jsonld files in the test directory; then iterating of the the list of manifest files specified there.
- Set up tests for json-ld-api, json-ld-framing, and json-ld-1.0 - Run only json-ld-1.0 suite in Maven build (by naming convention) - Run all suites for development of 1.1 features (AllSuites.java)
- Skip only failing tests to make sure we run 1.1 code paths in CI - Use resolvable URLs in skip files, they can act like TODO lists - Delete `*-skip` files to re-generate them during next test run Now `mvn test` runs all tests that are not listed in the files.
|
Added setup for skipping tests in 79fdafa (before this, coverage went down, now it's up).
Now |
- be more verbose in some error messages - add some more comments pointing to the specs - add a blank node condition according to spec 16.4
|
I think this is could be a good starting point to implement json-ld 1.1 step-by-step. What do you think @ansell ? |
|
To clarify what this does compared to the current test setup in master:
For all of these:
The propsed workflow towards 1.1 would be:
|
Run tests with `-Dreport.format=jsonld`, convert to ttl, then: https://github.com/w3c/json-ld-api/blob/master/reports/README.md
For integration in https://w3c.github.io/json-ld-api/reports/ convert api and framing jsonld reports to single ttl file and then: https://github.com/w3c/json-ld-api/blob/master/reports/README.md
|
A quick status update: I've pushed some tweaks to the tests and report setup, updated the tests and looked into integrating with https://w3c.github.io/json-ld-api/reports/ (see 6f1eb6c). With current spec tests and implementation (
I discussed our next steps with @dr0i and we think we should split up this branch/PR into two branches: one for the changes in the test setup only ( |
|
Any report would be a good start; you can always submit an update and we can regenerate the report. Time permitting, I'll have a look at your code and offer some suggestions for improving conformance. |
6 participants
I suggest this to resolve #262.
I've been looking some more into what I mentioned in #265 (comment) and I think this is a good way to avoid both regressions while working on 1.1 features, and a long running 1.1 branch:
mvn testruns only the original 1.0 suite,mvn -Dtest=AllSuites testruns all 3 suites)@versionkeyword. #263)I think that with this setup, we can keep releasing from master whenever the build passes, while making available the 1.1 support that we currently have.