Skip to content
This repository was archived by the owner on Oct 26, 2026. It is now read-only.

Rewrite SciJava Ops Engine Ops as Services - #48

Merged
ctrueden merged 14 commits into
mainfrom
scijava/scijava-ops-engine/explicit-op-environment
Jun 30, 2022
Merged

ctrueden merged 14 commits into
mainfrom
scijava/scijava-ops-engine/explicit-op-environment

Conversation

@gselzer

@gselzer gselzer commented Nov 15, 2021 •

Copy link
Copy Markdown
Member

This PR is targeted at rewriting all Ops contained within SciJava Ops Engine as Services, discoverable using the Discoverer.using(ServiceLoader::load) syntax. It will be the first in multiple PRs that work towards removing the SciJava Common dependency (and admittedly goes beyond its scope in that pursuit 😅 ). Major changes include:

  • The refactoring of SciJava Type's TypeService into a TypeReifier. Instead of using the SciJava PluginService to discover TypeExtractors, it uses one or more Discoverers to discover them. This allows us to remove the SciJava Common dependency from SciJava Types 🥳
  • The introduction of SciJava Log as an Incubator module. Arguably it should be called SciJava Log2?? Regardless, this partitions out org/scijava/log from SciJava Common, and refactors LogService into LoggerFactory. It still has the SciJava Common dependency though, which sucks.
  • The "turning-on" of ImageJ Ops2's tests. We could cherry-pick the commit to another branch if we wanted...
  • The replacement of the uber-OpEnvironment being constructed with many Discoverers in each test with an OpEnvironment that only runs with a StaticDiscoverer containing needed Ops. This nearly halved testing time on my machine.
  • The refactoring of all Ops in SciJava Ops Engine into Services (Ops or OpCollections) that are retrievable through ServiceLoader. Regression tests were added to ensure that all written Ops are discoverable from their designated Discoverer.
  • The Plugin discovery logic was moved into SciJava Discovery Plugin. We could delete it, but it really doesn't hurt anything to leave it...

TODO before merge:

  • Unfortunately, the test refactoring required me to turn off the subpackage dependence and package cycle enforcer rules; for this reason, this PR should not be merged until the tests do not break those rules. Add OpEnvironment API to Generate OpInfos from a Collection/Array of Classes scijava#80 should be solved to fix this
  • Remove SciJava Common dependency from SciJava Log
  • Update name to SciJava Log2?
  • Consider throwing LoggerFactory impls into an internal package? I actually don't feel strongly about this, since I could see the user wanting to select a particular implementation, but I also don't want to expose implementations...

@gselzer
gselzer force-pushed the scijava/scijava-ops-engine/explicit-op-environment branch from 32282c2 to bc677b8 Compare November 16, 2021 19:20
@ctrueden ctrueden self-assigned this Nov 22, 2021
@gselzer
gselzer force-pushed the scijava/scijava-ops-engine/explicit-op-environment branch from bc677b8 to b9dca36 Compare January 14, 2022 17:14
@ctrueden
ctrueden force-pushed the scijava/scijava-ops-engine/explicit-op-environment branch from b9dca36 to 1d159a9 Compare April 13, 2022 18:38

@ctrueden ctrueden left a comment •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm seeing test failures with this patchset, as follows:

check-branch.sh
0000 8a0a3aa7cae6a852d039a375ebbe9a28ba176d6b SUCCESS 76
0001 c465a04063725a82599e19f44cde83ff92528a5a SUCCESS 77
0002 5c384f47d5edafb48b2f26acfe705f5f02633953 FAILURE 25
0003 f373a551ea2da18231417fd8e3316bee1b716ba2 SUCCESS 80
0004 821ba57e0909fe35dceab559b7f932ec1d18b5fa FAILURE 148
0005 4a1114ceb97bbbeba2e61e2d0b6bf110b436858a FAILURE 130
0006 9ff1d7dac5b70aa657fdb70f6f405fa62e93e16c FAILURE 130
0007 a09d9124bd5e020a5b13cf8c690d14becbe4490d FAILURE 131
...

The tip of the branch (1d159a9 as of this writing) has failures earlier as well.

@gselzer Could you please try to reproduce on your system, and if so, investigate why things broke? Once you are able to fix and are happy with the branch, please mark it ready for review.

Edit: Here are the failing tests I saw on Ubuntu Linux 20.04:

[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.346 s <<< FAILURE! - in net.imagej.ops2.filter.vesselness.FrangiVesselnessTest
[ERROR] regressionTest  Time elapsed: 0.325 s  <<< ERROR!
java.lang.NullPointerException: Cannot invoke "org.scijava.script.ScriptLanguage.getScriptEngine()" because "js" is null
	at net.imagej.ops2@0-SNAPSHOT/net.imagej.ops2.filter.vesselness.FrangiVesselnessTest.regressionTest(FrangiVesselnessTest.java:101)
...
[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.22 s <<< FAILURE! - in net.imagej.ops2.filter.tubeness.TubenessTest
[ERROR] testTubeness  Time elapsed: 1.212 s  <<< FAILURE!
org.opentest4j.AssertionFailedError: expected: <true> but was: <false>
	at net.imagej.ops2@0-SNAPSHOT/net.imagej.ops2.filter.tubeness.TubenessTest.testTubeness(TubenessTest.java:66)

And on the tip of the branch:

[ERROR] Tests run: 4, Failures: 0, Errors: 4, Skipped: 0, Time elapsed: 0.085 s <<< FAILURE! - in org.scijava.ops.engine.impl.TherapiBasedOpTest
[ERROR] therapiOpFieldTest(org.scijava.ops.engine.impl.TherapiBasedOpTest)  Time elapsed: 0.082 s  <<< ERROR!
org.scijava.ops.api.features.OpMatchingException: No MatchingRoutine was able to produce a match!
	at org.scijava.ops.engine@0-SNAPSHOT/org.scijava.ops.engine.impl.TherapiBasedOpTest.therapiOpFieldTest(TherapiBasedOpTest.java:79)

[ERROR] therapiOpMethodTest(org.scijava.ops.engine.impl.TherapiBasedOpTest)  Time elapsed: 0 s  <<< ERROR!
org.scijava.ops.api.features.OpMatchingException: No MatchingRoutine was able to produce a match!
	at org.scijava.ops.engine@0-SNAPSHOT/org.scijava.ops.engine.impl.TherapiBasedOpTest.therapiOpMethodTest(TherapiBasedOpTest.java:93)

[ERROR] therapiOpClassTest(org.scijava.ops.engine.impl.TherapiBasedOpTest)  Time elapsed: 0.001 s  <<< ERROR!
org.scijava.ops.api.features.OpMatchingException: No MatchingRoutine was able to produce a match!
	at org.scijava.ops.engine@0-SNAPSHOT/org.scijava.ops.engine.impl.TherapiBasedOpTest.therapiOpClassTest(TherapiBasedOpTest.java:86)

[ERROR] therapiOpFieldPriorityTest(org.scijava.ops.engine.impl.TherapiBasedOpTest)  Time elapsed: 0 s  <<< ERROR!
org.scijava.ops.api.features.OpMatchingException: No MatchingRoutine was able to produce a match!
	at org.scijava.ops.engine@0-SNAPSHOT/org.scijava.ops.engine.impl.TherapiBasedOpTest.therapiOpFieldPriorityTest(TherapiBasedOpTest.java:122)

@gselzer

gselzer commented Apr 15, 2022

Copy link
Copy Markdown
Member Author

Strange. The tip of the branch before your force push (b9dca36) does not fail. I could not reproduce failures on 1d159a9 either:

[INFO] Results:
[INFO] 
[ERROR] Failures: 
[ERROR]   TubenessTest.testTubeness:66 expected: <true> but was: <false>
[INFO] 
[ERROR] Tests run: 513, Failures: 1, Errors: 0, Skipped: 1
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for SciJava Incubator 0-SNAPSHOT:
[INFO] 
[INFO] SciJava Incubator .................................. SUCCESS [  2.501 s]
[INFO] SciJava Functions .................................. SUCCESS [  3.088 s]
[INFO] SciJava Discovery .................................. SUCCESS [  1.437 s]
[INFO] SciJava Discovery Plugin ........................... SUCCESS [  3.795 s]
[INFO] SciJava Log 2 ...................................... SUCCESS [  3.433 s]
[INFO] SciJava Types ...................................... SUCCESS [  6.828 s]
[INFO] SciJava Struct ..................................... SUCCESS [  1.822 s]
[INFO] SciJava Ops API .................................... SUCCESS [  6.393 s]
[INFO] SciJava Ops SPI .................................... SUCCESS [  1.237 s]
[INFO] SciJava Parse 2 .................................... SUCCESS [  3.210 s]
[INFO] SciJava Discovery Therapi .......................... SUCCESS [  3.629 s]
[INFO] SciJava Progress ................................... SUCCESS [  3.692 s]
[INFO] SciJava Ops Engine ................................. SUCCESS [ 25.405 s]
[INFO] SciJava Test Util .................................. SUCCESS [  2.741 s]
[INFO] ImageJ Test Util ................................... SUCCESS [  1.439 s]
[INFO] ImageJ Ops 2 ....................................... FAILURE [02:07 min]
[INFO] SciJava Discovery Test ............................. SKIPPED
[INFO] SciJava Persist .................................... SKIPPED
[INFO] SciJava Taglets .................................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  03:19 min
[INFO] Finished at: 2022-04-15T14:39:11-05:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.22.2:test (default-test) on project imagej-ops2: There are test failures.
[ERROR] 
[ERROR] Please refer to /home/gselzer/code/scijava/incubator/imagej/imagej-ops2/target/surefire-reports for the individual test results.
[ERROR] Please refer to dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream.
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <args> -rf :imagej-ops2
(base) : gselzer@gselzer-OptiPlex-5090 ~/code/scijava/incubator (scijava/scijava-ops-engine/explicit-op-environment) [base] »

Thus I am getting the same failures as on CI. I'll look into why TubenessTest is failing...

@gselzer

gselzer commented Apr 15, 2022 •

Copy link
Copy Markdown
Member Author

I did a little digging, and it seems that the test data differs first after this call to HessianMatrix.calculateMatrix. I'll have to keep digging next week, though; ran out of time today.

EDIT: One thing I noticed was that there was a bump in imglib2-algorithm, from 0.11.2 in b9dca36 to 0.12.0 in 1d159a9. The numerical differences behind the failing test were pretty close, this could be something stemming from that update.

EDIT 2: TubenessTest passes if I hardcode imglib2-algorithm's version to 0.11.2 on 1d159a9

@hinerm

hinerm commented Jun 22, 2022

Copy link
Copy Markdown
Member

@gselzer

EDIT 2: TubenessTest passes if I hardcode imglib2-algorithm's version to 0.11.2 on 1d159a9

The ground truth image needs to be updated, as per imagej/imagej-ops@ca5b449.

I tried using the updated ground truth from imagej-ops and the test still fails. That seems like it might be a problem, that we get different results now? Or maybe it's fine and we just need to create a new tube.tif using imagej-ops2

@ctrueden
ctrueden marked this pull request as ready for review June 23, 2022 14:44
gselzer added 4 commits June 23, 2022 11:21
The goal is to remove the SciJava Common dependency on SciJava Types

This commit rewrites all Plugin-based TypeExtractors to Services that
can be loaded through ServiceLoader.
@ctrueden
ctrueden force-pushed the scijava/scijava-ops-engine/explicit-op-environment branch from 1d159a9 to 2ce3afb Compare June 23, 2022 16:21
gselzer and others added 9 commits June 30, 2022 14:45
They were not running because of the removal of JUnit Jupiter Engine
from the pom.

Once enabled, many of them failed. So there are a set of fixes here that
result in passing tests:

* Export net.imagej.ops2.types to org.scijava.types

* getOpMethod fallback: For the purposes of parsing javadoc parameters,
  we need to obtain the Op (read: functional) method of the Class of
  interest. Sometimes, however, this Op method isn't in the Class of
  interest, instead living in a superclass (and superinterface?). We
  cannot throw an error when such an Op class is discovered; we should
  just parse the superclass (superinterface?) method instead.

* Partition SciJava Ops ServiceLoader main/test

* Fix tag type from plugin type

* HACK: avoid NPEs when RichOp list null

* Euler tests: correct the conforms tests

* Replace IAE with DependencyMatchingException

* Use candidate type over info type

* Update the tubeness ground truth image for imglib2-algorithm 0.12.0;
  see also imagej/imagej-ops@ca5b449

Co-authored-by: Curtis Rueden <ctrueden@wisc.edu>
Co-authored-by: Mark Hiner <hinerm@gmail.com>
We are reworking discovery to work under the following paradigm, which
will help to remove the SciJava Common 2 dependency:

* SciJava Types: Replace (Default)TypeService with (Default)TypeReifier,
  discover TypeExtractors using ServiceLoader.

* Discoverers: Extract common logic

* Rewrite Ops Tests to use StaticDiscoverer under the
  BarebonesTestEnvironment class

* Further SciJava purging

* HACK: skip package enforcement
  See scijava/scijava issue #80 for the reasoning behind this hack

* Change up ServiceLoaderDiscoverer syntax to make it easier to call
  from the module using it.
We need to change the name in the while loop, or we will never get out
of it.
Use ImgLib2Assert.assertImageEquals instead.
It is simpler with better failure messages.
@ctrueden
ctrueden force-pushed the scijava/scijava-ops-engine/explicit-op-environment branch from 2ce3afb to 0b1a42c Compare June 30, 2022 20:04
@ctrueden

Copy link
Copy Markdown
Member

@hinerm @gselzer Good news: copying the tube.tif expected image from imagej-ops to incubator does fix the failing TubenessTest on my Linux system. So I integrated it into the "Turn on ImageJ Ops2 tests" as 91d3044. There is still at least one other failing test for me, though; will keep you posted.

In some Java environments, JavaScript is not available or not reliable.
Java 6's Rhino became Java 8's Nashorn and was then removed in Java 14.
Let's just compute this input image the old-fashioned way: in Java. ;-)
@ctrueden

Copy link
Copy Markdown
Member

The FrangiVesselnessTest failure I see on my Linux system is due to the use of DefaultEquation op. It relies on the JavaScript script engine, which is not available when I build. My weak theory is that maybe my Maven is using Java 17 rather than Java 11 at runtime, and Nashorn (the JavaScript engine) was removed after Java 14. Regardless, I avoid the issue now by building the FrangiVesselnessTest input image via simple population of a double[] rather than op fanciness.

The reason I'm not seeing a test failure for the DefaultEquation op itself is because it isn't being tested; only @bnorthan's Function equation implementation is. For now, let's leave it that way, till we figure out how to proceed. We will need to do something eventually, because we cannot rely on JavaScript being available at runtime going forward, especially as we move toward Java 17+.

@ctrueden
ctrueden merged commit 72176fd into main Jun 30, 2022
@ctrueden
ctrueden deleted the scijava/scijava-ops-engine/explicit-op-environment branch June 30, 2022 20:46
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

No open projects

Development

Successfully merging this pull request may close these issues.

3 participants