Skip to content
This repository was archived by the owner on Oct 26, 2026. It is now read-only.

Create simplification framework - #25

Merged
hinerm merged 46 commits into
masterfrom
scijava/scijava-ops/simplification
Jan 19, 2021
Merged

hinerm merged 46 commits into
masterfrom
scijava/scijava-ops/simplification

Conversation

@gselzer

@gselzer gselzer commented Jan 13, 2021 •

Copy link
Copy Markdown
Member

This PR introduces a matching extension dubbed "simplification" that allows "on the fly" argument type conversion. This type conversion is extensibly facilitated by three sets of Ops:

  • a set of simplify Functions that produce a general type from a precise type.
  • a set of focus Functions that produce a precise type from a general type.
  • a set of lossReporter LossReporters that describe the worst-case loss resulting from a conversion between two precise types.

For example, to facilitate a conversion from Double to Integer Ops would require:

  • a simplify Op that produces a general type (e.g. Number) from a Double
  • a focus Op that produces an Integer from the same general type output by the simplify Op (i.e. Number)
  • a lossReporter Op that describes the worst-case loss of a conversion from a Double to an Integer.

During the matching process, simplification is performed after direct matching and after adaptation. This ordering was devised with two claims in mind:

  • Direct matches should always be used over adaptation/simplification to avoid any tampering.
  • Adaptation is preferable to simplification as it better preserves user intent; adaptation modifies Ops to suit matching calls, while simplification modifies matching calls to suit available Ops

The simplification process is currently performed as follows:

  • A SimplifiedOpRef is created, knowing:
    • each simplify Op that can simplify each of its inputs
    • all focus Ops that can focus its output
    • a copy Op that can be used to store the Op's result into a preallocated output (if applicable)
  • SimplifiedOpInfos are created, knowing:
    • each focus Op that can focus each of its inputs
    • all simplify Ops that can simplify its output
  • for each SimplifiedInfo, we attempt to find a pathway from each OpRef input to its corresponding OpInfo input using the simplify and focus Functions of the SimplifiedOpRef and SimplifiedOpInfo, respectively. If multiple pathways exist for a particular set of inputs, we sort by the number of identity simplify/focus Ops and choose the pathway with the most identity Ops. Ties are broken by hashcode to ensure deterministic behavior. A pathway is also found for the output, however the pathway is found from the OpInfo output to the OpRef output.
  • SimplifiedOpCandidates are recorded for all SimplifiedOpRef/SimplifiedOpInfo combinations with a complete set of valid pathways. Of these valid candidates, the candidate with the smallest loss (as determined by a set of LossReporters) is chosen.
  • Javassist is used to generate a Class that wraps the Op being simplified. This Class is responsible for:
    • simplifying and then focusing all inputs
    • calling the Op with the mutated inputs
    • simplifying and then focusing the output
    • copying the output back into the preallocated output (if applicable)

TODO:

  • Consider simplification and adaptation within a single Op call (e.g. can we call a Computers.Arity1<Integer, Integer[]> as a Function<Double, Double[]>`?) Should such a monstrosity be allowed?
  • Ensure correct behavior when multiple pathways exist
  • Clean up Git history. Much of the process involved failing tests/compile errors due to the nature and magnitude of the additions, thus there are many WIP commits. These should probably all be squashed.

@gselzer
gselzer force-pushed the scijava/scijava-ops/simplification branch 4 times, most recently from ab00995 to 8a94ee8 Compare January 15, 2021 22:17
@gselzer
gselzer marked this pull request as ready for review January 19, 2021 15:19
The DefaultOpMatcher already has methods to return a single match from a
list of OpRefs, which means that if we change the return of simplifyOp
to a list of OpRefs, we can make use of this code.
Now the tests depend on an Op within the test itself, which prevents
changes to external Ops from affecting the outcome of the test
This allows us to extensibly discover simplifiers
OpRefs with more types are:
* More complicated
* Never used in practice
These things (not sure what they will be yet) calculate the magnitude of
the (worst-case) loss when converting between two types
These Ops are now called LossReporters
This commit does two things. Firstly, it provides a proof of concept for
generic simplifiers. Secondly, it adds the logic necessary to simplify
to/from types that are not directly related to each other (as
illustrated in the test, where the simplification chain is Byte[] ->
ObjectArray<Number> -> Integer[])
The startup time was greatly increased due to the simplification
process. We can mitigate a lot of this time by lazily initializing
elements of the SimplifiedOpInfos, such as their simplified type and
their simplified Struct.
We now create the simplifiedOpInfos for Ops whose name matches the Op
request. This lazy approach saves time and opens the possibility for us
to refactor Simplifiers into Ops
Optional parameters are no longer allowed
In rare edge cases, functional interfaces don't declare their abstract
method. This might happen for a convenience interface like:

interface AllSameFunction<T> extends BiFunction<T, T, T> {}

such an interface does not have a declared abstract method, so we have
to find it via its superinterface BiFunction. The method we get out of
BiFunction, however, will not know of the mapping of the Ts in
AllSameFunction to the T, U, R of BiFunction; this is a bug in the
method.

This commit adds logic that determines the mappings of the type
variables of interfaces like AllSameFunction to the type variables of
BiFunction, so that Ops implementing AllSameFunction can be retyped.
This data strucutre held the Simplifiers, which we no longer use (since
    we converted to an Ops-based simplification framework)
This better aligns with the results of simplification
SimplifiedOpCandidate was taking on too much work. I would rather this
work be extracted to a helper class, so that the SimplifiedOpInfo can be
called without needing the SimplifiedOpCandidate. Of course,
SimplifiedOpCandidate can't go away, since some functionality (like
determining loss) is probably better performed there.
When Op wrappers exist for both a class and one of its superclasses, Op
wrapping would fail as multiple wrappers would be suitable. The Op
wrapping logic was not smart enough to filter these matches in any way.

This commit adds a method to filter superclasses out of the list of
suitable wrappers. Since these wrappers also preserve the exact
supertype of the Op w.r.t. the wrapper class, it is desirable to have
subclasses declare their own wrappers, and this fix prevents multiple
suitable wrappers from preventing a wrapper match (so long as all
classes are related).

This commit also fixes a bug in the error message when multiple wrappers
would match. The error was designed to print the classes of each wrapper
that matched, but would instead vomit out the classes of all known
wrappers. This bug has also been fixed.
@gselzer
gselzer force-pushed the scijava/scijava-ops/simplification branch from e084881 to 5d3b11d Compare January 19, 2021 15:34
@hinerm
hinerm merged commit 854288f into master Jan 19, 2021
@hinerm
hinerm deleted the scijava/scijava-ops/simplification branch January 19, 2021 15:40
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants