Skip to content

Implement Type Conversion #4

Closed
Closed

Description

@Treiblesschorle

The matcher should be able to find Ops that do not exactly match the inputs/outputs, but where requested input/output argument types can be converted to/from the op candidate's actual parameter types via Converter plugins.

Activity

  1. gselzer commented on Aug 27, 2019

    @gselzer
    Member

    I have been thinking about this issue recently, especially with respect to primitives. It would be quite nice if the matcher had the ability to convert Integers to Doubles or Longs on the fly (these are my two bothersome use cases. I am wondering, however, where and/or when we would want to be doing the converting. It would be most convenient if we could place this logic within isAssignable: this would be by far the best place to do so if you are looking for the conversion while using ops.run. In comparison with another method like implementing transformations, it would be faster, more flexible, and require less transformation boilerplate, however we might (probably would) run into ClassCastExceptions. So we might have to go with the transformations.

  2. ctrueden commented on Aug 30, 2019

    @ctrueden
    Member

    @gselzer and I talked through two possible approaches for implementing converters:

    1. As an op transformer.
    2. Checking within the "assignability" layer.

    We concluded that (1) is not tenable: the targets may not be finite/closed. Suppose you have a RAI<FloatType> and want to pass it to an op that requires a II<DoubleType>. As a human being, we can see that this is a straightforward two-step conversion: use Converters.convert to go from RAI<FloatType> to RAI<DoubleType> and then call Views.flatIterable to go from RAI to II. But from the perspective of a conversion transformer: given a RAI<FloatType>: what is the list of destination generic types you support (see the Collection<OpRef> getRefsTransformingTo(OpRef targetRef) method of OpTransformer)? It may not be a finite answer; you can only state, for a given destination generic type, whether you could do it. But there could be arbitrarily many supported destination types.

    We lean toward (2) now because it would be feasible for any given converter plugin to answer the question "can you convert type T to type U?", regardless of the breadth/constraints on that converter plugin.

    That said, for many (all?) converter plugins, they can be expressed as java.util.function.Functions? I.e. the sorts of types upon which they operate are expressible via the Java generics grammar? Seems true, but we didn't think deeply about all the possible sorts of converters we want to support. There is also an issue of not knowing if the conversion will be lossy without knowing the specific source and destination types for a particular conversion. Think e.g. RAI<T> to RAI<U> for T extends RealType<T> and U extends RealType<U>—it might be lossy depending on the exact T and U.

    If we have a closed set of source and destination types supported by each converter plugin, we could maintain a data structure of all (A -> B) conversion chains. When a new converter for (P -> Q) is added to system, assuming we can efficiently query the existing conversion chains, we can look up all (A -> P) chains (for all A), and all (Q -> Z) chains (for all Z), then append/prepend the (P -> Q) converter to those chains and register those as well in the (A -> Q) and (P -> Z) lists, respectively.

    So maybe it's worth having two sorts of converters: the very general ones, and the ones that have closed sets of input and output types. This would be useful to avoid implementing a large number of converters across a range of types. Think about implementing converters for all six numeric primitive type wrappers to all other numeric type wrappers, and then the same for all RealType equivalents, and then again between each numeric type wrapper and RealType wrapper! Instead, we almost certainly will want to support implicit multi-step conversion.

    The generics of converters are also tricky. Converters that operate solely on raw types are not expressive enough (we know this from experience with the current SJC convert framework). But for a "closed converter" (see above) with generic conversion (P -> Q) where P and Q are generic types, it becomes more challenging to efficiently query the system for all (A -> P') and (Q' -> Z) chains, because P != P' and Q != Q'; rather, we want only that P' is assignable to P and Q is assignable to Q. So we may be stuck looping through all converters in the system every time a new converter is added in order to maintain this index of conversion chains. 🤮

    So maybe we want to impose a further restriction of only reified types for chainable converters? With this restriction, we could still chain convert e.g. Byte ➡️ Double ➡️ DoubleType. But sadly that would prevent chained conversion of e.g. RandomAccessibleInterval<FloatType> ➡️ RandomAccessibleInterval<DoubleType> ➡️ IterableInterval<DoubleType>.

    All of this complexity gives me pause. In the previous incarnation of ImageJ Ops, the conversion capability leads to an open-ended set of possible legal op executions (type-wise). It is not possible to "list all method signatures of ops"—certainly not without an exponential explosion of signatures, even assuming all converters are closed. Even for one single op in the system add8(Double, Double, Double, Double, Double, Double, Double, Double), if there are converters capable of going from all five other numeric types to Double, we now have 6^8 > 1 million method signatures for this one op! 🤯

    With the op transformer layer, we probably already have the same problem, so I guess this is not something to worry about.

    The only way I know to move forward on a new conversion framework is to start trying to code one here in scijava-ops. I expect it will take quite some work to get right.

  3. added this to the 1.0.0 milestone on Aug 26, 2020
  4. gselzer commented on Sep 28, 2020

    @gselzer
    Member

    I've done some thinking about how this would be implemented, and am using this to record my thoughts. The steps would be as follows:

    1. For every OpInfo that does not directly match the OpRef, record a list of the incompatible parameter indices. This information will need to be recorded (maybe we need an ImperfectOpCandidate?), and will require a good deal of change within DefaultOpMatcher and MatchingUtils. It would be nice if we can avoid changing the signatures of OpMatcher, but this may be unavoidable. This returned list should be sorted (probably by increasing number of incompatible indices).

    2. If and only if no OpInfo matches the OpRef (either directly or through adaptation), we should turn to this list of ImperfectOpCandidates. I am going to argue that adaptation should be prioritized over conversion as adaptation does not modify any of the user's arguments, better preserving the intended behavior.

    3. For each ImperfectOpCandidate, determine whether Converters exist for each incompatible parameter index.

    4. If some ImperfectOpCandidate can be converted, use Javassist to create some class to do the conversion.

    My attitude towards adaptation is that it should be thought of as a convenience layer instead of a core layer. Adaptation should be thought of as a way of saving a failing request, but should not be considered responsible for it (because we cannot possibly cover every adaptation the user might want). I will argue that conversion should be thought of in the same way, for the same reasons. It is the responsibility of the user to ensure that their request can be matched, not the responsibility of the matcher to massage the request into something that can be matched. To assume otherwise would be a futile task, since we cannot fathom the intentions of the user.

  5. gselzer commented on May 6, 2021

    @gselzer
    Member

    @ctrueden this issue is solved by scijava/incubator#25, yeah?

  6. gselzer commented on Jun 16, 2023

    @gselzer
    Member

    Type conversion works via the simplification paradigm, so I'm considering this completed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions