You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
If I convert it with Image > Type > 8-bit in Fiji I get:
If I denoise this image I get this result (which I'm happy with):
But if I try to convert it with e.g. result = ops.unary("convert.int8").input(img).apply() I get.. a 32-bit image??
convert.uint8 is 8-bit, at least..
Smoothing the uint-8 version requires bumping up the smoothing strength significantly and it still doesn't look great:
So, how I account for this behavior is:
In the ImageJ type conversion, values are scaled proportionately to the new range
In the SJ Ops type conversion, values are directly translated. This can result in type overflow (note the split tails in both the int8 and uint8 conversions) which fundamentally changes the behavior of subsequent image processing.
These byte casts are perhaps not working as intended?
@gselzer@ctrueden Is there rationale for not scaling the converters to the target data range?
@hinerm The first problem is due to limitations of the original ImageJ. Image → Type → 8-bit converts to uint8, not int8. When you run the convert.int8 op, you end up with a pixel type not supported by the original ImageJ, so it defaults to coercing it into 32-bit floats. We could potentially improve this for certain special cases, like int8 becoming ByteProcessor with a calibration offset so that it behaves as signed, although there are probably wrinkles here to consider. ImageJ is not really designed to support int8, fundamentally, so some maths may be broken in this circumstance, I'm not sure.
As for convert.uint8, another reason it looks bad is due to type casting versus clamping. There were big conversations I had with Alison Walter years ago about how this behavior should work, and IIRC we ended up settling on something that let you choose either behavior? But it's been a long time, and I'd need to revisit it to be sure. Certainly the raw type casting is much faster to execute, since you don't need to perform two numeric tests per pixel. But I agree that raw type casting is probably the wrong thing, expectations wise, the majority of the time. And neither raw type casting nor clamping will scale the data to the new data type. That should be a different op, in my opinion—it's not just a type conversion anymore at that point. IIRC, ImageJ Ops has a convert.scale but the design of it wasn't super great, so we should probably rediscuss how we want these sorts of operations to work.
That should be a different op, in my opinion—it's not just a type conversion anymore at that point. IIRC, ImageJ Ops has a convert.scale but the design of it wasn't super great, so we should probably rediscuss how we want these sorts of operations to work.
Yeah, I do agree that the best way forward is to make separate Ops for scale vs cast vs clamp, because this is how it worked in ImageJ Ops. I'd also note that NumPy's main mechanism for converting data types, numpy.ndarray.astype, also casts without scaling:
Add normalize ops, e.g. see those available in OpenCV. Written as a computer that takes an I input and O output and uses the O type min/max for normalizing.
Rename current type convert ops to be called cast
Possibly refactor current convert.clip Op that accepts a min and max, and make all params I. As a Computer<T extends Type> and/or Function<T extends Comparable>, renamed to math.clip
One other thing that occurred to me about the needed clip ops: a most general version could have separate T min, T max, T belowMinValue, T aboveMaxValue arguments, in case you want to do things like NaN out everything out of range. Even more general would be Predicate<T> boundsTest, T outOfBoundsValue or even multiple of those... but it starts to get ridiculous of course.
It works for my use case. I do find it a little unwieldy. I'd like to see overriding signatures variations/optionals, e.g.:
-[ ] No input sourceMin/sourceMax, instead using stats.min and stats.max to get the display min/max values
-[ ] No targetMin/targetMin, instead having a signature that accepts an O and uses the getMinValue/getMaxValue of that type.
-[ ] Could also just omit the O and pull it from the output in the case of a computer
None of these are absolutely required, just sugar that I think would be nice for users
Activity
ctrueden commentedon Mar 12, 2024
@hinerm The first problem is due to limitations of the original ImageJ. Image → Type → 8-bit converts to
uint8, notint8. When you run theconvert.int8op, you end up with a pixel type not supported by the original ImageJ, so it defaults to coercing it into 32-bit floats. We could potentially improve this for certain special cases, like int8 becomingByteProcessorwith a calibration offset so that it behaves as signed, although there are probably wrinkles here to consider. ImageJ is not really designed to support int8, fundamentally, so some maths may be broken in this circumstance, I'm not sure.As for
convert.uint8, another reason it looks bad is due to type casting versus clamping. There were big conversations I had with Alison Walter years ago about how this behavior should work, and IIRC we ended up settling on something that let you choose either behavior? But it's been a long time, and I'd need to revisit it to be sure. Certainly the raw type casting is much faster to execute, since you don't need to perform two numeric tests per pixel. But I agree that raw type casting is probably the wrong thing, expectations wise, the majority of the time. And neither raw type casting nor clamping will scale the data to the new data type. That should be a different op, in my opinion—it's not just a type conversion anymore at that point. IIRC, ImageJ Ops has aconvert.scalebut the design of it wasn't super great, so we should probably rediscuss how we want these sorts of operations to work.[-]Disappointing Converter performance[/-][+]Disappointing Converter behavior[/+]gselzer commentedon Mar 12, 2024
Yeah, I do agree that the best way forward is to make separate Ops for scale vs cast vs clamp, because this is how it worked in ImageJ Ops. I'd also note that NumPy's main mechanism for converting data types, numpy.ndarray.astype, also casts without scaling:
hinerm commentedon Mar 12, 2024
I was looking around and OpenCV does have a set of normalize methods that we could wrap to Ops.
Caveats:
Matsis limited and requires lots of copying.hinerm commentedon Mar 12, 2024
Current plan:
Iinput andOoutput and uses theOtype min/max for normalizing.castconvert.clipOp that accepts aminandmax, and make all paramsI. As aComputer<T extends Type>and/orFunction<T extends Comparable>, renamed tomath.clipctrueden commentedon Mar 14, 2024
One other thing that occurred to me about the needed
clipops: a most general version could have separateT min, T max, T belowMinValue, T aboveMaxValuearguments, in case you want to do things like NaN out everything out of range. Even more general would bePredicate<T> boundsTest, T outOfBoundsValueor even multiple of those... but it starts to get ridiculous of course.hinerm commentedon Apr 1, 2024
It turns out there is a suitable op:
image.normalize.It works for my use case. I do find it a little unwieldy. I'd like to see overriding signatures variations/optionals, e.g.:
-[ ] No input
sourceMin/sourceMax, instead usingstats.minandstats.maxto get the display min/max values-[ ] No
targetMin/targetMin, instead having a signature that accepts anOand uses thegetMinValue/getMaxValueof that type.-[ ] Could also just omit the
Oand pull it from theoutputin the case of a computerNone of these are absolutely required, just sugar that I think would be nice for users