feat(image-source): modern internals, fixed legacy APIs, new functions - #11428
sitefinitysteve wants to merge 1 commit into
Conversation
Public API stays backwards compatible. Existing method names and signatures are
unchanged; new methods are additive. Every method has an automated test in
apps/automated/src/image-source that runs the same assertion on iOS and Android.
## Why existing internals were swapped
- iOS resize/resizeAsync/fromFontIconCodeSync and NativeScriptUtils.scaleImage
used UIGraphicsBeginImageContextWithOptions, deprecated since iOS 17. On
wide-gamut (Display P3 / HEIC) photos it returned nil, which surfaced as
NaNxNaN ImageSources and failed saves. Replaced with UIGraphicsImageRenderer
at scale 1 and preferredRange = .standard, so maxSize is now real pixels on
both platforms (it was points times the image scale before).
- iOS async loaders (fromFile, fromResource, fromData) never settled when the
decode returned nil, and fromBase64 resolved an ImageSource wrapping nil.
They now reject with a descriptive error.
- iOS async encode/resize used dispatch_get_current_queue (deprecated since
iOS 6) and ran UIKit drawing on a global concurrent queue. The work now runs
in NativeScriptUtils and completes on the main queue.
- iOS saveToFile used a non-atomic createFileAtPath, which let a reader see a
half-written file. It now uses NSData writeToFile:atomically:. Android
writes to a temp file and renames for the same guarantee.
- Default JPEG quality was 90 on iOS and 100 on Android while the docs said
"maximum". Both platforms now default to 100.
- Android fromFileSync decoded at full size with the framework ExifInterface
and left the rotation pending in rotationAngle. resize() then dropped that
angle and saveToFile wrote the unrotated pixels, so a portrait photo loaded,
resized and saved came out landscape. Decoding now goes through the androidx
ExifInterface (already a widgets dependency), supports inSampleSize bounded
decodes, and bakes the full EXIF orientation (including mirrored variants)
into the pixels. Every pixel operation and encoder now works on the upright
bitmap.
- Android resize used createScaledBitmap with filtering off, giving
nearest-neighbour aliasing on photos. Filtering now defaults to true and
scaling draws through a Canvas with FILTER_BITMAP.
- Android toBase64StringAsync read the ByteArrayOutputStream before the
Base64OutputStream was closed, truncating the tail of the string. Fixed.
- Android "async" loaders decoded synchronously inside the Promise executor.
They now run on a worker thread in ImageUtils.
- Android loadImageAsync leaked its ParcelFileDescriptor on some failure
paths. It now closes in finally.
- Utils.snapshotView (iOS) used the same deprecated UIGraphics context; it now
renders through UIGraphicsImageRenderer.
## API
Full table with iOS/Android implementation columns: packages/core/image-source/Readme.md
All sizes are pixels. Every method that returns an ImageSource returns a NEW instance and never mutates its input. EXIF orientation is baked into pixels before any geometry work, so callers never reason about orientation flags after loading. Every method has an automated test in apps/automated/src/image-source that runs the same assertion on iOS and Android.
### Loading (static)
- fromFileSync(path, { maxSize? }) / fromFile(path, { maxSize? })
- iOS: Without maxSize: UIImage.imageWithContentsOfFile / TNSWidgets background decode. With maxSize: CGImageSourceCreateThumbnailAtIndex with kCGImageSourceThumbnailMaxPixelSize + CreateThumbnailWithTransform
- Android: ImageUtils.decodeFile: BitmapFactory bounds pass + inSampleSize, then androidx ExifInterface orientation baked in (was: full decode + framework ExifInterface, rotation left pending)
- Does: Loads an image from disk. With maxSize it shrinks while decoding so the result is never bigger than that many pixels on its long edge. The async form rejects instead of hanging when the file cannot be decoded. Returns a new ImageSource.
- Use: Load each photo in a list at 200 px to fill a thumbnail grid
- fromDataSync(data, { maxSize? }) / fromData(data, { maxSize? })
- iOS: NSData from ArrayBuffer/typed array, then UIImage.imageWithData or ImageIO thumbnail
- Android: ImageUtils.decodeBuffer from java.nio.ByteBuffer with EXIF orientation read from the bytes and baked in, or legacy BitmapFactory.decodeStream for an InputStream
- Does: Turns raw image bytes into an image. Accepts an ArrayBuffer or typed array on both platforms as well as the legacy native types. Returns a new ImageSource.
- Use: Show an image received from a fetch response or a plugin
- fromBase64Sync(source, { maxSize? }) / fromBase64(source, { maxSize? })
- iOS: Base64 → NSData → same path as fromData
- Android: ImageUtils.decodeBase64 on a worker thread (was: synchronous inside the Promise)
- Does: Decodes a base64 string into an image. Rejects on invalid input. Returns a new ImageSource.
- Use: Display an inline image from an API payload
- fromResourceSync(name) / fromResource(name)
- iOS: UIImage.imageNamed via TNSWidgets
- Android: Resources.getDrawable
- Does: Loads a bundled app resource by name. The async form now rejects when the resource is missing.
- Use: Load an icon from App_Resources
- fromFileOrResourceSync(path)
- iOS: Routes to file, res:// or sys:// loaders
- Android: Routes to file or res:// loaders
- Does: Loads from whichever source the prefix names.
- Use: Resolve a src string the way the Image view does
- fromSystemImageSync(name) / fromSystemImage(name)
- iOS: UIImage.systemImageNamed (SF Symbols)
- Android: Falls back to fromResource
- Does: Loads a platform system icon.
- Use: Show an SF Symbol on iOS
- fromUrl(url)
- iOS: http module then UIImage.imageWithData
- Android: http module then BitmapFactory
- Does: Downloads and decodes a remote image.
- Use: Fetch a profile picture
- fromAsset(asset)
- iOS: PHImageManager request via ImageAsset
- Android: Utils.loadImageAsync via ImageAsset
- Does: Decodes a picker ImageAsset at the asset's requested size.
- Use: Load a photo the user picked
- fromFontIconCodeSync(code, font, color)
- iOS: NSAttributedString drawn through UIGraphicsImageRenderer (was: deprecated UIGraphicsBeginImageContext)
- Android: Canvas.drawText
- Does: Renders a font glyph into an image.
- Use: Use an icon font glyph as a tab icon
- getMetadataSync(path) / getMetadata(path)
- iOS: CGImageSourceCopyPropertiesAtIndex
- Android: androidx ExifInterface + BitmapFactory bounds pass
- Does: Reads the information stored inside an image file, such as its pixel size, which way the camera was held, when it was taken and where, without decoding the picture itself. Returns { width, height, orientation, mimeType, hasAlpha, colorSpace, dpi, dateTaken, gps }.
- Use: Sort photos by date taken, or skip ones smaller than 1000 px
- fromView(view, scale?)
- iOS: UIGraphicsImageRenderer + drawViewHierarchyInRect
- Android: Utils.getBitmapFromView (View.draw(Canvas))
- Does: Takes a picture of any NativeScript view exactly as it is drawn on screen. Returns a new ImageSource.
- Use: Share a screenshot of a trip summary card
### Properties
- width / height
- iOS: UIImage.size (points)
- Android: Bitmap.getWidth/getHeight (pixels)
- Legacy size. Use getPixelSize() when you need pixels on both platforms.
- rotationAngle
- iOS: Always NaN
- Android: Pending rotation for bitmaps loaded before orientation was baked in
- Kept for compatibility. New loaders bake orientation in, so this is 0.
- ios / android
- iOS: UIImage
- Android: android.graphics.Bitmap
- The native image.
### Saving and encoding
- saveToFile(path, format, quality?) / saveToFileAsync
- iOS: NSData.writeToFile(atomically:) (was: non-atomic createFileAtPath)
- Android: ImageUtils.saveToFile: temp file + rename (was: direct stream)
- Does: Writes the image to disk safely, so the file is either complete or not there. Default quality is now 100 on both platforms (was 90 on iOS). Encoding from pixels never carries EXIF, so no camera or GPS metadata is written. Returns true on success.
- Use: Persist a picked photo to the app's documents folder
- toBase64String(format, quality?) / toBase64StringAsync
- iOS: NSData.base64EncodedStringWithOptions
- Android: Base64.encodeToString (async: closes the Base64 stream before reading, fixing truncated output)
- Does: Encodes the image and returns it as a base64 string.
- Use: Embed an image in a JSON payload
- toData(format, quality?) / toDataAsync
- iOS: UIImageJPEGRepresentation / UIImagePNGRepresentation → NSData exposed as ArrayBuffer
- Android: Bitmap.compress → direct ByteBuffer exposed as ArrayBuffer
- Does: Encodes the image as JPEG or PNG and hands you the bytes in memory instead of writing a file. Returns an ArrayBuffer.
- Use: Put the bytes straight into an HTTP upload body
- compressToFit(maxBytes, format?) / compressToFitAsync
- iOS: Binary search over JPEG quality in NativeScriptUtils
- Android: Binary search over JPEG quality in ImageUtils
- Does: Keeps re-encoding the image at lower quality until it fits under a byte limit you give it. Returns { data, quality }, or null when even the lowest quality is over budget (resize first).
- Use: Shrink a photo under 500 KB before uploading to an API with a size limit
### Geometry
- getPixelSize()
- iOS: UIImage.size * scale
- Android: Bitmap.getWidth/getHeight (swapped when a rotation is pending)
- Does: Tells you the true size of the image in pixels, on both platforms. Returns { width, height }.
- Use: Warn the user that a chosen cover image is too small
- resize(maxSize, options?) / resizeAsync
- iOS: UIGraphicsImageRenderer, scale 1, preferredRange = .standard (was: deprecated UIGraphicsBeginImageContext at the image's scale, which returned nil for wide-gamut photos and gave 2x/3x pixels)
- Android: Canvas.drawBitmap with Paint(FILTER_BITMAP); filter now defaults to true (was: unfiltered createScaledBitmap, pending rotation dropped)
- Does: Scales the image so its longest edge is at most maxSize pixels, keeping the aspect ratio. Never upscales. Returns a new ImageSource.
- Use: Make a 1080 px copy for storage
- resizeTo(width, height, { mode?, background? })
- iOS: Renderer with fit/fill/stretch rect maths
- Android: Canvas.drawBitmap with fit/fill/stretch rect maths
- Does: Makes an exact width and height: fit letterboxes (padding with background, transparent by default), fill covers and centre-crops, stretch ignores aspect. Returns a new ImageSource.
- Use: A 400×300 card cover with fill
- normalizeOrientation()
- iOS: Redraw through the renderer when imageOrientation != Up
- Android: Bitmap.createBitmap with rotation Matrix from rotationAngle
- Does: Rotates the pixels so the picture is upright according to its EXIF orientation, then clears the flag. Returns a new ImageSource.
- Use: Run before crop or saveToFile so portrait shots don't come out sideways
- crop(x, y, width, height)
- iOS: Renderer draw with negative offset
- Android: Bitmap.createBitmap(src, x, y, w, h)
- Does: Keeps only the rectangle you specify. Throws when the rectangle is outside the image. Returns a new ImageSource.
- Use: Cut the user's selection out of a photo
- rotate(degrees)
- iOS: Renderer with CGContextRotateCTM
- Android: Bitmap.createBitmap with Matrix.setRotate
- Does: Turns the image clockwise by the given degrees. Returns a new ImageSource.
- Use: Rotate-left button in an editor
- flip(axis)
- iOS: Renderer with CGContextScaleCTM(-1, 1)
- Android: Bitmap.createBitmap with Matrix.setScale(-1, 1)
- Does: Mirrors the image left-right, top-bottom, or both. Returns a new ImageSource.
- Use: Un-mirror a selfie
- transform({ crop, rotate, flip, resize }) / transformAsync
- iOS: One native pass, fixed order normalize → crop → rotate → flip → resize
- Android: Same order in ImageUtils.transform
- Does: Runs several edits in one native call, in a fixed order, without handing intermediate results back to JavaScript. Returns one new ImageSource.
- Use: Apply the user's crop, rotation and output size in one pass when they tap Done
### Compositing
- roundCorners(radius) / circleCrop()
- iOS: UIBezierPath clip inside the renderer
- Android: Canvas.drawRoundRect / drawCircle with a BitmapShader
- Does: Makes the corners transparent with the radius you give, or masks the whole image to a circle. Returns a new ImageSource with alpha.
- Use: Render a round avatar
- overlay(other, { x?, y?, opacity? })
- iOS: drawInRect:blendMode:alpha: inside the renderer
- Android: Canvas.drawBitmap with Paint.setAlpha
- Does: Draws another image on top of this one at a position and opacity you choose. Returns a new combined ImageSource.
- Use: Stamp a logo or watermark on a photo before sharing
- drawText(text, { x, y, font?, fontSize?, color? })
- iOS: NSString.drawAtPoint:withAttributes: inside the renderer
- Android: Canvas.drawText with a Paint from Font.getAndroidTypeface()
- Does: Draws a string onto the image with its top-left corner at (x, y). Returns a new ImageSource with the text baked in.
- Use: Burn a date or trip name into a photo
- tint(color)
- iOS: UIRectFillUsingBlendMode(SourceIn)
- Android: PorterDuffColorFilter(SRC_IN)
- Does: Recolours every visible pixel to one colour while keeping transparency, the way template icons work. Returns a new ImageSource.
- Use: Recolour a monochrome icon to the current theme
### Filters and analysis
- applyFilters([...]) / applyFiltersAsync — grayscale, sepia, invert, brightness, contrast, saturation
- iOS: CIColorControls, CISepiaTone, CIColorInvert rendered via CIContext to CGImage
- Android: One ColorMatrixColorFilter built from the filter list
- Does: Applies colour adjustments in the order given: black-and-white, sepia (amount 0..1), invert, brightness (-1..1), contrast (0..2, 1 = unchanged), saturation (0..2, 1 = unchanged). Returns a new ImageSource.
- Use: Offer black-and-white, sepia, or brightness sliders in an editor
- applyFilters([{ type: 'blur', radius }])
- iOS: CIGaussianBlur with edge clamping
- Android: ImageUtils.stackBlur (CPU stack blur, a close gaussian approximation that gives identical output on every API level)
- Does: Softens the whole image with a blur of the radius you give. Returns a new ImageSource.
- Use: Soft blurred background behind a card, or hide a licence plate
- averageColor()
- iOS: 32 px downscale, pixels sampled in ObjC
- Android: 32 px downscale, pixels sampled in Java
- Does: Works out the single average colour of the image, ignoring transparent pixels. Returns a Color, or null when nothing is visible.
- Use: Pick a background colour that matches the photo
- dominantColors(count?)
- iOS: Same sample, 4-bit-per-channel buckets
- Android: Same sample, 4-bit-per-channel buckets
- Does: Finds the handful of colours that appear most, most common first. Returns Color[].
- Use: Build a palette from a photo
- perceptualHash()
- iOS: 9×8 grayscale difference hash in ObjC
- Android: Same algorithm in Java
- Does: Computes a short fingerprint of what the picture looks like, so resized copies score nearly the same. Returns a 16-character hex string.
- Use: Store a fingerprint alongside each saved photo
- isSimilarTo(other, threshold?)
- iOS: Hamming distance of the two hashes
- Android: Same
- Does: True when the two images' fingerprints differ in at most threshold bits (default 10).
- Use: Skip a photo the user already added, even if it was resized
### Deprecated instance loaders
loadFromFile, loadFromResource, loadFromData, loadFromBase64, loadFromFontIconCode and the instance forms of fromFile, fromResource, fromData, fromBase64, fromAsset remain for compatibility and forward to the static loaders above.
### Explicitly out of scope for core
Face / text detection, QR encode / decode (Android needs ML Kit or ZXing), and HEIC / AVIF output (Android has no CompressFormat for them; androidx.heifwriter is an extra dependency) belong in plugins.
|
View your CI Pipeline Execution ↗ for commit 0a2849d
💡 Verify your cache is correct by running tasks in a sandbox. Read docs ↗ ☁️ Nx Cloud last updated this comment at |
commit: |
|
Background for WHY here team... I maintain a NativeScript-Vue app that does a lot of local photo work: users pick photos from the library, we downscale them for storage, build thumbnails for grids, and save JPEGs to the app's documents folder, yada yada. I've just been accumulating a pile of workarounds because core's ImageSource kept crapping the bed on iOS:
I'd reimplemented resize, save, thumb-nailing and orientation handling in app code with UIGraphicsImageRenderer, then found Android had its own set of problems (rotation dropped by resize, unfiltered downscales, a truncation bug in async base64).... the consistancy just isn't there (wasnt there) This PR is an attempt to give NativeScript a proper, modern image story in core for everyone... it's as tested as I can get it and validated with Fable and Astras sign off.
The goal is that ordinary photo handling works with @nativescript/core alone, with no plugin and no platform branches in app code, and that the maintenance of the native side sits in one place. Happy to split this into smaller PRs if that makes review easier... but COMMON lol. The API is the same as it is now we just have MORE available and faster modern backend implimentations. |
Public API stays backwards compatible. Existing method names and signatures are
unchanged; new methods are additive. Every method has an automated test in
apps/automated/src/image-source that runs the same assertion on iOS and Android.
Why existing internals were swapped
used UIGraphicsBeginImageContextWithOptions, deprecated since iOS 17. On
wide-gamut (Display P3 / HEIC) photos it returned nil, which surfaced as
NaNxNaN ImageSources and failed saves. Replaced with UIGraphicsImageRenderer
at scale 1 and preferredRange = .standard, so maxSize is now real pixels on
both platforms (it was points times the image scale before).
decode returned nil, and fromBase64 resolved an ImageSource wrapping nil.
They now reject with a descriptive error.
iOS 6) and ran UIKit drawing on a global concurrent queue. The work now runs
in NativeScriptUtils and completes on the main queue.
half-written file. It now uses NSData writeToFile:atomically:. Android
writes to a temp file and renames for the same guarantee.
"maximum". Both platforms now default to 100.
and left the rotation pending in rotationAngle. resize() then dropped that
angle and saveToFile wrote the unrotated pixels, so a portrait photo loaded,
resized and saved came out landscape. Decoding now goes through the androidx
ExifInterface (already a widgets dependency), supports inSampleSize bounded
decodes, and bakes the full EXIF orientation (including mirrored variants)
into the pixels. Every pixel operation and encoder now works on the upright
bitmap.
nearest-neighbour aliasing on photos. Filtering now defaults to true and
scaling draws through a Canvas with FILTER_BITMAP.
Base64OutputStream was closed, truncating the tail of the string. Fixed.
They now run on a worker thread in ImageUtils.
paths. It now closes in finally.
renders through UIGraphicsImageRenderer.
API
Full table with iOS/Android implementation columns: packages/core/image-source/Readme.md
All sizes are pixels. Every method that returns an ImageSource returns a NEW instance and never mutates its input. EXIF orientation is baked into pixels before any geometry work, so callers never reason about orientation flags after loading. Every method has an automated test in apps/automated/src/image-source that runs the same assertion on iOS and Android.
Loading (static)
Properties
Saving and encoding
Geometry
Compositing
Filters and analysis
Deprecated instance loaders
loadFromFile, loadFromResource, loadFromData, loadFromBase64, loadFromFontIconCode and the instance forms of fromFile, fromResource, fromData, fromBase64, fromAsset remain for compatibility and forward to the static loaders above.
Explicitly out of scope for core
Face / text detection, QR encode / decode (Android needs ML Kit or ZXing), and HEIC / AVIF output (Android has no CompressFormat for them; androidx.heifwriter is an extra dependency) belong in plugins.