GPU RAW Processor: RAW to RGB Pipeline for Camera Applications
Updated:
GPU RAW Processor is our complete RAW-to-RGB processing scheme implemented on NVIDIA GPUs: every stage from RAW parsing and linearization to demosaicing, color correction, tone mapping, and encoded output runs in CUDA with a 16/32-bit workflow. The output image quality is comparable to Adobe Camera Raw, Lightroom, and RawTherapee — at a speed those tools cannot approach: in offline processing the GPU pipeline is faster than an SSD can feed it, and the same scheme runs in real time for machine vision cameras.
One Pipeline, Two Products
The scheme described on this page is built from the modules of our GPU Image & Video Processing SDK and ships in two ready-to-use forms. For offline work — primarily digital cinema and batch processing of recorded footage — it is implemented in the Fast CinemaDNG Processor application, with TIFF/EXR/ProRes export and a real-time RAW video player. For live camera processing it is implemented, almost in full, in the FastVCR software for machine vision cameras, with recording and low-latency RTSP streaming. And for custom applications, the same modules are available directly from the SDK, so you can assemble exactly the pipeline your task needs.
Advantages of RAW Processing on GPU
Why process RAW images on the GPU?
- The whole pipeline in GPU memory: data crosses the PCIe bus once in and once out, which is what makes faster-than-real-time rendering possible
- High quality demosaicing and adaptive denoising in a flexible 16/32-bit workflow
- Output image quality comparable to Adobe Camera Raw, Lightroom, and RawTherapee
- Smooth full-resolution RAW video playback without screen tearing
- Real-time resize before OpenGL output to monitor (no aliasing)
- Online histogram, RGB parade, and vectorscope for monitoring
- Per-frame image output (JPEG, TIFF, EXR) and video output (MJPEG, H.264/H.265, ProRes)
Features of GPU RAW Processor
- Input: 8–16-bit RAW, PGM, or byte array in CPU/GPU memory
- Output for image series: JPEG (24-bit), PPM (48-bit), TIFF (24/48-bit), EXR (48-bit)
- Output for video: AVI (Motion JPEG), MP4, MOV (Apple ProRes)
- Output color spaces: sRGB, Rec.709, Rec.2020, Adobe RGB, ProPhoto, BMD Film, BMD Film 4K, BMD Film 4.6K
- Demosaicing algorithms with 16-bit pipeline: L7, DFPD, MG
- Denoising for luma and chroma (float workflow): wavelet, bilateral, NLM
- Resize: Lanczos (float workflow); sharpening: USM (float workflow)
- Real-time playback of processed RAW images via OpenGL
- Optional export to Apple ProRes 422 LT / 422 / 422 HQ / 4444 / 4444 XQ — Apple's CPU-based codec, available for offline workflows
Image Processing Pipeline on GPU
- Ingestion of RAW images from HDD/SSD
- Multithreaded reading of RAW frames from SSD to CPU memory
- RAW parsing; optional RAW unpacking or decoding
- Data transfer from CPU to GPU
- Data linearization and transform to 16-bit representation
- Dark frame subtraction and flat-field correction
- RAW histogram
- White/black points
- White balance (R, G1, G2, B)
- Exposure compensation
- Raw tone curve for each channel of Bayer data
- Raw denoising
- Demosaicing
- Adaptive noise suppression
- Correlated color temperature and tint control
- Color correction with CCM matrix or DCP profile
- RGB/HSV curves for each channel
- Tone mapping
- Histogram and RGB parade for the processed image
- Monitor output
- Crop, resize to the output window, sharpening with USM
- Transform with monitor profile; 16-bit to 8/10-bit LUT per color channel
- Copy to OpenGL texture and real-time monitor output (RAW video player)
- Histogram, RGB parade, and vectorscope output
- Compressed video output
- Crop, resize, sharpening with USM; 16-bit to 8/10-bit LUT per channel
- Native MJPEG compression on GPU
- Optional native encoding to Apple ProRes (CPU)
- External compression via FFmpeg to MP4
- Transfer to CPU memory and saving to SSD/HDD
- Image sequence output
- Transform to 16-bit TIFF or 8-bit JPEG per frame
- Transfer to CPU memory and saving to SSD/HDD
Performance
For offline processing the honest bottleneck is not the GPU — it is the storage. The complete pipeline for 12-bit 4K RAW images runs at up to 60 fps end-to-end on the NVIDIA GeForce RTX 4090, including multithreaded reading from SSD, with the raw and processed data residing in CPU memory (without denoising and USM). The GPU stages themselves are far faster than that — per-module numbers are published for demosaicing, denoising, and the other SDK modules — so faster storage directly translates into higher throughput.
Camera Support
For offline processing, camera support is a matter of the file format, not the vendor: the pipeline accepts RAW, PGM, and DNG/CinemaDNG footage from any source — cinema cameras, drones, and industrial cameras alike. For real-time processing, the pipeline runs with the cameras supported by FastVCR: XIMEA (USB3 and PCIe), Hikrobot (USB3 and GigE), Emergent Vision (10/25/50/100-GigE), MIPI CSI2 cameras on NVIDIA Jetson, and GenICam (GenTL) compatible cameras from Basler, Imperx, FLIR, Lucid, IO Industries, Daheng Imaging, MindVision, and other vendors.
Typical Tasks and Workflow
- Batch RAW-to-RGB processing of image sequences: import RAW footage from a camera or card reader, process on the GPU, export JPEG/TIFF/EXR series or MJPEG, H.264/H.265, or ProRes video
- Real-time full-frame rendering, compression, preview, and playback of RAW video streams
- Full image processing pipeline for industrial cameras with 8–16-bit Bayer data
- RAW video player: smooth full-resolution playback of unprocessed footage with the full pipeline applied on the fly
Compatibility
- OS: Windows 10/11 (64-bit); the Linux version is in development
- CUDA 12.6
- NVIDIA GPUs (Pascal, Volta, Turing, Ampere, Ada) for mobile, laptop, desktop, and server
- Compatibility with FFmpeg to read/write MJPEG streams (FFmpeg is under LGPL v2.1)
Downloads and Related Software
- Fast CinemaDNG Processor — the offline implementation of this pipeline for Windows
- FastVCR portable demo — the real-time implementation; also processes RAW/PGM frames from SSD, no camera required
- Fastvideo SDK — the same modules for your own application, demo upon request
Roadmap for GPU RAW Processor
Recently released:
- ALTMapper — adaptive local tone mapping engine for single shot HDR
- Low-latency ISP on CUDA
- NLM and bilateral denoisers on the GPU
- Highlight recovery algorithm
In progress:
- Linux version of the GPU RAW Processor
- Chromatic aberration removal in the RAW domain with self-calibration
- RawZipper codec for lossy compression and decompression