Image Resample

August 19

1. Basics

Item	Description
Purpose	When scaling, rotating, or warping images, recompute pixel values for higher quality, suppressing aliasing and blurring.
Sample Point	Output pixels often map to non-integer input coordinates, requiring interpolation.
Frequency Response	Ideal resampling is low-pass filtering to suppress high-frequency aliasing. Real algorithms balance sharpness, smoothness, and performance.
Color Space	Recommended to process in linear space; avoid direct sRGB interpolation to prevent color shift.

2. Common Resample Algorithms (Interpolation Kernels)

Algorithm/Type	Approach	Advantages	Disadvantages/Notes
Nearest Neighbor	Use nearest integer pixel, no interpolation.	Fastest, simplest code	Blocky, jagged, lowest quality
Bilinear	Weighted average of 2x2 neighbors, linear interpolation.	Smooth, fast	Edge blur, detail loss
Bicubic	4x4 neighbors, cubic kernel interpolation.	Balanced sharpness/smoothness, good detail	More computation, possible slight ringing
Lanczos (windowed sinc)	Larger neighborhood (e.g. 6x6), windowed sinc kernel.	Sharp, excellent anti-aliasing	High cost, possible ringing
Mitchell-Netravali	Cubic kernel, tunable parameters, used for high-quality scaling.	Good detail/smoothness, suitable for content pipeline	Moderate complexity
Area/Box	Output pixel is average of input region, used for downscaling.	Good anti-aliasing when shrinking, fast	Blurry when upscaling, detail loss
Gaussian	Blur with Gaussian weights before sampling.	Natural smoothness, good anti-aliasing	Detail loss, heavier computation
Catmull-Rom	Cubic family, high sharpness, used in content pipelines.	Sharp edges, good detail	Ringing risk, moderate cost

In practice, nonlinear remapping of bilinear interpolation (e.g. smoothstep) can yield excellent results in some scenarios. See [[Bivariate Interpolation with Smoothstep Reparameterization]].

3. Special Handling & Engineering Tips

Type/Scenario	Description
Color Space	sRGB → linear for processing, interpolate then convert back to sRGB to avoid color shift/darkening.
Alpha/Transparency	Premultiply alpha before interpolation for physically correct transparency.
Normal Maps	Transform to unit vector, interpolate then normalize; avoid direct RGB interpolation to prevent distortion.
Multi-channel/Component	Interpolate channels independently or as vectors; weight if needed (e.g. Z or curvature).
Boundary Handling	wrap/clamp affects edge sampling; ensure seamless tiling and border pixel copy.

4. Algorithm Selection Advice

Real-time Rendering (Games/Interactive)

Prefer bilinear for upscaling, area/box for downscaling
Important assets: bicubic or Mitchell-Netravali
Lanczos for static or high-quality content
Normal maps: interpolate unit vectors then normalize

Offline/Content Pipeline

Recommend Mitchell-Netravali, Lanczos
Strictly process in linear space, ensure color management

Mobile/Embedded

Prefer bilinear/box for cost control
Use bicubic if high quality needed

5. Common Pitfalls

Interpolating in sRGB space: causes color shift/darkening
Directly interpolating normal/specular roughness maps: physically incorrect
Improper boundary wrap/clamp: visible seams
Not premultiplying alpha for transparency: edge artifacts

6. Reference Formulas

Bilinear interpolation: $C = (1-a)(1-b)C_{00} + a(1-b)C_{10} + (1-a)bC_{01} + abC_{11}$
Bicubic interpolation: $C = \sum_{i=0}^{3}\sum_{j=0}^{3} w_i w_j C_{ij}$
Lanczos kernel: $L(x) = \mathrm{sinc}(x) \cdot \mathrm{sinc}(x/a)$ , commonly $a=2,3$