Super-Resolution on an Edge NPU

Super-resolution takes a blurry, low-detail image and predicts the sharp one it came from. Doing that on a server is a solved problem. Doing it on a small, low-power AI accelerator — the kind that sits in a camera or a robot rather than a data center — is a different one: the chip runs a fixed menu of operations at a fixed input size, in 8-bit integers rather than the floating point a model is trained in. This project builds a model for that chip from the start, rather than shrinking one that was designed for a GPU.

Fast enough for live video

On model latency alone, 8.55 ms an image is well inside the 33 ms a frame that 30 FPS video allows, which is the budget a live camera sets.

Designed around the chip

Every architectural choice was made to fit what the hardware runs well.

• A U-Net sized to the chip’s input. The NPU takes a fixed 256×256 image, so the network is built for exactly that shape instead of being resized to it afterwards.
• Depthwise-separable residual blocks. These split a convolution into a cheap per-channel filter and a cheap channel mix, doing much of the same work as a standard convolution for a fraction of the arithmetic.
• Nearest-neighbor upsampling. A fixed, simple operation for enlarging feature maps, in place of a learned one.
• Global residual learning. The network predicts only the correction to add to the degraded image, not the whole image from scratch, which is a smaller and easier thing to learn.

From floating point to 8-bit

The model was exported through ONNX and quantized to INT8 after training, which lets the chip run it with integer arithmetic. Quantization needs a calibration set to decide how to map each layer’s range of values onto 256 levels. I calibrated on the training images rather than a generic set, so the ranges it learned are the ones this model actually sees.

Training

L1 loss, mixed-precision training, and flip-and-rotation augmentation on image patches, which multiplies the effective size of 2,217 pairs without collecting more.