DEV Community
Follow
FP16 Quantization Made Our In-Browser Real-ESRGAN 4.8x Faster — and Nearly Destroyed Our Inpainting Model
ClearPix significantly accelerated browser-based super-resolution and watermark removal by optimizing ONNX Runtime Web on WebGPU. Their biggest speedup, a 4.8x boost, came from fp16 quantization on their super-resolution model. They found that browser GPU inference is typically memory-bandwidth bound, not compute-bound, making optimizations like worker pools and IOBinding ineffective. This led to the conclusion that reducing data transfer, such as by halving weight precision, is key.A crucial early decision was switching from the larger RRDBNet x4plus to the compact SRVGGNetCompact x4v3 architecture for super-resolution. This change alone yielded a 10x speedup and improved quality, demonstrating that selecting a model designed for the deployment environment is paramount. After adopting SRVGGNetCompact, fp16 quantization reduced the model size by half and improved WebGPU inference speed by 3.5x, leading to the overall 4.8x page-level improvement with negligible quality loss.However, applying fp16 quantization to their mask-gated LaMa inpainting model resulted in severe quality regressions. The mask-gating mechanism, which relies on precise decision boundaries, was disrupted by reduced numeric precision, causing the model to fail at watermark removal. This experience highlighted that quantization feasibility depends on a model's architecture and its sensitivity to precision-sensitive structures like gating.ClearPix now follows a strict optimization order: first, select an architecture suited for the browser environment; second, consider bit width only if the model lacks precision-sensitive internals; and third, explore runtime optimizations with low expectations. They also emphasize that quality benchmarks, not subjective visual inspection, are essential for validating model changes. Their optimized models are deployed client-side, enabling private, in-browser media processing tools.