Design of Lightweight Linear Diffusion Layers from Near