Modern mobile cameras rely on intricate in-camera processing systems, called Image Signal Processors (ISPs), to transform raw sensor data into the final, enhanced images we see. These ISPs perform a series of sequential tasks, from noise reduction and white balancing to color and contrast adjustment. However, hardware limitations, like small sensors and compact lenses, reduce the image quality of mobile cameras compared to high-end DSLRs. Traditional ISPs are also tailored to specific sensor and optical setups, which limits their flexibility across devices.
In the paper, the authors propose an end-to-end deep learning model named PyNET that performs the entire ISP pipeline. Trained with no knowledge of specific camera sensors or optics, PyNET processes raw Bayer data from mobile cameras into high-quality images that resemble those captured by professional DSLR cameras.
The “watercolor effect” is a common artifact in mobile phone cameras, particularly noticeable in images with fine details or textures. It manifests as a loss of sharpness and a smoothing of edges, giving the image a painted or watercolor-like appearance. This effect is often caused by aggressive noise reduction algorithms within the ISP that over-smooth the image, removing too much detail in an attempt to reduce noise.