1: \begin{abstract}

2: 	We present a learning-based approach to relight a single image of Lambertian and low-frequency specular objects. 

3: 	Our method enables inserting objects from photographs into new scenes and relighting them under the new environment lighting, which is essential for AR applications. To relight the object, we solve both inverse rendering and re-rendering. 

4: 	%There are two main challenges to solve this problem. The first one is the shortage of labeled training data for inverse rendering. The second one is the shortage of differentiable specular renderers for environment lighting. 

5: 	To resolve the ill-posed inverse rendering, we propose a weakly-supervised method by a low-rank constraint.  

6: 	%Based on the observation that reflectance is invariant to illumination change, it imposes that the reflectance maps of the same object under changing illuminations are linearly correlated. %(i.e., the reflectance matrix storing different reflectance maps to be rank one). 

7: 	%We have proven that the low-rank loss is mathematically correct and leads to training convergence. 

8: 	%Specular component can be separated similarly by low-rank loss on the diffuse chromaticity. 

9: 	To facilitate the weakly-supervised training, we contribute Relit, a large-scale (750K images) dataset of videos with aligned objects under changing illuminations.

10: 	%The capturing of Relit is effortless in the data capture setting. 

11: 	For re-rendering, we propose a differentiable specular rendering layer to render low-frequency non-Lambertian materials under various illuminations of spherical harmonics. 

12: 	The whole pipeline is end-to-end and efficient, allowing for a mobile app implementation of AR object insertion. Extensive evaluations demonstrate that our method achieves state-of-the-art performance. Project page: \href{https://renjiaoyi.github.io/relighting/}{https://renjiaoyi.github.io/relighting/}. 

13: \end{abstract}

14: