Prof. Ohad Ben-Shahar

The reconstruction of specular shape from images is a very challenging task, especially when the illumination environment is unknown. In such cases, the problem becomes more tractable once the camera observes relative motion between the object and the environment, which induces a type of optical flow field that has become known as specular flow. Unfortunately, however, the estimation of specular flow from image sequences is even more challenging, putting in question the effectiveness of the shape from specular flow approach as a whole. Here we show that instead of the traditional (and somewhat futile) process of first estimating the specular flow and then using it for the recovery of specular shape, an approach that addresses these two inference problems simultaneously improves the estimation of both structures. We formulate the problem in a variational setting, we identify and address numerical issues unique to its application on specular flows, and we employ a polar representation of motion, all to result in the first ever practical method to compute specular shape from real image sequences under unknown illumination.