Seminar by Prof. David Forsyth (UIUC)

Host: Deep NeuroCognition Lab

Contact: Mengmi ([email protected])

Bio:

I am currently Fulton-Watson-Copp chair in computer science at U. Illinois at Urbana-Champaign, where I moved from U.C Berkeley, where I was also full professor.  I have occupied the Fulton-Watson-Copp chair in Computer Science at the University of Illinois since 2014. I have published over 170 papers on computer vision, computer graphics and machine learning. I have served as program co-chair for IEEE Computer Vision and Pattern Recognition in 2000, 2011, 2018 and 2021, general co-chair for CVPR 2006 and 2015 and ICCV 2019, program co-chair for the European Conference on Computer Vision 2008, and am a regular member of the program committee of all major international conferences on computer vision.  I have served six years on the SIGGRAPH program committee, and am a regular reviewer for that conference. I have received best paper awards at the International Conference on Computer Vision and at the European Conference on Computer Vision. I received an IEEE technical achievement award for 2005 for my research.  I became an IEEE Fellow in 2009, and an ACM Fellow in 2014.  My textbook, "Computer Vision: A Modern Approach" (joint with J. Ponce and published by Prentice Hall) is now widely adopted as a course text (adoptions include MIT, U. Wisconsin-Madison, UIUC, Georgia Tech and U.C. Berkeley).  Two further textbooks, “Probability and Statistics for Computer Science” and “Applied Machine Learning” are now in print.  I have served two terms as Editor in Chief, IEEE TPAMI.  I serve on a number of scientific advisory boards and have an active practice as an expert witness. 

Talk title:  Shading, Lighting, Geometry and Generation

Abstract: Computer vision research has been revolutionized by a relatively straightforward recipe: obtain annotated data, and apply modern classification or regression techniques, as appropriate. This recipe has solved commercially valuable problems and built fame and fortune for many. But do we really believe that animals have vision because the proprietor issued some early owners of an eyeball with a gold standard dataset? What do we do if we don’t have, or can’t get,  appropriately labelled data? Intrinsic images are maps of surface properties, like depth, normal and albedo. I will show the results of simple experiments that suggest that very good modern depth and normal predictors are strongly sensitive to lighting – if you relight a scene in a reasonable way, the reported depth will change. This is intolerable. To fix this problem, we need to be able to produce many different lightings of the same scene. I will describe a method to do so, using modern image generation technology. First, one learns a method to estimate albedo from images without any labelled training data (which turns out to perform well under traditional evaluations). Then, one forces an image generator to produce many different images that have the same albedo -- with care, these are relightings of the same scene.  I will show some interim results suggesting that learned relightings might genuinely improve estimates of depth, normal and albedo. But if an image generator can relight a scene, it likely has a representation of depth, normal, albedo and other useful scene properties somewhere.  I will show strong evidence that depth, normal and albedo can be extracted from two kinds of image generator, with minimal inconvenience or training data.   Furthermore, all these intrinsics are much less sensitive to lighting changes.  This suggests that the right way to obtain intrinsic images might be to recover them from image generators.  It also suggests image generators might "know" more about scene appearance than we realize. So what do image generators know? I will show strong evidence that image generators make reliable and significant errors in geometry.   The usual explanation  -- the method hasn't seen enough data  -- may not apply; instead, there could be problems with the underlying architecture of the generators.