Simple Stupid Gaze Correction in a Video Call.
Making eye contact in a video conference call is almost impossible. If I want to make myself look someone in the eyes, I have to look at the camera, which means I cannot see the opponent’s face, vice versa.
There has been a number of research works that have been done to accomplish gaze correction with advanced computer graphics techniques. For example, the following video is a research result by a team at ETH Zurich. See this for more information.
This is pretty impressive, in fact (it looks so natural) compared to a project that I have done in the past with Kinect (as a master student), where I look like a distorted virtual devil trapped inside a computer.
As you can see, this looks awful. After this project (also 2012), I wonder how I could solve the problem. One simple stupid grumble was that “if the webcam was placed between my eyes, I wouldn’t even have to worry about this problem, because when the opponent look me in the eyes, they will stare at the same camera so I can actually feel like making eye contact as long as they are looking at my eyes on the screen (and same has to be true for my side).
And I thought maybe it is possible. Probably we cannot put motorized web camera behind a laptop screen but we can make a system that does this. (something like this but it should be 2D.) With a proper screen (translucent glass), a projector that will show conference call screen, and a motorized webcam behind the translucent glass that will be placed between two eyes.
Here is the very rough sketch.
The idea is that computer vision will keep track of the position of two eyes and give the location information to the system that will move the webcam to a right position. Therefore, they can, in fact, make physical eye contact (eye = camera)! One of the coolest things about this idea is that if a person on the screen moves their head and the perspective will move too. For example, if a person on the other side stands up you can see the upper part than before!
Well, it may not be a practical idea for a laptop but who knows when we will have a device in the future that can do this! (e.g. large-scale display in a gallery, a video conference room).
Another thing that is pretty exciting is that we may be able to replace the whole motorized webcam part with a drone! I just realized this idea while I am writing this. (Update) Speaking of drone, I believe it can be even a flying telepresence robot. Imagine that there will be a drone inside the cube below.
Anyway, I am hoping to find a collaborator that would like to work on this!
Relevant projects:
Leave a Reply