One of the problems with conventional video conferencing is that each person sees the other from the point of view of the other person's camera. When each camera is directly above the screen, and each person looks at the image of the other, each person sees the other looking downwards.

This idea solves that, by having each person's computer convert the 2D video stream to a 3D video stream, then immediately convert that 3D back to 2D, but with a modified point of view.

The easiest way to make the 2D->3D conversion is for each computer to have a second video camera aimed at the user; this doesn't need to be *part of* the computer, it could easily be a camera-phone on a tripod, or even just propped up next to the computer. The two image streams provide a binocular image stream, from which distance data can be easily derived.

Each person's computer does two things with this 3D video stream...

First, it locates the position of the face of the person looking at the screen, and continually streams this position to the other computer.

Second, it converts the 3D stream to 2D, using the streamed position data from the other computer as the perspective from which to view the scene.

This perspective video stream is then transmitted over the internet, and displayed just like any ordinary video.

If we want to remove or replace the background, for privacy or for fun, it's easy, since we can chop out parts of the 3D model which are significantly further than the user's face.

This idea was inspired by the Instagram Hyperlapse app, which converts a 2D video stream to a 3D intermediate form, then changes it back to 2D for output.

Prior art is [link], which, instead of actually making a 3D stream, creates a mask to divide the 2D stream into foreground (the user) and background, and then recomposes them with altered parallax, based on the face position of the viewer.
-- goldbb, Dec 08 2014

Almost 3D Video Conferencing http://www.chrishar...p/Research/Pseudo3D
[goldbb, Dec 08 2014]

EP 0567597 http://www.google.c...s/EP0567597B1?cl=en
Eye contact apparatus for video conferencing [xaviergisz, Dec 08 2014]

Kinda already exists http://www.cisco.co...ence/index.html#~fp
Used this one. When set up properly, it really feels like eye contact. Creepy. Other vendors have equivalent products I presume. [Custardguts, Dec 08 2014]

3d with two cameras only works for a perspective equidistant between the two cameras, otherwise the cameras don't have the necessary data to fill in the full 3d object (A camera to the left of the built in still sees you looking down, and can't fill in, say, the area under your nose because it's shadowed from both cameras).

Better to use two built ins, above and below the screen (or better yet, all 4 corners) providing the complete required data.

It's still going to be very resource intensive, however.
-- MechE, Dec 08 2014


Not exactly the same as you describe, but high-end telepresence is pretty damned good. The one I've used a fair bit (pretty sure that's it I've linked) uses multiple screens and multiple cameras, positioned relative to the participants. Best used either as two-sides-of-a-table type setup, or one person-to-a-group setup. The room has to be configured especially for this setup.

Eye contact feels like looking at the other person.

Makes MS communicator/webex sessions feel like tapping morse code in the dark.
-- Custardguts, Dec 08 2014


Eye-contact is for weirdos.
-- pocmloc, Dec 08 2014


There's a possibly simpler solution: a display with a camera built-in, at the level where the eyes would appear.

The screen would look normal, apart from a small couple of millimetres) black spot where the camera is. In fact, it may even be possible to make a camera that looks out _through_ a display, eliminating the black dot.

Using such screen/camera devices, everyone in the video conference would make eye contact.
-- MaxwellBuchanan, Dec 08 2014



random, halfbakery