It turns out that a couple of people have demonstrated what we are attempting, to combine web cam head tracking and a 3D video display:
djine85's video is in French, and seems to work quite well.

From Danny's blog, http://danielbaggio.blogspot.com/, he seems to be getting pretty good performance.

Nonetheless, we will forge on!