kuubee

Now what?

Derek — Wed, 09 Apr 2008 04:16:35 +0000

Thank you to all those who've tried our program. Sorry to those who are unsuccessful in running it. Unfortunately, we don't have a wide range of equipment so we can't debug the problems that some of you are having.

We are now onto trying to developing something new, so stay tuned ...

Virtual Viewpoint Demo Download (Updated 2008-03-01-23:10)

Derek — Thu, 28 Feb 2008 09:43:55 +0000

You can now download an executable version of the Virtual Viewpoint code. Warning: this is a draft of an experimental program and there are many problems that still need to be worked on. Updated: A new version of the executable that fixes the display resolution problem is now available. The link at the bottom of this post has been updated to point to virtualviewpoint_002.zip. Please tell us if this works for you.

What you'll need:

Webcam (must be automatically recognized by Windows)
Windows XP (Sorry, we've only tested on Windows XP.)
Executable and associated files. See download instructions in the next section.
You will need to have Version 2.0 (2.0.50727.42) of the Microsoft .NET framework runtime installed. Microsoft .NET Download

Download:

Download, save to a directory, and unzip. Download link at bottom of post.
VirtualViewpoint.exe - executable. Click on this to run.
bubbles.png - picture of the bubbles.
background.png - background picture. You can substitute your own .png picture by naming it background.png.
config.dat - configures screens. You can edit this file if you are adventurous. The default is for a 15 inch wide screen.

Operation and Controls:

Place the webcam above the screen and connect to your computer.
For best results, the camera should see a plain background and the space should be well-lit.
Sit back about a meter from the camera and screen.
Execute the program VirtualViewpoint.exe. (Webcam must already be connected.)
Press "S" to start the eye tracking.
Now shift your head position and watch the screen change accordingly. Be careful not to go outside the view of the webcam, which is typically fairly narrow.
If the algorithm loses track of your eyes, press "S" again to reset the eye tracking.
To increase reality of the 3-D virtual viewpoint experience, try closing one eye. Be careful not to cover your eye with anything as it will fool the eye-tracking.
You might try pressing "SPACE" when your head is centered to recalibrate.
Press "ESC" to exit the program
Have fun, and give us your feedback.

Download Virtual Viewpoint Demo Here

Viewer's Perspective Video

Derek — Wed, 20 Feb 2008 06:52:49 +0000

Based on emails that we received asking us to show what it looks like from the viewer's perspective, we've decided to post another video. In this video, we are using the same QuickCam webcam but we are now using a Core2 Duo laptop instead of the old Pentium 4 machine. We've also changed the background to simulate the view looking out the window of an Airbus A320. I don't know if the camera shows it too well, but it does look remarkably like you're there looking out into the clouds.

Head Tracking Virtual Viewpoint Resources

Derek — Tue, 19 Feb 2008 04:21:11 +0000

Here is a list of resources that we found useful:

OpenCV - open source computer vision software library from Intel - the standard. http://www.intel.com/technology/computing/opencv/
Johnny Lee's website - http://www.cs.cmu.edu/~johnny/projects/wii/
Microsoft C# Express - free tools to build the software. http://www.microsoft.com/express/vcsharp/
OpenCVDotNet - C# wrapper for OpenCV. http://code.google.com/p/opencvdotnet/

There's plenty more out there, but these are the main ones we used.

Success! Virtual Viewpoint Video

trungvo — Wed, 13 Feb 2008 08:34:35 +0000

After a few long hours of coding we finally got some success! And we're getting awesome performance on my old P4!

Here's a demo video of what we have accomplished with the Virtual Viewpoint program. We've combined Johnny Lee's 3D display with our OpenCV-based eye tracking code to create our program.

We just need a few days to clean some things up and we'll release the software for all of you to try at home! Stay tuned...

Please do leave us a comment and let us know what you think!

The Competition

Derek — Thu, 07 Feb 2008 09:15:40 +0000

It turns out that a couple of people have demonstrated what we are attempting, to combine web cam head tracking and a 3D video display:
djine85's video is in French, and seems to work quite well.

From Danny's blog, http://danielbaggio.blogspot.com/, he seems to be getting pretty good performance.

Nonetheless, we will forge on!

Inspiration

trungvo — Tue, 05 Feb 2008 10:17:43 +0000

A few weeks ago I stumbled upon this amazing video on youtube about Head Tracking for Desktop VR Displays using the WiiRemote:

How can you not say wow? This Johnny Lee is truly a genius! The guy is the definition of "out of the box" thinking. Who knew that a wii remote could serve such a purpose...and holy batman he's publishing his source code so that people can try it at home?!

As any other tech geek, I had to give this a try right away! I jumped off my seat, quickly blew the dust off my laptop, downloaded the source code and realized I had little problem: I don't own a Nintendo Wii...

Dishing out $250+tx for a console that I don't think I'll be using after this project didn't really appeal to me. There's gotta be another way to do this, yes there must be. So I thought, hey there's no reason why we wouldn't be able to achieve the same kind of effect using a cheap $20 webcam...or is there?

I phoned up my buddy Derek to do a little brainstorming and decided to start this blog. Here we'll keep you updated on our progress in creating a webcam based virtual viewpoint (or wii free desktop VR). Stay tuned...

In the meantime you can check out Johnny Lee's other amazing Wii projets on his project site.