Google actually seems serious about its wild “Project Starline” video booth idea. The mysterious project was announced as part of the Google I/O 2021 keynote but was initially overshadowed by more tangible Wear OS and Android announcements. It’s a year later now and Google is still plowing ahead with the idea, announcing expanded enterprise testing with third parties. Google says it’s also working on making Starline “more accessible.”
Project Starline basically asks the question, “What if Zoom was a giant, sit-down arcade machine?” While the home console version of video chat just involves a tiny camera above your laptop screen, Starline brings 3D video chat to life in a 7×7-foot sit-down booth, with seemingly no regard given to cost, size, or commercialization. The goal is to make it seem like the other person is in the room with you, and Google categorizes it as a “research project.”
As for what Starline actually is, a Google Research paper contains a good amount of detail. The display side of the video booth features 14 cameras and 16 IR projectors, which all work to create, capture, and track a real-time, photorealistic 3D avatar of the user. Four microphones and two speakers don’t just play back speech; spatialized audio and dynamic beamforming supposedly make the speech sound like it’s coming out of the avatar’s mouth.
That’s a lot of sensors.
GoogleAn annotated version of what everything does. It’s a really big version of Face ID.
GoogleHere are some of the 3D avatars that get generated, plus their meatspace counterparts.
Google
Sending a 3D avatar over the video chat connection allows Google to correct for eyeline, which is continually a problem with normal video chat. Where a webcam on top of a display makes it impossible to make eye contact while looking at a display, a 3D avatar can fudge the disconnect between the center of a camera and the center of a display, allowing for mutual eye contact. Google is crunching all this data in a beefy dual-Xeon workstation with “four NVIDIA GPUs (two Quadro RTX 6000 and two Titan RTX).”
The display is a 65-inch, 8 K, 60 Hz autostereoscopic lenticular panel that generates a glasses-free 3D view of a life-size avatar. It’s basically a big Nintendo 3DS—but with a bigger sweet spot thanks to head tracking. The other side of the booth features an infrared backlight and a rather rigid-looking bench to restrict the user into the display’s 3D sweet spot and to limit the scope of the whole avatar-generation system. Google even built a small barrier between the bench and the display to hide the bottom of the display. Rather than have an avatar awkwardly end once you hit the bottom of the screen, a physical occlusion over the bottom of the display supposedly tricks your brain into thinking the rest of the avatar exists behind the barrier. Google seems committed to controlling every possible variable with Project Starline, to the point that the booth even features its own lighting system, with both diffuse visible lighting to help with 3D texture capture and a big infrared backlight to help with 3D imaging,
People who have tried Starline seem to like it, but considering you have to be personally invited by Google to try it, that’s only a very small handful of people. It’s hard to imagine much of a market for what must be a six-figure video booth the size of a small bathroom, but Google is pushing ahead with more testing. A Google statement says: “Today, Project Starline prototypes are found in Google offices across the US, with employees using the technology every day for meetings, employee onboarding and building rapport between colleagues.”
Unlike the highly polished teaser video, this Siggraph presentation explains what’s going on a lot better.
The company continues: “Beyond Google employees, we’ve also invited more than 100 enterprise partners in areas like media, healthcare and retail to participate in demos at Google’s offices and provide us with feedback on the experience and applications to their businesses. We see many ways Project Starline can add business value across a number of industries, and we remain focused on making it more accessible.” Salesforce, WeWork, T-Mobile, and Hackensack Meridian Health have signed up to try it. WeWork, a company based around renting too-expensive-to-own office space, seems particularly enthused with the idea.
Google can talk all it wants about “research,” but the company is famously aggressive when it comes to killing things that don’t have hundreds of millions of users. Will there ever be a product here? Starline is sort of adjacent to the huge enterprise meeting equipment market, but a few limitations make Starline tough for serious meetings. Enterprise meeting equipment is usually geared for a big group sitting at a table, and wide compatibility means anyone can call into the meeting with almost any hardware. Starline only works with one person for one-to-one chats, and you can only talk to other Starline booths. Is there a market for VIP-to-VIP communication booths, like a modern version of the president’s red phone?