Apple’s Face ID, Animoji, and TrueDepth camera are years ahead of other 3D imaging systems on the market, but that didn’t stop the company’s closest competitor, Samsung, from employing a watered-down variation of it dubbed 3D Emoji on the Galaxy S9 and S9+.
And while Samsung tried (and failed, laughably) to implement the feature using years-old camera technology on its latest smartphones, it appears from the South Korean company’s latest patent hints that 3D Emoji could be just a precursor to Samsung’s larger, and far more ambitious, plans for the future of 3D facial imaging.
3D Emoji Sets the Stage
Since as far back as 2013, Samsung has allegedly been working on a robust new 3D imaging platform that it hopes will one day rival (if not overtake) video conferencing platforms like Skype, Hangouts, and even FaceTime, according to a Patently Apple report which outlines the technical details of the work in progress.
Samsung’s invention is based on a patent the company filed back in Q1 2016 with the U.S. Patent and Trademark Office, which it was officially granted last week and loosely describes a “future application” the Galaxy-maker is hoping will one day challenge the likes of current 2D communication systems like Skype and FaceTime.
Notably, this will be possible via a future Galaxy-branded smartphone boasting an upgraded “3D face camera.”
“Communication will be made between advanced 3D avatars,” Patently Apple explains, meaning that what Samsung introduced with AR Emoji is technically “just the appetizer before their video communications app.”
A 3D Revolution?
Patently Apple explains that while the current slate of video communication apps like Skype, FaceTime and Hangouts are designed to “transmit 2D video streams between devices,” sending “streams of compressed sequential images paired with audio streams” between them, Samsung’s patent inherently describes “methods and systems for visual communication between a first device and a second device” which includes “creating a 3D mesh model of a user, receiving sensor data from a sensor array during the visual communication session, wherein the image data includes motion of the first device user, and determining 3D mesh model updates using the image data.”
According to Samsung’s patent, transmitting so-called “3D mesh model updates” requires “significantly less bandwidth than sending [2D] image sequences, allowing for smooth communication in bandwidth constrained environments.”
See Samsung’s patent FIG. 1, 3, & 4 below for a visual illustration of this purely technical patent. The company states that the application would be cross-platform and compatible with either Android, iOS, Windows Phone and Blackberry devices — though it’s unclear how that would be possible if not all devices are equipped with compatible 3D image sensors.
The literature goes on to describe various embodiments, including one where 3D image data might be collected via a combination of activity sensors — such as accelerometers, gyroscopes and magnetometers — which may be used to “determine movement of the first device and/or the first device user” during a video transmission.
Ultimately, as Samsung illustrates in patent FIG. 3 and 4, 3D mesh models of a user’s face and head may be stored in a “3D model database,” which might also be capable of storing additional facial expressions and body orientations as templates for future transmissions.
There’s no telling if or when Samsung might be ready to release a video communications platform along these lines, however the literature explicitly points to a future Galaxy device — so it’s safe to say there’s still plenty of time until then. As far as Samsung’s platform ever overtaking apps like Apple’s FaceTime or Microsoft’s Skype, well, that’s debatable, but they can always hope for the best..