Face tracking in action. The CCK node is scoring in real time how alike to the 'Smile' preset my face is.
Face tracking in the CCK offers an accessible way of leveraging the most modern headsets hardware to scan the face of the user and interpret their emotive features. As a developer in the CCK, if you want to use the face tracking node, you will be presented with a choice of preset emotions and a set of conditions which will be used while measuring the emotive output.
For example, a developer might wish to measure the degree to which the user is smiling.
In this case they would pick the smile emotion and then state that for the ten seconds that this node is active, I want to make sure the user smiles for at least five seconds of it. The logic of the node can navigate whether this is true or false by a method of comparing what the data we receive during the node's activity.
The way all of this works for both Oculus and PICO is by producing a series of values known as blend shapes. A blend shape is a computer graphic and animation technique which is basically a way to have a smooth interpolation between two different poses or shapes of a three dimensional object. Each frame, a set of these values are recorded and compared to the preset emotion.
Creation of the preset emotion asset.
An interesting aspect about the way we create the emotion assets is the level of potential flexibility within the system. Since they are effectively a recording, it becomes open to what the developer (or face actor) chooses to portray, meaning you could track whatever facial configuration you want.
After a name for the asset is chosen, then a countdown will begin, and three seconds of recording will occur. During this time the user is instructed to hold the pose they wish to create. Each frame we record a set of blend shapes and track the highest and lowest values produced per shape. After the time is up, we average all the values, cull the miniscule values and divide priority to the remaining blend shapes which have the highest values. iA weighting is produced with this method to better score the user inside the CCK when they are attempting to copy the set of shapes.
This is currently implemented in a separate project for internal use, but I’d intent to integrate this process into the CCK, when priority allows.
댓글