Use Case and High-Level Description

This is a custom VGG-like convolutional neural network for gaze direction estimation.

Example and Gaze Vector Definition


The network takes three inputs: square crop of left eye image, square crop of right eye image, and three head pose angles – (yaw, pitch, and roll) (see figure). The network outputs 3-D vector corresponding to the direction of a person's gaze in a Cartesian coordinate system in which z-axis is directed from person's eyes (mid-point between left and right eyes' centers) to the camera center, y-axis is vertical, and x-axis is orthogonal to both z,y axes so that (x,y,z) constitute a right-handed coordinate system.


Metric Value
GFlops 0.139
MParams 1.882
Source framework Caffe2

Validation Dataset

Two random held out inviduals from an internal dataset containing images of 60 people with different gaze directions.

Validation Results

The accuracy of gaze direction prediction is evaluated through the use of MAE of angle (in degrees) between the ground truth and predicted gaze direction.

Dataset MAE, degrees Standard deviation of AE, degrees
Internal dataset 6.95 3.58



with the name left_eye_image and the shape [1x3x60x60].

with the name right_eye_image and the shape [1x3x60x60].

with the name head_pose_angles and the shape [1x3].


The net outputs a blob with the shape: [1, 3], containing Cartesian coordinates of gaze direction vector. Please note that the output vector is not normalizes and has non-unit length.

Output layer name in Inference Engine format:


Output layer name in Caffe2 format:


Legal Information

[*] Other names and brands may be claimed as the property of others.