The top 5 commercial VR innovations enabled by Tobii Ocumen

  • by Robert Malmström
  • 6 min

VR headset on a table, looking at the left lens from the viewer's persepective

My position as the product manager for Tobii Ocumen has given me a front-row seat on the convergence of XR and eye tracking, and I’ve had the opportunity to get close to some of the amazing solutions that result. Since 2020, when we launched
Tobii Ocumen — our advanced VR toolkit — we’ve spent some time refactoring the code to ensure robustness, and we’ve engaged with developers from different specialties and disciplines, helping them to commercialize VR solutions that leverage our eye tracking technology and the advanced biometrics and tools for development and deployment included in the product.

In this post, I will share five of what I think are some of the most innovative commercial solutions that have been developed with Tobii Ocumen — in the hope that you might recognize the problems their innovators solved and the obstacles they overcame. At the bottom of the post, you’ll find some details about the toolkit, how the filters work, and how we’ve developed the data structure to ensure an open architecture. You will also find a link to request a free trial if Tobii Ocumen is helpful to your research, application, or solution. Enjoy!

#1 Concussion assessment — NeuroSync

One of our first customers to use Tobii Ocumen was NeuroSync (formerly SyncThink). As Scott Anderson (chief clinical officer) points out in a 2021 interview finding the keys to scalability, one of the main advantages of VR is mobility. NeuroSync has developed a solution — EYE-SYNC® — that helps clinicians assess traumatic brain injury (TBI). VR not only brings the solution directly to the sidelines of the football field or hockey rink, its closed environment makes it ideal for helping people ignore external influences, which can be intense at live sports events. Crucially, Anderson mentions the accurate and systematic measurement of eye movements and how they used the high-level filters included in Tobii Ocumen to ease the work of transforming raw eye tracking data into reliable biomarkers.

Tobii Ocumen makes it easier for us to run our software and do our analysis. Everything is being filtered for us. All the high-frequency gaze data are captured, processed, and filtered — a step we hadn't previously been able to utilize. When the data gets to us, it's ready for our implementation. It's a tremendous time-saver, and it puts the raw data into a consumable format.
Scott Anderson, chief clinical officer, NeuroSync

#2 Cognitive performance — REACT Neuro and Aperion Life

As the global population ages, the significance of cognitive performance is gaining attention as we attempt to ensure quality of life and enable therapies for neurodegenerative diseases such as Alzheimer’s, ALS, and Parkinson’s. REACT Neuro is a Cambridge-based platform developer who has been working on shifting traditional cognitive tests into VR. In a trial to determine the effects of a healthy lifestyle on cognitive performance, Apeiron Life (a Silicon Valley startup) used REACT Neuro’s VR simulation of the classic paper-based trail test.

The application leverages advanced biometrics from Tobii Ocumen to reveal saccades and fixations that cannot be observed manually. Our data robustness ensures that performance can be tracked over time.

Since the trial’s conclusion both companies have been enthusiastic about the potential for deep metrics generated by advanced sensor technologies like eye tracking, motion tracking, spirometers, EEG, and thermographic sensors and how the macro trend of wellbeing will ensure that some of these advanced technologies will trickle down to the mainstream.

Gif
In this classic trail test, our technology reveals the person’s gaze pattern and how long they dwell on each subtask — the larger the circles (in the rightmost image), the longer person needs to think about what number comes next. In manual testing, only time to completion can be measured.
Extracting the high-resolution data associated with both eye and hand movements is simply not possible with traditional manual observation. The new technology provides potential to see how the brain works in unique and hopefully meaningful ways.
Dr. David Martin, chief scientist, Apeiron Life

You can read the full story in this blog post: Cognitive performance through decades of life.

#3 VR-based training — Cineon Training

Figures vary, but some sources forecast a global deficit of up to 80,000 trained pilots in the coming decade. The shortage is due to a multitude of factors, layoffs and early retirement during covid, fewer military pilots (causing a drop in availability for civil aviation), post-covid recovery of the travel industry, and fully booked simulation facilities, which are expensive to build and hourly rates are soaring.

Cineon Training, a UK-based company, has developed a VR-based application (TACET) to build communication competency and accelerate skills acquisition. It uses eye tracking and advanced signals from Tobii Ocumen to improve the efficiency of time spent in the simulator.

VR image inside aircraft cockpit
Image courtesy of Cineon Immersive Training

TACET uses Tobii Ocumen on a Pico Neo 3 Pro Eye — untethered VR headset equipped with Tobii’s eye tracking — connected to a Microsoft Azure cloud solution for uploading and reviewing recordings. The image shows how an expert carries out a task (the green lines) compared with a novice (white lines).

One of the main takeaways is the impact eye tracking has on the robustness of skills learned. And we’ve heard this from other partners as well. Eye tracking provides a window into the mind. It allows us to assess emotions and feelings and speeds up learning. When acquiring a new skill, being able to see how your eyes move around (what you focus on) and compare with how an expert does the job helps solve the cognitive problem. It also allows people who are already good at something to refine their solutions.

Eye tracking and VR are the primary research tools our lab uses to develop innovative tools for training in aviation and other high-risk industries. Tobii Ocumen allows us to develop our unique tools, much more effectively, spending more time on our core business, and less on programming.
Dr. Sam Vine, co-founder of Cineon Training and associate professor at University of Exeter

#4 Digital transformation in MedTech — Olleyes

As a digital tool that delivers insights about people, eye tracking helps to transform traditional care based on manual observation into the digital sphere. As I mentioned earlier, in NeuroSync’s concussion assessment, VR brings the bonus of mobility, often leading to highly flexible solutions.

Olleyes have used Tobii Ocumen on a Pico Neo 3 Pro Eye VR headset to deliver a comprehensive assessment of the visual field, analyzing retinal sensitivity, for example, for patients with glaucoma. They have developed a suite of eye tests that a person can take rapidly (minutes instead of hours) either lying down or sitting up — increasing the solution’s flexibility and how many people a clinic can assess daily. Like many others in MedTech, Olleyes seeks to make solutions more convenient by enabling the shift-to-the-home.

If you want to know more about our collaboration with Olleyes, I recommend this VRARA Healthcare forum talk with my colleague Amanda Bentley and Olleyes CEO Alberto Gonzalez Build more accessible and effective vision care.

Our VisuALL Virtual Reality Platform (VRP) provides automatic, objective, and standardized visual function tests. Our partnership with Tobii allows us to increase the accuracy of our technology even more. Tobii Ocumen has helped us develop and deploy our product offerings and is a step forward in automating the assessment of visual function.
Alberto Gonzalez, CEO, Olleyes

#5 Pilot training – VTR

To be able to fly a new aircraft type, say a Boeing 737 or an Airbus A320, pilots need a type rating certificate (a license). To learn the position and functionality of all the instruments, they typically use what’s known as “paper tigers” — mockups of the cockpit. Tigers can be advanced touchscreen setups, but they tend to be costly and fail to provide a realistic sense of the 3D space. This situation isn’t ideal for pilots who typically have about 2-3 weeks to learn a new aircraft type and what every light, button, switch, and gauge in the cockpit means. The manual for a Boeing 737, for example, is about 2,000 pages.

To overcome the poor ecological validity of paper tigers, VTR has developed a VR simulation, accurately replicating the cockpit environment to ensure robust skills acquisition. Crucially, the company collaborated with JetBlue — one of the largest airlines in the US — to gain use case data to support ROI.

For a deep dive into VTR, I recommend this AWE talk with my colleague Johan Bouvin and Rick Parker, one of VTR’s co-founders Build disruptive pilot training with XR and eye tracking.

About Tobii Ocumen

These five highly innovative use cases are commercially viable because they leverage the closed nature of VR, its mobility, and the realism offered by the medium to deliver a cost-efficient solution. And all have been developed using Tobii Ocumen.

So, what does Tobii Ocumen bring to the table that enables the development of such applications?

Advanced signals and filters

Tobii Ocumen offers advanced eye tracking signals, including:

  • Gaze vectors (left and right eye separated).
  • Pupil diameter (left and right eye separated).
  • High-precision system and device timestamps.
  • Metadata, such as eye tracking hardware, firmware, and runtime software versions.

The filter library, which our customers seem to really appreciate, provides research-based, pre-validated filter components for detecting eye movements such as fixations, saccades, and smooth pursuit. These filters transform the data captured by our eye tracking hardware into building blocks of standard biomarkers.

Calibration and positioning

To ensure Tobii Ocumen delivers quality data and images display correctly, VR headsets need to sit correctly on the user’s head, and our eye tracking needs to be calibrated for the individual.

To ease the process of calibration and positioning, we’ve developed some cool features that you can embed directly into your application so that users don’t have to follow the awkward steps of quitting your application, starting a separate calibration/positioning process, and resuming.

Calibration screen with 5 points and validation check
Eye tracking calibration for embedding into an application
Tobii Ocumen positioning guide
A simple guide for headset positioning

Recording and replay

To facilitate development, Tobii Ocumen combines context data, such as 3D object movements and user-view video, with sensor data from advanced eye tracking and controller events into a single format.

We’ve included some ready-made recorders and templates for common research study paradigms that you can quickly integrate into your Unity project. And to help you replay, we’ve developed Tobii Ocumen Studio (see video below), which presents recorded data (context and sensor) in an easy-to-understand visual format.

Data integrity

In the recent development of Tobii Ocumen, we’ve put some effort into the data structure, optimizing for speed and file size and creating a flexible product architecture with an open I/O interface. The product supports data conversion, and you can easily import data gathered by any device or machine, regardless of the programming language you prefer.

I think our deep relationship with the scientific research community is worth mentioning. Because of it, we are keenly aware of the need for scientific-grade validation data in commercial solutions that require regulatory certification. Our validations include population coverage, accuracy, precision, and trackability performance.

AWE USA 2023 - Healthcare & Wellness: Disrupting Healthcare, Training, and More

AWE USA 2023 - Healthcare & Wellness: Disrupting Healthcare, Training, and More

Useful links

Request a free trial of Tobii Ocumen

We offer a 14-day free trial for qualified business to business (B2B) requests, not individuals.

    Written by

    • Headshot of Robert Malmström

      Robert Malmström

      Product Manager Tobii Ocumen

      Hi, I am the product manager for Tobii Ocumen, which means I get to work closely with our development teams and customers to determine when and what features we should create or improve. It’s my job to ensure that our products are functional and intuitive without creating a burden for developers. I believe in the enjoyment factor. It should be fun and easy to make XR applications while getting the most out of a headset that includes our technology.

    Related content

    Subscribe to our blog

    Subscribe to our stories about how people are using eye tracking and attention computing.