VRguy podcast Episode 13: Dr. Christian Lange, CSO of Ergoneers on why Eye Tracking Is Expensive

The VRguy's podcast

My guest today is Dr. Christian Lange, Chief Strategy Officer of Ergoneers. This episode was recorded on May 23rd, 2016.

Christian and I talk about why eye trackers are so expensive, about why they are difficult to build. He tells about some of the coolest applications he saw as shares his thoughts on the future of eye tracking.

http://sensics.com/wp-content/uploads/2016/05/rec_ergo_lange_23_May_2016_15_42_49.mp3

Interview transcript

Yuval Boger (VRguy): Hello Christian, and welcome to the program.

Christian Lange: How are you? Nice talking to you.

VRguy: So, Christian, who are you and what do you do?

Christian: I am founder and CSO of Ergoneers, which is a German company developing eye-tracking hardware, head-mounted eye-tracking hardware, and related software to measure and analyze eye-tracking data.

VRguy: And Ergoneers has been in business for how long?

Christian: Slightly more than 10 years now, so we founded the company in September 2005 and we just celebrated our 10th anniversary.

VRguy: Well, congratulations. At Sensics I’ve been doing this for 10 years as well and I feel like we’ve been walking the desert for so long and finally, there’s signs of water.

Christian: Yeah, seems like. To be honest, it’s a growing business, so in the early days, we sold like 5 eye trackers per year and now, it’s close to at least 200.

VRguy: Excellent. So, I got to ask you, why are eye trackers so expensive?

Christian: Okay, that’s a good question and I think lots of people want to know that. What I hear quite often, when speaking to customers or potential customers about that topic, they just show me any webcam they can buy for like 100 bucks on the internet and that camera gets them a nice image on 30 Hertz, which looks pretty okay, and then there’s some open-source source code available and they can at least get some results. But if you want to have really good results and if you want comparable data and highly precise and accurate data, then it’s a little bit more than just buying any kind of camera and using any kind of open-source software.

In order to be precise and accurate in eye tracking, you have to fine-tune several parameters. Typically, you have an IR LED, which is enlightening the pupil of the eye, then you have a band-pass filter, which sits in front of the eye camera. This band-pass filter only lets pass through the light of the infrared LED and then you have to have a camera which has a camera chip, which is sensitive in exactly that bandwidth. You have to have a special type of camera with a special type of chip and a special type of infrared filter and a special type of infrared LED.

Those three hardware components have to play nicely together, just to get a pretty good image of the pupil itself. And then you have to have good image processing algorithms, which can not only detect the pupil in perfect lighting conditions, meaning indoors, but also under any kind of lighting conditions, and also, under changing lighting conditions. Then in order to be accurate and precise, you have to have really good calibration algorithms that do not only work under perfect conditions, but also with corrective lenses, with glasses and with sunglasses.

And then, by having the calibration done, what you typically get is a video of the scene camera, which films the environment and a crosshair within that scene camera. But what a typical customer wants to know at the end is, when does the subject look at which certain area of interest? That means, you have to have a so-called data analysis method in between that calculates glances towards areas of interest for you.

What we did in our software is to fully automate that process based on markers we find in the environment, that provide us a reference in the environment, and then the user can just define areas of interest by drawing them as polygons in the view of the scene camera. Our software is automatically calculating when does the subject look at those defined areas of interest, plus provides them the related eye-tracking metrics. And those eye-tracking metrics are, again, conforming to different ISO standards.

If you see this whole chain from, you can buy a camera for 100 bucks on the internet, until you really get valid, precise, accurate, comparable eye-tracking metrics at the end, is a totally different story and this requires lots of development manpower for several months and years. This is simply pretty expensive to do. And then by just selling 200 eye trackers per year, you can divide developing costs through the numbers of eye trackers you sell per year, and then you end up having this still high price.

VRguy: Okay, so the hardware cost is not that big, but it’s really, today when you sell primarily to the professional market, it’s really to cover the significant engineering costs in developing the software. Did I understand it correctly?

Christian: Absolutely 100%. Plus you also have to get a CE Certificate. You have to get an FCC Certificate and this costs you between 10,000 to 20,000 USD per country and so this is another huge upfront investment. This simply explains the pretty high price for professional eye trackers.

VRguy: So is the interest in VR growing? In your expertise and doing the professional systems, do you find yourself being approached by consumer electronics companies that might want to use your products in consumer settings?

Christian: Yes, we did get approached by consumer product companies that want to use some eye trackers in their products. There are different kinds of applications. The ones you know are in the head-mounted displays, but we also do have something we call an Eye-Tracking Hardware Development Kit, which consists of our eye-tracking cameras, which are pretty small. And we have customers in the medical area. They want to integrate respectively they did already integrate in prototypes our eye-tracking cameras into medical devices, such as those teleoperation systems. In order to observe the medical doctor when doing some medical surgery on patients, using those teleoperation systems.

VRguy: Got it. So in the professional space, what, in terms of recent memory, what’s the coolest thing that you saw, that you’ve seen customers do with the product?

Christian: One really amazing thing was the combination of the Sensics dSight head-mounted display with the Dikablis Eye Tracker integrated used by an airplane manufacturer. What they did was, that they had ideas for new interior concepts, but the issue is that building a prototype is extremely expensive, so the idea was to have a virtual prototype that they can show to potential pilots in the head-mounted display, and by having the eye-tracking cameras inside the head-mounted display, they can already see and test if the pilots can see displays, if they can see warnings, how are they going to orient in those new-designed cockpits. Plus what they also did is so-called motion capturing of the pilots’ hands and they had some new input devices in this virtual cockpit.

And the really cool thing was that the input devices were also there in the real world but just prototypes of them that looked unspectacular and were poorly designed. But in the HMD they looked really bright and nice and shiny, and in the reality they were in the same position but they didn’t look that great. So the thing was that the pilot could see them in this bright and nice way in the head-mounted display. Also the hand was tracked, and was visible in the head-mounted display, so the pilot could interact with those new input devices and could control displays using them. And all this was visible in the virtual head-mounted display and the pilot was really thinking that he’s interacting with this newly-designed interior design of a cockpit, but in reality they were just some prototype input devices and nothing else besides that.

And all the rest was fully virtual, and the company was already able to test those new interior cockpit concepts in an extremely early stage without even having a real functional prototype, and everything was just virtual. And this is going to save them millions and millions of dollars that would normally have to be spent in actually building interactive prototypes and now this can all be done in CAD.

VRguy: Yeah, that’s very cool. So going back to the technical stuff: what’s the most difficult part in creating eye-tracking software? Is it calibration? Is it finding the pupil in varying light conditions? What do you think is the most difficult part?

Christian: What is pretty difficult is to fulfill all the customer requirements. All the single bits and pieces are there to be honest. So we have really great image-processing algorithms, the calibration is extremely precise, the whole data analysis process is automated. All the metrics are there, they are ISO-standardized, but each customer has different requirements, has different tasks to do, and they are also from different backgrounds and branches, ranging from automotive via market research, to biomechanics and sports and even professional scientists. And to cover all their requirements in one software which is still usable and intuitive for all of them is the biggest challenge, to be honest.

VRguy: How fast does eye tracking need to be, 10 Hertz? 50 Hertz? 500 Hertz? Give me a sense of how fast you think it needs to be.

Christian: That’s also a really important question you’re asking. This depends on what you want to do. If you want to know where someone is looking at in the environment, that means, did someone see the Coca-Cola on the shelf? Then 10 Hertz are enough, because just from the physics of the eye, the shortest duration of a fixation is around 100 milliseconds. The eye is not faster than this, and also the human information processing isn’t faster than this. This means 1000 divided by 100 equals 10 Hertz. This is what you need if you want to capture glances or fixations toward objects of interest.

If you want to look at things like the pure pupil movements, meaning if you want to know what’s the peak speed of the pupil while doing a so-called saccade, then you have to go a little bit higher. That means, then you have to go somewhere up to 50 Hertz. And if you want to look at things like pupil acceleration phase and pupil deceleration phase, so really to plot that over time, then you have to go 1000 Hertz and beyond that. So you really have to know what you want to do in order to select the appropriate speed of an eye tracker.

VRguy: Now in my experience as a user of eye tracking or as a company that integrates eye-tracking solutions, including of course from Ergoneers, into our products, we found that there’s really sort of a garbage-in, garbage-out principle, meaning that if the camera position is not right, or the lighting is bad, then no amount of image processing is going to help. So the critical portion is really figuring out where to correctly place the camera so it has a good view of the eye across many different types of individuals. Is that your experience as well or should I be looking elsewhere?

Christian: No, what you’re saying is 100% correct: it all starts with a good image of the eye. If the image of the eye is not good, then you can have the best image processing in the world, you will never ever find the pupil. We have someone which is now a customer of ours, and he was using a different system and he was complaining that the system is not working with eyeglasses. And I was telling him, “Look, I promise you our system is going to work with eyeglasses,” and he was betting “no, it’s not going to work”.

Then I was trying to explain it to him but he didn´t listen because he was really upset because it never worked with glasses. So I told him, “Look, I’ll show you why some systems might not work, and why our system is going to work.” Then I put on the eye tracker- I put on our eye tracker on this subject, and he was wearing the glasses, and the glasses had a really thick frame, and then in the default position of the eye cameras it didn’t work. Then he told me, “Look, I was right, I was right.” Then I told him, “Yes, you are right, and now I show you what the image of the eye camera looks like.” And the image of the eye camera was mostly the frame of his glasses. So the camera wasn’t even able to see the pupil.

So this is why it’s important that head-mounted eye trackers are adjustable. Then we slightly moved the eye camera so that it was able to film the pupil, and then there was the tracking, there was the calibration, and he was suddenly able to do eye tracking with his glasses that had a thick frame. And this is a nice example showing that it’s really key to have a good image of the pupil or to at least have an image of the pupil. And then the better the image, the better the result.

What is also important, what some people might not be aware of, that it’s not only the speed of the eye tracker, meaning how many Hertz does the camera have, but also the resolution in pixels of the eye-tracking camera. You can have an extremely fast eye-tracking camera, but if the resolution is bad, then you’re also not able to measure eye movement precisely, and then you’re also not able to capture small saccades. So this is why camera speed and camera resolution and also the lighting of the eye has to be perfectly balanced out in order to get a really good result at the end.

VRguy: So Christian, as we head towards the end of this conversation, you mentioned that a lot of the eye-tracking problems have been solved in terms of calibration, in image processing, algorithms and so on. So what do you think the future holds for eye tracking? What still needs to be solved? How do you expect your products three or four years from now to be different than what they are today?

Christian: I think there are some things that we have to be less obtrusive. What I want to have is an eye tracker running on something like more than 100 Hertz so that we’re able to also get some more information about things like peak speed of a saccade, acceleration and deceleration phase of a saccade. And also being able to measure automatically in any kind of environment where someone is looking at.

Today we solved that for controlled environments like in cars or in supermarkets, but it’s not solved yet if you walk down a pedestrian walk or just outside on the street where we cannot place our patented QR codes that help us analyze automatically where someone is looking at. In order to do this, we have to implement something which is called Natural Feature Tracking so that we can automatically identify things like ‘there’s a pedestrian, there’s a traffic sign, there’s something specific’, so that we can really classify objects out of having just a picture of them. So that someone can just put on the eye tracker and ask our software, “How often do my subjects look at a specific traffic sign?” without having to do this manually. If we can technically solve that, then this would be a huge step forward and this would greatly help our customers.

VRguy: Excellent. Christian, thank you very much. Where could people connect with you to learn more about your work?

Christian: They can of course always write me an email, it’s [e-mail]. Or they can of course visit our website, www.ergoneers.com. Or we also have a nice Facebook page, just enter Ergoneers on Facebook and you’ll find us on Facebook. We always also have some latest cool videos about eye tracking and these applications. So these are the, I would say, the three best channels to connect with us.

VRguy: Excellent. Thanks again for coming to my program.

Christian: You’re welcome, it was a pleasure to speak with you.

VRguy: Thank you.

The post VRguy podcast Episode 13: Dr. Christian Lange, CSO of Ergoneers on why Eye Tracking Is Expensive appeared first on Sensics.