Thoughts on improved eye-tracking and media authoring tools

Posted on August 24, 2020

Question

In this unit, you learned about the eye-tracking (BubbleView) and media authoring (Voice Script and Pentimento) tools that Professor Frédo Durand and his team developed. These tools are examples of improved user interfaces and enable democratized user interaction by providing greater ease of use and accessibility. In your small group, engage with your peers on the following points:

– Eye-tracking, specifically, can be implemented at large scale. This article lists the demands for eye-tracking technology in various fields, such as gaming, medicine, and advertising. However, users have also experienced marketing algorithms as intrusive. Based on such user feedback, do you think interfaces with eye-tracking technology will be embraced or rejected? Why do you think so?

– Consider BubbleView and the mouse-contingent, moving-window methodology that it applies. Discuss other proxy-based methodologies that allow for the gathering of large amounts of less sensitive data. It can be a methodology used in any industry or field.

– Reflect on the media authoring tools covered in this unit and how they have improved user experience. Identify the advantages that these improvements can inspire for media authors in specific fields or industries.

Response

Eye-Tracking

The main issue with eye-tracking is the same as with all the different flavors of tracking: consent, or the lackthereof.

When a technology implementation is well-explained and allows users to freely choose to use it or not, then users are most likely to choose to adopt it. In the case of entertainment—e.g. games and contexts that require a VR headsets—and also in the case of augmented tools for medecine and other fields, the benefits can be made clear and understood.

There is less of a benefit to users if their actions and gaze are tracked in a shopping mall, and there is no social acceptability of using such a technology if the designers and people deploying it try to conceal it from users, even on a private property such as a mall. Rideau Hall in Ottawa, CA, faced a backlash after it was discovered they did so without the explicit consent of their customers.

Sadly, even if users explicitly give their consent to the party using tracking technology, there is a chance that additional data is leaked to third parties afterwards. It’s hard to put the blame completely on the users, as the issue with data collection is complex, usually more complex than necessary for most users.

It falls onto the designers to ensure that any implementation of such a technology is made clear to users. For consent to be truly freely given, designers must made clear all that is done with the data collected. As this is still a long ways away, I would say that it is likely that that this kind of tech will be implemented and deployed, but that there will be more occasions for backlash and leaks before any change in behaviour occurs.

BubbleView

Pr. Durand discovered a weakness in a current technology, and developed an alternative that gathered data via a proxy. Another way to proxy data to a less sensitive format is to not send or store the data anywhere, but rather only store the results obtained with the data on the spot.

A good example of this could be sound recognition. In order to be quick when comparing sounds, the recorded sample is usually converted to a spectrum graph, which is then saved as an image. In a current implementation—e.g. how the app Shazam does it—the image is sent to a server, which then compares the image with the ones in its database, and returns the most likely match. In such a scenario, the recorded sound could contain sensitive information—e.g. a private conversation—and the image could be decoded back into its containing sound, thus potentially exposing sensitive data.

Instead, the comparison should be done locally on the machine that generated the image, and if needed, the match possibility percentage could be what is saved in the remote database.

Understandably, this example may not work, since having a database local to users’ phones would be too big, and storing the possibility percentage of image match would mean nothing in this case. However, the principle of proxying how the data is transported and stored could help reducing large amount of data transfers and reduce data leaks.

Media Authoring Tools

Concerning Voice Script, I have a hard time believing that the film and voice recording industry would use this flow. The way voice actors work is by having someone at the studio console control when to punch in the recording, and the actor is elsewhere in a sound booth, doing their acting job. This is demonstrated in How Pokémon Is Dubbed From Japanese To English, in which voice actor Sarah Natochenny described this process and why she must be free to follow or go off-script. Voice Script seems to be meant for a one-person team, which is not scalable to a professional setting.

However, Pentimento seems to be more promising. There are already many platforms for online classes, and there are many people who animate conference presentations—e.g. RSA Animates—and it is possible that remote teaching will gain popularity with the rise of remote work currently happening. This tool would be appropriate in such a case.