UX Accountability for Successful Everyday AI

Most news stories about AI focus on the technology and how revolutionary its functionality is. We marvel at what it does and its sometimes superhuman abilities. And we demonize it as a technology when it goes wrong -- when it exhibits bias or fails at achieving its task. 

While it’s important to scrutinize the technology, a more nuanced approach considers  how humans and machines interact. How do we ensure we’re using these capabilities for human advancement? When we incorporate AI into an application or system, how do people interact with, experience, use, or employ the AI for their benefit?

These questions are directly connected to how UX designers can work alongside partners like data scientists to develop more human-centered AI systems. To take a closer look, let's consider the realm of “Everyday AI.” What I mean by Everyday AI is the machine intelligence that underlies or is integrated into many of our everyday experiences. The tech field refers to this as Narrow AI -- machine intelligence that’s focused on a specific task. This includes the underlying machine learning (ML) techniques used in GPS and wayfinding, recommender systems, computer vision for facial recognition and the like. Tools with these capabilities have become such a part of our everyday lives that we might not even realize they have AI built into them. 

On the surface, an Everyday AI app might not appear all that different from a traditional, non-ML-driven app. But if we think about what’s happening on the back end, we can see that ML extends the user experience under the hood.

Think about the ways your user interacts with your product. Every click, input, interaction, time on task -- all of these are data points. The user is your data producer and creates a continuous cycle for data collection. 

Let’s take the example of mapping. When the user enters a destination address, traditional computing would find the best routes and return those options. Those routes are static, meaning, they are the same regardless of the traffic or time of day. Now let’s look at the ML version. In this case, the model learns from traffic patterns across many dimensions -- time of day, day of the week, real-time congestion, etc. Not only are the route options more dynamic, they also give us more options for the user. For example, because the ML algorithm looks at time of day, we can give the user the option to change departure time or arrival time. 

In this sense, we’re increasing the data value loop by providing more data input options to refine their output.

In machine learning, there’s a concept called “model drift,” which is when a model’s predictive power deteriorates over time. This could be caused by a number of factors such as input data or relationships between variables changing over time. Therefore the model should be retrained periodically to account for those changes. For instance, a model trained to predict housing prices in New York City might work fine until a pandemic lockdown changes the housing market dynamics. By retraining the model, it makes the performance better and ensures it makes fewer errors.

Similarly, as a person uses an AI product, their data essentially is the product. What the product does may change dynamically as it acts on that data. Think of the way a photo management app might surface different types of memories for a user. As a person grows with the AI, what they do, think, and feel may evolve. The interplay between the agentive technology and the user experience will shift. As the machine does more for the user, they may need to interact less or differently with the product.

So how do we start to design differently for Everyday AI?

• UX Should Encompass Human + AI Value Creation: We can begin to think of AI as an extension of the user. For instance, in traditional computing, the backend was likely a simple decision tree that took the user’s input and provided an output back to them. However, with AI, it performs actions on behalf of the user. Let’s take a look at self-driving cars. The model is taking camera inputs and making decisions based on perceived dangers such as lane drift or an obstacle. In this sense, the AI is extending the capabilities of the person. UX designers should be considering the AI’s actions under the hood as part of the human’s user experience.
• Experience Monitoring: When considering “behavior drift” we need to find ways to monitor how the AI is affecting people. One way to do this is by keeping user research always on. Even if we don’t change the UX, the UX is changing. As the AI assists the user, the product evolves as it learns and the user may change how they interact with the product. For instance, in a photo app, I don’t need to keep identifying my husband once the AI has learned what he looks like. How should the experience change to reflect that the AI is now accurate in identification? How should it change to reduce the burden from the user to the AI?
• UX Transparency and Agency: In addition to privacy laws, UX designers should always be thinking about how to give people agency over their AI experience. How can we envision AI more as assistive rather than prescriptive? Take a movie recommendation system. The AI looks at what content a person watched, how long, whether they binge watched, and so on. Should the experience of that automated recommendation system be purely passive? Or should the viewer get a say in how it works? Can they remove a movie from consideration in the AI (eg, “my husband picked it, not me”)? 
• Technical Acumen: Being responsible for the user experience, UX designers need to stay abreast of what technology capabilities are available to enhance experiences. In addition to understanding machine learning, we need to be aware of how UX can be helped or hindered by different technologies. For instance, edge computing and privacy-preserving ML can have dramatic positive impacts for users and UX professionals should be vocal advocates for the adoption of these capabilities.

As companies and individuals succeed with UX and AI, we'll begin to see how these two disciplines can harmonize to create exceptional products. Because these products revolve around data, we’ll see all disciplines become more conversant in aspects of data usage. The best data scientists and UX designers will align on the human implications of their work and will enhance the user experience through the confluence of their practices.