Designing Agentive Technology: AI That Works for People

🎀 Highlights

Similar to intelligence, agency can be thought of as a spectrum. Some things are more agentive than others.

is an internet search an example of an agent? Certainly the user states a need, and the software rummages through its internal model of the internet to retrieve likely matches. This direct cause-and-effect means that it’s more like the hammer with its practically instantaneous cause-and-effect. Still a tool.

when Google lets you save that search, such that it sits out there, letting you pay attention to other things, and lets you know when new results come in, now you’re talking about something that is much more clearly acting on behalf of its user in a way that is distinct from a tool.

Agentive technology watches a data stream for triggers and then responds with narrow artificial intelligence to help its user accomplish some goal. In a phrase, it’s a persistent, background assistant.

user gains competence. You’ll learn about these in Part II, “Doing,”

most of the design and development process has been built around building good tools, it’s instructive to compare and contrast them to good agents—because they are different in significant ways.

A good agent does a task for you per your preferences.

A valet might be the canonical model.

When the agent is working, it’s out of sight. When a user must engage its touchpoints, they require conscious attention and consideration.

design of agentive systems will often entail designing assistive aspects, but they are not the same thing.

It’s Not Conversational Agents

I think an assistant should assist you with a task, and an agent takes agency and does things for you. So “agent” and “agentive” are the right terms for what I’m talking about.

can design a better light switch when I think of it as a problem that can be solved either manually with a switch or agentively with a motion detector or a camera

Can the task be reliably performed with no user input, including preferences and goals? If it can, then why bother the user with it at

it’s tempting to think of agency within a system as a switch that gets turned on or off when necessary. But the humans involved with agentive systems will be unpracticed and unprepared to take over a system that suddenly loses an agentive member. (More about this in Chapter 9, “Handoff and Takeback.”)

Deciding when to inform the user of actions that have been taken

“Seven Cardinal Virtues of Human-Machine Teamwork:

what is welcome and suppress the rest? Think of presentation software that suppresses notifications that would distract your audience. • Finalization:

• Optimization: Can the agent observe available possibilities and pick the right one for the user’s goals? Think of your smartphone picking between cellular data, Wi-Fi, and Bluetooth, as the circumstances need. • Advising: Can agents observe tasks in progress and suggest better or alternate options? Think of Waze and Google Maps suggesting new traffic routes to riders behind a freeway accident. • Manipulation: What can the agent do on its own when it is absolutely certain it is what the user wants? Think of Gmail creating an item on your calendar when an email clearly describes it to a recipient. • Inhibition: Can the agent understand enough of the context to know what is welcome and suppress the rest? Think of presentation software that suppresses notifications that would distract your audience. • Finalization: What can the agent end or close when it is no longer in use? Think of your phone or desktop going to “sleep” to save on energy.

examine what makes agentive tech interesting and valuable across lots of examples, and then wrapping it all up with a history of the concepts involved in agentive design.

How do we evaluate the success of an agent?

Most computer-related fields have some version of the following diagram. Lisanne Bainbridge calls it “Monitor → Diagnose → Operate” with academically appropriate Latinate language. Computer programming texts refer to it from the computer’s perspective as “Input → Processing → Output.” I prefer the simplicity of the Germanic labels “See → Think → Do.”

and responding in a conversation with the other, but more to our purposes, that

Notably, the person is always the primary problem solver, with the agent augmenting and supporting them. This augmented-human idea isn’t new. It goes at least as far back as 1960, when J.C.R. Licklider described such a system as “Man-Computer Symbiosis.”

Tuning triggers and behaviors such that they perform better in the future.

Seeing The agent needs to be able to sense everything it needs in order to perform its job at least as well as the user,

This can be as simple as awareness of the calendar or a fixed pattern for sweeping a floor,

Allow a computer to predict within a range of confidence variable outcomes based on a set of givens, and act according to its confidence in a particular outcome happening.

Predictive algorithms: Allow a computer to predict within a range of confidence variable outcomes based on a set of givens, and act according to its confidence in a particular outcome happening.

Machine learning: Enables a computer to identify patterns in data, as well as improve its performance of a task to be more effective toward a goal.

Trade-off analytics: Can make a recommendation for users balancing multiple objectives, even with many factors.

Prediction: By comparing individual cases against past examples, algorithms can predict

Prediction: By comparing individual cases against past examples, algorithms can predict what will likely happen next. This can be as small as the next letters in an incomplete word, next words in an incomplete sentence, or as large as what a user is likely to do or take interest in next.

than building smart tools for users to operate—requires some new approaches compared to traditional product design.

Designing systems that act on behalf of their users—rather than building smart tools for users to operate—requires some new approaches compared to traditional product design.

Designing systems that act on behalf of their users—rather than building smart tools for users to operate—requires some new approaches compared to traditional product design. • You must reconceive see-think-do loops with the computer system doing the heavy lifting and the human’s occasional involvement. • This fact means that there are new use cases to consider when designing scenarios. (More on these in subsequent chapters.) • You must keep abreast of new technologies that allow the agent to do its seeing, thinking, and doing.

Chuck has become inspired to take up gardening for himself. The trouble is, he has a brown thumb, a busy schedule, and a lot of pride. He’s not going to ask her, and needs a lot of help to keep up. That’s where Mr. McGregor can step in to help.

Here are Chuck’s goals: • Have a gorgeous garden he can show off to his sister. • Have fresh veggies and herbs on hand for cooking.

Another way to implicitly convey preferences is to have the agent watch how a task is currently being performed.

If you’re familiar with If This . . . Then That, then you’re familiar with this structure since this is their whole model and, yes, their name.

there are around 1,500 books published in my mother tongue around the world every day. That’s one every 57.6 seconds. Even if only one in 10,000 of them is truly amazing, that means there’s a new one to add to the stack each week. There’s just no way to keep up.

We’re all under pressure to do more and more with the time we have. If it’s not an existential bony finger reminding us to carpe every diem, it’s just the nature of the world to tell you that you should be doing more.

should be looking at your finances more, meditating more, getting outside and exercising more. Sleeping more.

If you’re one of their mythical average Americans, you dedicate all of 16—count-em—16 minutes of time to relax and think each day.1 Even if you try to carve time away from the optional activities like television and movies, it’s quite likely some of the non-optional activities like sleeping and household chores could easily expand to consume the excess.

External time pressures

First, I want you to stop seeing it as a collection of tools or gadgets and instead see it as an evolutionary flow around human problems, whose parts ultimately integrate to become a new category of thing.

we’ll move our focus from the present into the speculative future to see how this new category of technology will change the world.

Fireplaces, furnaces, and furnace doors allow some manual control of warm air. Even architectural features like curtains, doors, and windows act as simple tools that help control the flow of air, keeping the comfortable air on you and the uncomfortable air at bay. These tools allowed you to physically work to control the temperature. That was on top of all of the labor required by the rest of living. Sometimes it was too much. (Get up and bar the door!) Such was the poor state of human thermal management until the Mennonites brought us a Dutch inventor by the name of Cornelis Drebbel.

Born in Alkmaar, North Holland in 1572, Drebbel was the fair-haired, handsome son of a landowner (or farmer, history is vague on the details).1 At an early age, he was sent to apprentice under the engraver Hendrick Goltzius, whose interest in alchemy rubbed off on the young pupil. Alkmaar at the time was also home to a large group of Mennonite scientists and inventors,

warming air would cause a column of mercury to slowly rise until it would close a damper on the heat source. This closure allowed the temperature to drop back down slowly, lowering the mercury and reopening the damper.

aforementioned leap of thermostat evolution is, of course, the Nest Learning Thermostat.

the Nest Thermostat connects to the home network to know the home’s location and the calendar date. It knows the humidity and current weather. It knows how long it takes to change your home’s temperature.

From Tool to Agent The palm frond and fan are tools that let you cool yourself. Drebbel’s incubator was a system that let you set a threshold to run against until further notice. The Nest Thermostat acts as a personal temperature regulator. It is as if you had hired a wise and happy butler to stand there at the thermostat, using everything that he knows about the clock, public and personal calendars, as well as the general preferences and statuses of the people in the house,

agents are a natural solution to a great many computable human problems, as designers attempt to reduce effort and maximize results.

technology can also help us with the information work involved in a task. Of course, thermometers give a user some

technology can also help us with the information work involved in a task.

Physical and Information Work Together to Become Agentive

It all becomes agentive with the introduction of x.ai. Subscribers to this meeting scheduler only need to CC “Amy Ingram” (we see what you did there, x.) in an email asking her to “find us a time to meet” and “she” handles the rest.

X.ai finds good times in your calendar, suggests them to the other people, works through conflicts, and lets you know when a fitting time has been found and a calendar reminder has been added to your calendar.

Recently, Google Inbox released its Smart Reply, which parses incoming emails and provides several short, likely responses from which the user can simply select.

In the products I’ve designed and workshops I’ve run, it’s been clear that only very simple agents will be purely agentive or purely assistive. Sophisticated products will move between these modes, depending on the confidence of their algorithms, the context, and user needs.

Suggestions One of the nifty promises of agents is that they can aid in the discovery of new opportunities.

helps you find a better route, or reminds you of something. Speaking of promises, if the promise of an agent is to do work on the user’s behalf, suggestions run a grave risk of bothering the user and

helps you find a better route, or reminds you of something. Speaking of promises, if the promise of an agent is to do work on the user’s behalf, suggestions run a grave risk of bothering the user and undermining the main point, so they need to be handled very gingerly, and only delivered where the confidence of value is very high.

offer an opt-out or controls for frequency of such contact.

What to do if the agent is still working fine, but it hasn’t run into any triggers? Does it just fade away? If the user hasn’t interacted with the agent for a while, it can reach out to confirm that it’s still running.

notification is an opportunity to remind the user of the value they receive from the agent and even the brand promise. My Roomba named Rusty has a bright and goofy little tune it plays when it makes it back to the charging station after a successful run and if I’m around to hear it,

come in the form of tuning its instructions—either

• When the agent runs into trouble executing its instructions, users need ways to save the individual

x.ai (2016) Once you grant access to your calendar(s), you can CC “Amy Ingram” and have the agent find the best time for a meeting between people with busy schedules. She provides options to your attendees and confirms the ones they’ve chosen. She will contact you again if there’s a problem or after the meeting has been scheduled, and, of course, make a calendar reminder as well.

You’ll note that, most of the time a good agent just does its thing and comes back with results. That does not mean that good agentive technology is free of the unbearable lightness of interface. Actually, they are chock-full of interfaces: interfaces for signing up and setting up. Interfaces for testing and launching, receiving notifications, and handling exceptions

For In-Progress Agents For designs-in-progress, designers can build experience prototypes to test with candidate users. These can be paper-and-pen prototypes for the traditional interface components, and person-behind-the-curtain for the agentive aspects. Either can be thrown together very rapidly.

User confidence: How confident is the user in the agent? Does the user feel the agent is informative and

User confidence: How confident is the user in the agent? Does the user feel the agent is informative and polite?

Similar to intelligence, agency can be thought of as a spectrum. Some things are more agentive than others.

is an internet search an example of an agent? Certainly the user states a need, and the software rummages through its internal model of the internet to retrieve likely matches. This direct cause-and-effect means that it’s more like the hammer with its practically instantaneous cause-and-effect. Still a tool.

when Google lets you save that search, such that it sits out there, letting you pay attention to other things, and lets you know when new results come in, now you’re talking about something that is much more clearly acting on behalf of its user in a way that is distinct from a tool.

Agentive technology watches a data stream for triggers and then responds with narrow artificial intelligence to help its user accomplish some goal. In a phrase, it’s a persistent, background assistant.

user gains competence. You’ll learn about these in Part II, “Doing,”

most of the design and development process has been built around building good tools, it’s instructive to compare and contrast them to good agents—because they are different in significant ways.

A good agent does a task for you per your preferences.

A valet might be the canonical model.

When the agent is working, it’s out of sight. When a user must engage its touchpoints, they require conscious attention and consideration.

design of agentive systems will often entail designing assistive aspects, but they are not the same thing.

It’s Not Conversational Agents

I think an assistant should assist you with a task, and an agent takes agency and does things for you. So “agent” and “agentive” are the right terms for what I’m talking about.

can design a better light switch when I think of it as a problem that can be solved either manually with a switch or agentively with a motion detector or a camera

Can the task be reliably performed with no user input, including preferences and goals? If it can, then why bother the user with it at

it’s tempting to think of agency within a system as a switch that gets turned on or off when necessary. But the humans involved with agentive systems will be unpracticed and unprepared to take over a system that suddenly loses an agentive member. (More about this in Chapter 9, “Handoff and Takeback.”)

Deciding when to inform the user of actions that have been taken

“Seven Cardinal Virtues of Human-Machine Teamwork:

what is welcome and suppress the rest? Think of presentation software that suppresses notifications that would distract your audience. • Finalization:

• Optimization: Can the agent observe available possibilities and pick the right one for the user’s goals? Think of your smartphone picking between cellular data, Wi-Fi, and Bluetooth, as the circumstances need. • Advising: Can agents observe tasks in progress and suggest better or alternate options? Think of Waze and Google Maps suggesting new traffic routes to riders behind a freeway accident. • Manipulation: What can the agent do on its own when it is absolutely certain it is what the user wants? Think of Gmail creating an item on your calendar when an email clearly describes it to a recipient. • Inhibition: Can the agent understand enough of the context to know what is welcome and suppress the rest? Think of presentation software that suppresses notifications that would distract your audience. • Finalization: What can the agent end or close when it is no longer in use? Think of your phone or desktop going to “sleep” to save on energy.

examine what makes agentive tech interesting and valuable across lots of examples, and then wrapping it all up with a history of the concepts involved in agentive design.

How do we evaluate the success of an agent?

Most computer-related fields have some version of the following diagram. Lisanne Bainbridge calls it “Monitor → Diagnose → Operate” with academically appropriate Latinate language. Computer programming texts refer to it from the computer’s perspective as “Input → Processing → Output.” I prefer the simplicity of the Germanic labels “See → Think → Do.”

and responding in a conversation with the other, but more to our purposes, that

Notably, the person is always the primary problem solver, with the agent augmenting and supporting them. This augmented-human idea isn’t new. It goes at least as far back as 1960, when J.C.R. Licklider described such a system as “Man-Computer Symbiosis.”

Tuning triggers and behaviors such that they perform better in the future.

Seeing The agent needs to be able to sense everything it needs in order to perform its job at least as well as the user,

This can be as simple as awareness of the calendar or a fixed pattern for sweeping a floor,

Allow a computer to predict within a range of confidence variable outcomes based on a set of givens, and act according to its confidence in a particular outcome happening.

Predictive algorithms: Allow a computer to predict within a range of confidence variable outcomes based on a set of givens, and act according to its confidence in a particular outcome happening.

Machine learning: Enables a computer to identify patterns in data, as well as improve its performance of a task to be more effective toward a goal.

Trade-off analytics: Can make a recommendation for users balancing multiple objectives, even with many factors.

Prediction: By comparing individual cases against past examples, algorithms can predict

Prediction: By comparing individual cases against past examples, algorithms can predict what will likely happen next. This can be as small as the next letters in an incomplete word, next words in an incomplete sentence, or as large as what a user is likely to do or take interest in next.

than building smart tools for users to operate—requires some new approaches compared to traditional product design.

Designing systems that act on behalf of their users—rather than building smart tools for users to operate—requires some new approaches compared to traditional product design.

Designing systems that act on behalf of their users—rather than building smart tools for users to operate—requires some new approaches compared to traditional product design. • You must reconceive see-think-do loops with the computer system doing the heavy lifting and the human’s occasional involvement. • This fact means that there are new use cases to consider when designing scenarios. (More on these in subsequent chapters.) • You must keep abreast of new technologies that allow the agent to do its seeing, thinking, and doing.

Chuck has become inspired to take up gardening for himself. The trouble is, he has a brown thumb, a busy schedule, and a lot of pride. He’s not going to ask her, and needs a lot of help to keep up. That’s where Mr. McGregor can step in to help.

Here are Chuck’s goals: • Have a gorgeous garden he can show off to his sister. • Have fresh veggies and herbs on hand for cooking.

Another way to implicitly convey preferences is to have the agent watch how a task is currently being performed.

If you’re familiar with If This . . . Then That, then you’re familiar with this structure since this is their whole model and, yes, their name.

there are around 1,500 books published in my mother tongue around the world every day. That’s one every 57.6 seconds. Even if only one in 10,000 of them is truly amazing, that means there’s a new one to add to the stack each week. There’s just no way to keep up.

We’re all under pressure to do more and more with the time we have. If it’s not an existential bony finger reminding us to carpe every diem, it’s just the nature of the world to tell you that you should be doing more.

should be looking at your finances more, meditating more, getting outside and exercising more. Sleeping more.

If you’re one of their mythical average Americans, you dedicate all of 16—count-em—16 minutes of time to relax and think each day.1 Even if you try to carve time away from the optional activities like television and movies, it’s quite likely some of the non-optional activities like sleeping and household chores could easily expand to consume the excess.

External time pressures

First, I want you to stop seeing it as a collection of tools or gadgets and instead see it as an evolutionary flow around human problems, whose parts ultimately integrate to become a new category of thing.

we’ll move our focus from the present into the speculative future to see how this new category of technology will change the world.

Fireplaces, furnaces, and furnace doors allow some manual control of warm air. Even architectural features like curtains, doors, and windows act as simple tools that help control the flow of air, keeping the comfortable air on you and the uncomfortable air at bay. These tools allowed you to physically work to control the temperature. That was on top of all of the labor required by the rest of living. Sometimes it was too much. (Get up and bar the door!) Such was the poor state of human thermal management until the Mennonites brought us a Dutch inventor by the name of Cornelis Drebbel.

Born in Alkmaar, North Holland in 1572, Drebbel was the fair-haired, handsome son of a landowner (or farmer, history is vague on the details).1 At an early age, he was sent to apprentice under the engraver Hendrick Goltzius, whose interest in alchemy rubbed off on the young pupil. Alkmaar at the time was also home to a large group of Mennonite scientists and inventors,

warming air would cause a column of mercury to slowly rise until it would close a damper on the heat source. This closure allowed the temperature to drop back down slowly, lowering the mercury and reopening the damper.

aforementioned leap of thermostat evolution is, of course, the Nest Learning Thermostat.

the Nest Thermostat connects to the home network to know the home’s location and the calendar date. It knows the humidity and current weather. It knows how long it takes to change your home’s temperature.

From Tool to Agent The palm frond and fan are tools that let you cool yourself. Drebbel’s incubator was a system that let you set a threshold to run against until further notice. The Nest Thermostat acts as a personal temperature regulator. It is as if you had hired a wise and happy butler to stand there at the thermostat, using everything that he knows about the clock, public and personal calendars, as well as the general preferences and statuses of the people in the house,

agents are a natural solution to a great many computable human problems, as designers attempt to reduce effort and maximize results.

technology can also help us with the information work involved in a task. Of course, thermometers give a user some

technology can also help us with the information work involved in a task.

Physical and Information Work Together to Become Agentive

It all becomes agentive with the introduction of x.ai. Subscribers to this meeting scheduler only need to CC “Amy Ingram” (we see what you did there, x.) in an email asking her to “find us a time to meet” and “she” handles the rest.

X.ai finds good times in your calendar, suggests them to the other people, works through conflicts, and lets you know when a fitting time has been found and a calendar reminder has been added to your calendar.

Recently, Google Inbox released its Smart Reply, which parses incoming emails and provides several short, likely responses from which the user can simply select.

In the products I’ve designed and workshops I’ve run, it’s been clear that only very simple agents will be purely agentive or purely assistive. Sophisticated products will move between these modes, depending on the confidence of their algorithms, the context, and user needs.

Suggestions One of the nifty promises of agents is that they can aid in the discovery of new opportunities.

helps you find a better route, or reminds you of something. Speaking of promises, if the promise of an agent is to do work on the user’s behalf, suggestions run a grave risk of bothering the user and

helps you find a better route, or reminds you of something. Speaking of promises, if the promise of an agent is to do work on the user’s behalf, suggestions run a grave risk of bothering the user and undermining the main point, so they need to be handled very gingerly, and only delivered where the confidence of value is very high.

offer an opt-out or controls for frequency of such contact.

What to do if the agent is still working fine, but it hasn’t run into any triggers? Does it just fade away? If the user hasn’t interacted with the agent for a while, it can reach out to confirm that it’s still running.

notification is an opportunity to remind the user of the value they receive from the agent and even the brand promise. My Roomba named Rusty has a bright and goofy little tune it plays when it makes it back to the charging station after a successful run and if I’m around to hear it,

come in the form of tuning its instructions—either

• When the agent runs into trouble executing its instructions, users need ways to save the individual

x.ai (2016) Once you grant access to your calendar(s), you can CC “Amy Ingram” and have the agent find the best time for a meeting between people with busy schedules. She provides options to your attendees and confirms the ones they’ve chosen. She will contact you again if there’s a problem or after the meeting has been scheduled, and, of course, make a calendar reminder as well.

You’ll note that, most of the time a good agent just does its thing and comes back with results. That does not mean that good agentive technology is free of the unbearable lightness of interface. Actually, they are chock-full of interfaces: interfaces for signing up and setting up. Interfaces for testing and launching, receiving notifications, and handling exceptions

For In-Progress Agents For designs-in-progress, designers can build experience prototypes to test with candidate users. These can be paper-and-pen prototypes for the traditional interface components, and person-behind-the-curtain for the agentive aspects. Either can be thrown together very rapidly.

User confidence: How confident is the user in the agent? Does the user feel the agent is informative and

User confidence: How confident is the user in the agent? Does the user feel the agent is informative and polite?