How AI Is Making Prediction Cheaper

Avi Goldfarb, a professor at the University of Toronto’s Rotman School of Management, explains the economics of machine learning, a branch of artificial intelligence that makes predictions. He says as prediction gets cheaper and better, machines are going to be doing more of it. That means businesses — and individual workers — need to figure out how to take advantage of the technology to stay competitive. Goldfarb is the coauthor of the book Prediction Machines: The Simple Economics of Artificial Intelligence.

Download this podcast

CURT NICKISCH: Welcome to the HBR IdeaCast, from Harvard Business Review. I’m Curt Nickisch, in for Sarah Green Carmichael.

YOUTUBE: [Two women speaking] We’ve got this all tabbed up? Yup, it’s all tabbed up. OK, dialing. [Phone rings]

CURT NICKISCH: There’s a YouTube video with millions of views. In it, three young English-speaking women use Google Translate to order food in Hindi from an Indian restaurant. They copy and paste their order in English into the computer, and it translates items like “samosas” and reads them aloud in the foreign language.

At one point, they give their address for the delivery. And the worker asks – in Hindi — if they want anything else. They don’t know what he’s saying, so they give their address again.

YOUTUBE: [Man speaking over the phone in Hindi]

CURT NICKISCH: Despite the temporary miscommunication, when the order shows up, it’s correct.

YOUTUBE: [Women speaking] We got two basmati rice. Three samosas. Two fish curry! Here we go!

CURT NICKISCH: What’s remarkable about this video is that it is eight years old. Since then, Google Translate has gone from translating word by word to processing more at the sentence level. Pretty soon, you won’t have to copy and paste into a search box anymore.

AVI GOLDFARB: And you’ll be able to put something in your ear or have something on your phone and get instant translations for whatever language anywhere in the world and understand what people are talking about.

CURT NICKISCH: That’s our guest today, Avi Goldfarb. For him, one of the most mind-blowing uses of artificial intelligence technology is machine translation.

AVI GOLDFARB: It reads like language, and that change has helped me recognize that this is something that’s possible. It can really happen.

CURT NICKISCH: Goldfarb is a professor of marketing at the University of Toronto’s Rotman School of Management. And he’s the coauthor of the book Prediction Machines: The Simple Economics of Artificial Intelligence.

Avi, thanks so much for talking with the HBR IdeaCast.

AVI GOLDFARB: Thanks for having me here.

CURT NICKISCH: So, besides translation, what’s just another big example of how you think machine learning is setting us up for big changes in how we do business?

AVI GOLDFARB: Another thing that I think —  a technology that I’ve seen a few times — hasn’t, still not perfected but might get there soon, is this idea that you can see what something will look like on you online.

CURT NICKISCH: Right.

AVI GOLDFARB: So, Amazon has been trying that; a few people have been trying that. It’s still not close to what the experience is in a real store, a physical store, but the technology is getting better and a lot better very quickly. So, when I — I remember I worked with a startup a couple years ago that was trying to do something along these lines, and it was not close, and they didn’t get a lot of traction, for obvious reasons. Now I actually see Amazon and others trying to commercialize, recognizing that it’s imperfect, and with that trajectory, I can see in a few years we’re going to have something very different where we can really do and understand what things are gonna look like on us without having to go to a store.

CURT NICKISCH: Because Amazon and those other places are learning a lot right now.

AVI GOLDFARB: Because they’re learning a lot, and they’re investing in the technology, and the technology is getting better so that it can predict what something would look like on a human that has never been on before.

CURT NICKISCH: We’ve just talked about a few examples of just amazing ways that technology can change business and also how we consume and live our lives, and it’s just that drop in the bucket of a lot of the stuff that’s going on there. But what I found so fascinating about Prediction Machines, the book that you coauthored, is that you take what is kind of this very-hard-to-predict, sweeping trend, that it’s impossible to know exactly how it’s going to unfold, and you turn it into a basically a very simple economic model. And I’m a sucker for just like the economic argument. What is the simple economy of artificial intelligence so that we can understand it that way?

AVI GOLDFARB: Sure. So, artificial intelligence and the idea of artificial intelligence has been around for decades, and we’ve had fits and starts over the years. Starting in the fifties, if not earlier, you were talking about, oh, computers are going to actually learn to think. And it’s always been a bit of an unfair race in the sense that as soon as the computer can do something that a human can do, we no longer call it artificial intelligence. And so, there’s been the sense that artificial intelligence is what our computer can’t yet do, and once a computer can do that, then it’s, then that’s just computing and the remaining stuff is intelligence.

What’s happened in the last 10 years and especially in the last six years, is something a little bit different, which is that a particular branch of artificial intelligence called machine learning has improved a lot, to the point where a lot of things that just 10 years ago we thought of as inherently human problems can now be done by machines.

And so, in understanding why there’s been this excitement around AI in the last 10 years, if not the last two years, it’s all driven by machine learning, which is prediction technology. And so, you should think about prediction as the process of filling in missing information. So, what machines have gotten very good at is filling in missing information, is prediction. And so, I’m an economist, and so, how do I see the world? I think about, well, something’s gotten easier. We can think about that as a drop in costs. If you want to think about it as it’s gotten better, better quality, that gets you to the same place, but something for given quality is now cheaper. And as an economist, I know what happens when something gets cheaper. We all know what happens when something gets cheaper: we want to buy it more. So, as prediction gets cheaper, as prediction gets better, we’re going to do more and more and more prediction, and that prediction is gonna be done by machines.

CURT NICKISCH: Got it. And prediction, even for humans, is hard, right?

AVI GOLDFARB: It is. So, if it’s the kind of prediction that we know humans do badly, machines don’t make those kinds of mistakes, and there’s all sorts of aspects of that. So, another thing that we’ve seen is in the process of hiring. So, we know humans are pretty bad at predicting which applicants for a job are going to perform best. We make two kinds of mistakes. One kind of mistake is we’re just wrong. We think somebody’s gonna do really well, and they do really badly. We think someone else is gonna do really badly, and they end up doing really well. The other kind of mistake we make is related to our biases. We have stereotypes and people come in, we impose those stereotypes on the applicants and then assume that they’re going to be true. And we ended up not hiring people we should and hiring people we shouldn’t because of these biases.

Machines can improve on those human mistakes in both directions. So, first, the machines are just going to make fewer basic mistakes as long as they have a measure of what performance means. So, we’ve seen evidence of this in call centers. So, in call centers, the key thing that marks a good hire is somebody who can last a long time. It’s very expensive to train people up, and machines are very good at predicting tenure in the job much better than humans are. We’ve seen that from a couple of research projects, but not only that; the machines do that without some of the biases that humans have with respect to gender and race and things like that. That’s not to say that machines won’t also be biased in various ways, but it’s exactly because they’re programmed by people. So, the expectation is that they should be at least no worse than the humans who program them in terms of these issues.

CURT NICKISCH: So, let’s continue this economic model here. Prediction gets cheaper. What else is happening in this model?

AVI GOLDFARB: The way I like to think about the economics of the book is the first point is prediction is cheaper, right, and that means we’re gonna do more prediction, OK. Just like when the price of coffee falls, we buy more coffee —  nothing new there. The other thing to remember is, when the price coffee falls, we buy less tea, and so that’s the substitute story. That’s what, what are you going to have less of an organization? You’re going to have less people who do prediction. The other thing that happens, and I think the most important thing, is when the price of coffee falls, we buy more cream and sugar. And so, the core of the book is these are the compliments. What are the cream and sugar for prediction? If predictions cheaper, what becomes more valuable? What do we end up using more of and buying more of?

In the book we emphasize that data becomes more valuable. That being able to take an action — decisions aren’t useful without being able to do something about it. And perhaps most importantly, what we humans have is judgment, the ability to figure out which predictions to make and what to do with the prediction once we have it. And so, when you think about the opportunities around machine prediction, a lot of them are driven by the availability of data, the ability to take an action based on a prediction, and whether the people in charge, the people supervising the machine, can use it well enough to take advantage and help the organization.

CURT NICKISCH: So, let’s break each of those down, starting with data. What do companies and people need to know about it?

AVI GOLDFARB: Sure. So, lots of people think that if their company’s sitting on a lot of data, then data’s the new oil, and so their company’s sitting on some kind of great mine where they can make lots of money off it for years and years and years. The data-is-oil metaphor I think is really good, but it’s good because just like oil, once you’ve used your data, the data you’re sitting on, it’s used, and you’re going to extract value from it, but you’re going to extract value from it once. Say you’re a warehouse, and you’re sitting on data on past inventories and your warehouse, OK? That’s going to help you predict future inventories, and you’ll now have a better model if future inventories, and you can make money on it. But it’s only valuable for that one model, essentially. You might think of something a little bit different, but fundamentally you’ve extracted the core value from that data you’re sitting on, and it’s done. In order then to continually improve your models of inventory, you need new data on what’s coming in in order to make it better.

The key source of sustained competitive advantage from data is through we call feedback data, which is the ability to continually improve your AI. And that means you need to invest in learning. What does investing in learning mean? It means making your product potentially worse in order to improve the AI.

So, think about using Waze — it’s for driving directions — and if they learn that there’s an accident on the road somewhere and there’s a backup, the app, the software, the AI will take you a different route. And it’ll take most people a different route, but at some point Waze has to figure out if that backup, that accident has cleared, and so it has to invest and essentially sacrifice some drivers or some technology in order to figure out whether that backup has cleared. And someone has to invest in improving the AI sometimes even at the expense of the immediate customer experience.

CURT NICKISCH: Wow, that’s interesting. I had never thought of it that way. So, they may send somebody down that road and see if they get through or not.

AVI GOLDFARB: Correct. And so —

CURT NICKISCH: And that person is just like, oh, what terrible luck; an accident just happened.

AVI GOLDFARB: Or they might think Waze isn’t as good as it seemed like it was.

CURT NICKISCH: Right. OK. What about action?

AVI GOLDFARB: OK. The prediction is really only valuable if it leads to a change in behavior. So, a prediction on how much yogurt somebody’s going to sell, a prediction on a which stocks are going to go up or down — all of these things are only valuable if you can then go out and fill up your inventory, change the prices, by the stocks, these sorts of things. You know, if you’re sitting on a lot of data, we think, you’re a big incumbent company, and you might have some advantage.

But if you own the action, if you, if the customer relationship is with you and not with whatever startup is coming up with a better prediction on something you do, you have an advantage. That’s the takeaway from understanding the benefit of action.

CURT NICKISCH: OK. And now maybe the funnest one is judgment.

AVI GOLDFARB: Yeah. OK.

CURT NICKISCH: Which makes us feel good about being human again.

AVI GOLDFARB: Absolutely.

CURT NICKISCH: Yeah.

AVI GOLDFARB: So —

CURT NICKISCH Break that one down.

AVI GOLDFARB: OK, so judgment is the ability to figure out which predictions to make and what to do with the prediction once you have it. You can have a machine that gives you the best predictions in the world, but if you don’t know what kinds of predictions are gonna be valuable, or if you can’t tell the machine something of use to your organization or to you individually, then who cares? And so, what judgment is all about is knowing what you care about, what your organization cares about, and how to tell the machine to do that.

The simple example we like to use is thinking about the context of whether it’s going to rain. OK? So, you have a weather prediction about how likely it is to rain or not. And depending on whether it’s likely to rain or not, you may or may not carry an umbrella. OK? But that choice is going to depend on how much you hate getting wet and how much you hate carrying around an umbrella. And so, the prediction is rain, no rain, but the judgment is how costly that umbrella is to you to carry around versus how big a deal it will be to get wet. Now, an umbrella doesn’t sound like a very big deal, like a sort of a pedestrian decision. But that example is exactly the same as an insurance problem. So, if you think about, should you take out insurance, it is exactly the same situation: How costly is that insurance going to be is like, should you carry an umbrella? How much better off you are you if you’re protected by your insurance, you know, if something goes wrong is exactly like, well how important is it to have that umbrella with you if it starts to rain? And so, a whole bunch of decisions look like a lot like the umbrella decision. So first, you know, all the classic insurance problems, but also how much did you invest in security? It’s all about weighing risk, and these are all judgment problems. So, you can predict how risky something is going to be, but that’s only going to be useful if you know what to do with those predictions, and what better prediction allows you is to give final judgment.

CURT NICKISCH: The other message here is just that if judgment is a compliment of prediction prices going down, that’s a place where you can focus your career, aim your company, and just bring those strengths to bear.

AVI GOLDFARB: Absolutely. So, a key, a key message of the book, but also I think just the reality of the technology is if your job is prediction, then you’re going to have to rethink some of your skills and invest in learning new ones. That sounds pretty dour, and pretty bad, but it’s not in the sense that often the people who are good at prediction are going to be exactly the people who know what to do with those predictions. OK?

And to get a sense of that, I like to think about what happened with accounting as accounting move to spreadsheets. OK? So, accountants used to spend their time doing arithmetic. Accountants don’t do arithmetic anymore. Maybe they do a little tiny bit, but fundamentally that’s not part of the job, but we still have a lot of accountants. Because it turned out the people who were really good at arithmetic or exactly the kind of people who would know how to use arithmetic well to help companies deal with the tax code, manage changes, identify where profit centers can be. So that that skill set, which was arithmetic skill set, which is no longer useful, turned out to be a great baseline for learning what to do with arithmetic. And I think that analogy is going to work in lots of high skilled jobs around prediction.

CURT NICKISCH: If you’re in a company, and you’re just trying to figure out how to — how to employ people to make the best use of this technology within your firm, what’s a good way of thinking about that?

AVI GOLDFARB: What we need to think through is the workflow of a particular job in a particular process in your organization? So, increasingly driving is done by prediction. We’re not quite at autonomous driving yet, but there was an insight a while ago that we can teach machines to drive by telling them to predict what a good human driver would do. And so, over time, this is one of these prediction tasks that wasn’t an obvious prediction task 30 years ago that now we see, oh, we can do driving as prediction. And so, what’s happened is we need to think through the workflow of various people whose job is driving. And I think it’s useful to compare a, a bus driver to a school bus driver, like a long-distance bus driver to a school bus driver.

And if you look at the workflow of a long-distance bus driver, almost the entire workflow is driving. And that means that that job is in many ways likely to become automated over time. It’s not obvious what the need is for a human on the bus if what they’re doing is driving somebody from place to place or driving you from place to place. That’s different from a school bus driver, because the school bus driver, if you look at their workflow, they do a few things. I like to summarize two as one is drive, but the other is protect the children maybe from outsiders but particularly from each other. And so, even if over time the job of a school bus driver changes dramatically in the sense that they spend very little time driving, you’re still going to need some role for somebody to figure out how to do the protection side of that job.

Now, how that exactly plays out, I don’t know. One possibility is you end up with teachers on the bus, and you take advantage of that time as learning time. Another possibility is that it happens, one person remotely monitor several buses, and they have some system for protecting things, but you still need some, at least given our understanding the technology today, you’re going to need someone to do that  protection job even as driving becomes automated.

When you break down the workflow, you can see some things in some jobs are going to become completely automated, but lots of jobs, yes, certain tasks within the job are going to become automated, but there’s lots of other things that they do that our prediction tasks that a human needs to do. And potentially over time if it becomes cheap enough and easy enough, you can see even more jobs and more interesting jobs and better use of time because the mundane prediction part of the task is gone.

CURT NICKISCH: And that’s a really simple example, but you’re saying that you could do that with any job —

AVI GOLDFARB: You can do that —

CURT NICKISCH: Or any business process, business unit even.

AVI GOLDFARB: Absolutely. So, Goldman Sachs has broke down the steps to an IPO into over 100 small tasks. And when you look at each of those small tasks, you can identify which aspects of them are prediction, which aspects of them might be automated through some other technology, and which aspects require a human judgment or input of more data or an action done by a human or an action done by a machine. And so, once you break down the workflow into specific pieces, you can identify the opportunities to insert prediction machines as a tool for a particular task.

CURT NICKISCH: Especially at high-cost, high-value workflow like that one.

AVI GOLDFARB: Especially a high-value workflow like that one, absolutely. And the immediate results you might think is productivity enhancement in the sense of some people who used to have jobs won’t be doing those anymore and it will be lower cost with machines, but at the same time, other aspects of the workflow — once the bottleneck potentially becomes cheaper, we can use other aspects of the workflow more and get more value out of the humans there and even create more jobs in that part.

CURT NICKISCH: I mean, just going through that task example makes you think a lot of tradeoffs, and with tradeoffs we think of strategy. From a business unit, from a corporate strategic point of view, how do you apply this simple economics model to, to what you’re doing?

AVI GOLDFARB: The way to think about it is this, these are prediction machines and so what does prediction do? It reduces uncertainty. And so, if there’s aspects of your business that you can’t do everything you’d want to do because of uncertainty, because you have to hedge against the risk of uncertainty, then prediction machine could be transformative.

Since there’s so much uncertainty, but what an end consumer might want from a retailer, from Amazon, that Amazon has to wait for you to tell them what you want before it can ship to you. But if that uncertainty is resolved, and they know what you want, now, Amazon’s business model can change.

And so, the business model moves from a shopping-then-shipping model to a very different model, which is, they send me things I want, and they’re almost always right; and if I don’t happen to want them, then I can in principle send those very few things back.

We can do a similar example. We’ve talked a little bit about machines being better and better at, at hiring and HR. We can take that to the extreme similarly, which is that the reason we have to go through this onerous hiring process of posting a job and waiting for applications and screening through applications and interviewing and hiring is because there’s uncertainty at every stage on which of those — who’s gonna be interested in applying. If they apply, who’s going to be above some threshold and we to interview to see if they’re going to fit with culture and interest and we’re going to get along and all these things. But if we have a great prediction machine, all that uncertainty could go away to the point where we could look at the database of people who might be interested in the job, who exist in the profession —

CURT NICKISCH: And make it —

AVI GOLDFARB: And make an offer right away and skip all those intermediate steps and totally transform the industry because we’ve resolved that core uncertainty.

CURT NICKISCH: Avi, this has been great. Thank you so much for coming in and talking.

AVI GOLDFARB: Thank you very much.

CURT NICKISCH: That’s Avi Goldfarb. He’s a professor of marketing at the University of Toronto’s Rotman School of Management.

He’s also the coauthor of the book Prediction Machines: The Simple Economics of Artificial Intelligence. You can find it at HBR.org.

This episode was produced by Amanda Kersey. Adam Buchholz is our audio product manager. And we get technical and production help from Rob Eckhardt.

Thanks for listening to the HBR IdeaCast. I’m Curt Nickisch.