\n
![](\"https:\/\/cdn.flipboard.com\/hbr.org\/d46356e0dba4c0f67067880bbdfed72e48729df3\/original.jpg\")
<\/p>\n [ad_1]
\n<\/p>\n
Avi Goldfarb,<\/strong> a professor at the University of Toronto\u2019s Rotman School of Management, explains the economics of machine learning, a branch of artificial intelligence that makes predictions. He says as prediction gets cheaper and better, machines are going to be doing more of it. That means businesses \u2014 and individual workers \u2014 need to figure out how to take advantage of the technology to stay competitive. Goldfarb is the coauthor of the book Prediction Machines: The Simple Economics of Artificial Intelligence.<\/em><\/a><\/p>\n Download this podcast<\/a><\/p>\n CURT NICKISCH: Welcome to the HBR IdeaCast, from Harvard Business Review. I\u2019m Curt Nickisch, in for Sarah Green Carmichael.<\/p>\n YOUTUBE: [Two women speaking] We\u2019ve got this all tabbed up? Yup, it\u2019s all tabbed up. OK, dialing. [Phone rings]<\/p>\n CURT NICKISCH: There\u2019s a YouTube video with millions of views. In it, three young English-speaking women use Google Translate to order food in Hindi from an Indian restaurant. They copy and paste their order in English into the computer, and it translates items like \u201csamosas\u201d and reads them aloud in the foreign language.<\/p>\n At one point, they give their address for the delivery. And the worker asks \u2013 in Hindi \u2014 if they want anything else. They don\u2019t know what he\u2019s saying, so they give their address again.<\/p>\n YOUTUBE: [Man speaking over the phone in Hindi]<\/p>\n CURT NICKISCH: Despite the temporary miscommunication, when the order shows up, it\u2019s correct.<\/p>\n YOUTUBE: [Women speaking] We got two basmati rice. Three samosas. Two fish curry! Here we go!<\/p>\n CURT NICKISCH: What\u2019s remarkable about this video is that it is eight years old. Since then, Google Translate has gone from translating word by word to processing more at the sentence level. Pretty soon, you won\u2019t have to copy and paste into a search box anymore.<\/p>\n AVI GOLDFARB: And you\u2019ll be able to put something in your ear or have something on your phone and get instant translations for whatever language anywhere in the world and understand what people are talking about.<\/p>\n CURT NICKISCH: That\u2019s our guest today, Avi Goldfarb. For him, one of the most mind-blowing uses of artificial intelligence technology is machine translation.<\/p>\n AVI GOLDFARB: It reads like language, and that change has helped me recognize that this is something that\u2019s possible. It can really happen.<\/p>\n CURT NICKISCH: Goldfarb is a professor of marketing at the University of Toronto\u2019s Rotman School of Management. And he\u2019s the coauthor of the book Prediction Machines: The Simple Economics of Artificial Intelligence<\/em><\/a>.<\/p>\n Avi, thanks so much for talking with the HBR IdeaCast.<\/p>\n AVI GOLDFARB: Thanks for having me here.<\/p>\n CURT NICKISCH: So, besides translation, what\u2019s just another big example of how you think machine learning is setting us up for big changes in how we do business?<\/p>\n AVI GOLDFARB: Another thing that I think \u2014\u00a0 a technology that I\u2019ve seen a few times \u2014\u00a0hasn\u2019t, still not perfected but might get there soon, is this idea that you can see what something will look like on you online.<\/p>\n CURT NICKISCH: Right.<\/p>\n AVI GOLDFARB: So, Amazon has been trying that; a few people have been trying that. It\u2019s still not close to what the experience is in a real store, a physical store, but the technology is getting better and a lot better very quickly. So, when I \u2014 I remember I worked with a startup a couple years ago that was trying to do something along these lines, and it was not close, and they didn\u2019t get a lot of traction, for obvious reasons. Now I actually see Amazon and others trying to commercialize, recognizing that it\u2019s imperfect, and with that trajectory, I can see in a few years we\u2019re going to have something very different where we can really do and understand what things are gonna look like on us without having to go to a store.<\/p>\n CURT NICKISCH: Because Amazon and those other places are learning a lot right now.<\/p>\n AVI GOLDFARB: Because they\u2019re learning a lot, and they\u2019re investing in the technology, and the technology is getting better so that it can predict what something would look like on a human that has never been on before.<\/p>\n CURT NICKISCH: We\u2019ve just talked about a few examples of just amazing ways that technology can change business and also how we consume and live our lives, and it\u2019s just that drop in the bucket of a lot of the stuff that\u2019s going on there. But what I found so fascinating about Prediction Machines<\/em>, the book that you coauthored, is that you take what is kind of this very-hard-to-predict, sweeping trend, that it\u2019s impossible to know exactly how it\u2019s going to unfold, and you turn it into a basically a very simple economic model. And I\u2019m a sucker for just like the economic argument. What is the simple economy of artificial intelligence so that we can understand it that way?<\/p>\n AVI GOLDFARB: Sure. So, artificial intelligence and the idea of artificial intelligence has been around for decades, and we\u2019ve had fits and starts over the years. Starting in the fifties, if not earlier, you were talking about, oh, computers are going to actually learn to think<\/em>. And it\u2019s always been a bit of an unfair race in the sense that as soon as the computer can do something that a human can do, we no longer call it artificial intelligence. And so, there\u2019s been the sense that artificial intelligence is what our computer can\u2019t yet do, and once a computer can do that, then it\u2019s, then that\u2019s just computing and the remaining stuff is intelligence.<\/p>\n What\u2019s happened in the last 10 years and especially in the last six years, is something a little bit different, which is that a particular branch of artificial intelligence called machine learning has improved a lot, to the point where a lot of things that just 10 years ago we thought of as inherently human problems can now be done by machines.<\/p>\n And so, in understanding why there\u2019s been this excitement around AI in the last 10 years, if not the last two years, it\u2019s all driven by machine learning, which is prediction technology. And so, you should think about prediction as the process of filling in missing information. So, what machines have gotten very good at is filling in missing information, is prediction. And so, I\u2019m an economist, and so, how do I see the world? I think about, well, something\u2019s gotten easier. We can think about that as a drop in costs. If you want to think about it as it\u2019s gotten better, better quality, that gets you to the same place, but something for given quality is now cheaper. And as an economist, I know what happens when something gets cheaper. We all know what happens when something gets cheaper: we want to buy it more. So, as prediction gets cheaper, as prediction gets better, we\u2019re going to do more and more and more prediction, and that prediction is gonna be done by machines.<\/p>\n CURT NICKISCH: Got it. And prediction, even for humans, is hard, right?<\/p>\n AVI GOLDFARB: It is. So, if it\u2019s the kind of prediction that we know humans do badly, machines don\u2019t make those kinds of mistakes, and there\u2019s all sorts of aspects of that. So, another thing that we\u2019ve seen is in the process of hiring. So, we know humans are pretty bad at predicting which applicants for a job are going to perform best. We make two kinds of mistakes. One kind of mistake is we\u2019re just wrong. We think somebody\u2019s gonna do really well, and they do really badly. We think someone else is gonna do really badly, and they end up doing really well. The other kind of mistake we make is related to our biases. We have stereotypes and people come in, we impose those stereotypes on the applicants and then assume that they\u2019re going to be true. And we ended up not hiring people we should and hiring people we shouldn\u2019t because of these biases.<\/p>\n Machines can improve on those human mistakes in both directions. So, first, the machines are just going to make fewer basic mistakes as long as they have a measure of what performance means. So, we\u2019ve seen evidence of this in call centers. So, in call centers, the key thing that marks a good hire is somebody who can last a long time. It\u2019s very expensive to train people up, and machines are very good at predicting tenure in the job much better than humans are. We\u2019ve seen that from a couple of research projects, but not only that; the machines do that without some of the biases that humans have with respect to gender and race and things like that. That\u2019s not to say that machines won\u2019t also be biased in various ways, but it\u2019s exactly because they\u2019re programmed by people. So, the expectation is that they should be at least no worse than the humans who program them in terms of these issues.<\/p>\n CURT NICKISCH: So, let\u2019s continue this economic model here. Prediction gets cheaper. What else is happening in this model?<\/p>\n AVI GOLDFARB: The way I like to think about the economics of the book is the first point is prediction is cheaper, right, and that means we\u2019re gonna do more prediction, OK. Just like when the price of coffee falls, we buy more coffee \u2014 \u00a0nothing new there. The other thing to remember is, when the price coffee falls, we buy less tea, and so that\u2019s the substitute story. That\u2019s what, what are you going to have less of an organization? You\u2019re going to have less people who do prediction. The other thing that happens, and I think the most important thing, is when the price of coffee falls, we buy more cream and sugar. And so, the core of the book is these are the compliments. What are the cream and sugar for prediction? If predictions cheaper, what becomes more valuable? What do we end up using more of and buying more of?<\/p>\n In the book we emphasize that data becomes more valuable. That being able to take an action \u2014 decisions aren\u2019t useful without being able to do something about it. And perhaps most importantly, what we humans have is judgment, the ability to figure out which predictions to make and what to do with the prediction once we have it. And so, when you think about the opportunities around machine prediction, a lot of them are driven by the availability of data, the ability to take an action based on a prediction, and whether the people in charge, the people supervising the machine, can use it well enough to take advantage and help the organization.<\/p>\n CURT NICKISCH: So, let\u2019s break each of those down, starting with data. What do companies and people need to know about it?<\/p>\n AVI GOLDFARB: Sure. So, lots of people think that if their company\u2019s sitting on a lot of data, then data\u2019s the new oil, and so their company\u2019s sitting on some kind of great mine where they can make lots of money off it for years and years and years. The data-is-oil metaphor I think is really good, but it\u2019s good because just like oil, once you\u2019ve used your data, the data you\u2019re sitting on, it\u2019s used, and you\u2019re going to extract value from it, but you\u2019re going to extract value from it once. Say you\u2019re a warehouse, and you\u2019re sitting on data on past inventories and your warehouse, OK? That\u2019s going to help you predict future inventories, and you\u2019ll now have a better model if future inventories, and you can make money on it. But it\u2019s only valuable for that one model, essentially. You might think of something a little bit different, but fundamentally you\u2019ve extracted the core value from that data you\u2019re sitting on, and it\u2019s done. In order then to continually improve your models of inventory, you need new data on what\u2019s coming in in order to make it better.<\/p>\n The key source of sustained competitive advantage from data is through we call feedback data, which is the ability to continually improve your AI. And that means you need to invest in learning. What does investing in learning mean? It means making your product potentially worse in order to improve the AI.<\/p>\n So, think about using Waze \u2014 it\u2019s for driving directions \u2014\u00a0and if they learn that there\u2019s an accident on the road somewhere and there\u2019s a backup, the app, the software, the AI will take you a different route. And it\u2019ll take most people a different route, but at some point Waze has to figure out if that backup, that accident has cleared, and so it has to invest and essentially sacrifice some drivers or some technology in order to figure out whether that backup has cleared. And someone has to invest in improving the AI sometimes even at the expense of the immediate customer experience.<\/p>\n CURT NICKISCH: Wow, that\u2019s interesting. I had never thought of it that way. So, they may send somebody down that road and see if they get through or not.<\/p>\n AVI GOLDFARB: Correct. And so \u2014<\/p>\n CURT NICKISCH: And that person is just like, oh, what terrible luck; an accident just happened<\/em>.<\/p>\n AVI GOLDFARB: Or they might think Waze isn\u2019t as good as it seemed like it was.<\/p>\n CURT NICKISCH: Right. OK. What about action?<\/p>\n AVI GOLDFARB: OK. The prediction is really only valuable if it leads to a change in behavior. So, a prediction on how much yogurt somebody\u2019s going to sell, a prediction on a which stocks are going to go up or down \u2014\u00a0all of these things are only valuable if you can then go out and fill up your inventory, change the prices, by the stocks, these sorts of things. You know, if you\u2019re sitting on a lot of data, we think, you\u2019re a big incumbent company, and you might have some advantage.<\/p>\n But if you own the action, if you, if the customer relationship is with you and not with whatever startup is coming up with a better prediction on something you do, you have an advantage. That\u2019s the takeaway from understanding the benefit of action.<\/p>\n CURT NICKISCH: OK. And now maybe the funnest one is judgment.<\/p>\n AVI GOLDFARB: Yeah. OK.<\/p>\n CURT NICKISCH: Which makes us feel good about being human again.<\/p>\n AVI GOLDFARB: Absolutely.<\/p>\n CURT NICKISCH: Yeah.<\/p>\n AVI GOLDFARB: So \u2014<\/p>\n CURT NICKISCH Break that one down.<\/p>\n AVI GOLDFARB: OK, so judgment is the ability to figure out which predictions to make and what to do with the prediction once you have it. You can have a machine that gives you the best predictions in the world, but if you don\u2019t know what kinds of predictions are gonna be valuable, or if you can\u2019t tell the machine something of use to your organization or to you individually, then who cares? And so, what judgment is all about is knowing what you care about, what your organization cares about, and how to tell the machine to do that.<\/p>\n The simple example we like to use is thinking about the context of whether it\u2019s going to rain. OK? So, you have a weather prediction about how likely it is to rain or not. And depending on whether it\u2019s likely to rain or not, you may or may not carry an umbrella. OK? But that choice is going to depend on how much you hate getting wet and how much you hate carrying around an umbrella. And so, the prediction is rain, no rain, but the judgment is how costly that umbrella is to you to carry around versus how big a deal it will be to get wet. Now, an umbrella doesn\u2019t sound like a very big deal, like a sort of a pedestrian decision. But that example is exactly the same as an insurance problem. So, if you think about, should you take out insurance, it is exactly the same situation: How costly is that insurance going to be is like, should you carry an umbrella? How much better off you are you if you\u2019re protected by your insurance, you know, if something goes wrong is exactly like, well how important is it to have that umbrella with you if it starts to rain? And so, a whole bunch of decisions look like a lot like the umbrella decision. So first, you know, all the classic insurance problems, but also how much did you invest in security? It\u2019s all about weighing risk, and these are all judgment problems. So, you can predict how risky something is going to be, but that\u2019s only going to be useful if you know what to do with those predictions, and what better prediction allows you is to give final judgment.<\/p>\n CURT NICKISCH: The other message here is just that if judgment is a compliment of prediction prices going down, that\u2019s a place where you can focus your career, aim your company, and just bring those strengths to bear.<\/p>\n AVI GOLDFARB: Absolutely. So, a key, a key message of the book, but also I think just the reality of the technology is if your job is prediction, then you\u2019re going to have to rethink some of your skills and invest in learning new ones. That sounds pretty dour, and pretty bad, but it\u2019s not in the sense that often the people who are good at prediction are going to be exactly the people who know what to do with those predictions. OK?<\/p>\n And to get a sense of that, I like to think about what happened with accounting as accounting move to spreadsheets. OK? So, accountants used to spend their time doing arithmetic. Accountants don\u2019t do arithmetic anymore. Maybe they do a little tiny bit, but fundamentally that\u2019s not part of the job, but we still have a lot of accountants. Because it turned out the people who were really good at arithmetic or exactly the kind of people who would know how to use arithmetic well to help companies deal with the tax code, manage changes, identify where profit centers can be. So that that skill set, which was arithmetic skill set, which is no longer useful, turned out to be a great baseline for learning what to do with arithmetic. And I think that analogy is going to work in lots of high skilled jobs around prediction.<\/p>\n CURT NICKISCH: If you\u2019re in a company, and you\u2019re just trying to figure out how to \u2014\u00a0how to employ people to make the best use of this technology within your firm, what\u2019s a good way of thinking about that?<\/p>\n AVI GOLDFARB: What we need to think through is the workflow of a particular job in a particular process in your organization? So, increasingly driving is done by prediction. We\u2019re not quite at autonomous driving yet, but there was an insight a while ago that we can teach machines to drive by telling them to predict what a good human driver would do. And so, over time, this is one of these prediction tasks that wasn\u2019t an obvious prediction task 30 years ago that now we see, oh, we can do driving as prediction<\/em>. And so, what\u2019s happened is we need to think through the workflow of various people whose job is driving. And I think it\u2019s useful to compare a, a bus driver to a school bus driver, like a long-distance bus driver to a school bus driver.<\/p>\n And if you look at the workflow of a long-distance bus driver, almost the entire workflow is driving. And that means that that job is in many ways likely to become automated over time. It\u2019s not obvious what the need is for a human on the bus if what they\u2019re doing is driving somebody from place to place or driving you from place to place. That\u2019s different from a school bus driver, because the school bus driver, if you look at their workflow, they do a few things. I like to summarize two as one is drive, but the other is protect the children maybe from outsiders but particularly from each other. And so, even if over time the job of a school bus driver changes dramatically in the sense that they spend very little time driving, you\u2019re still going to need some role for somebody to figure out how to do the protection side of that job.<\/p>\n Now, how that exactly plays out, I don\u2019t know. One possibility is you end up with teachers on the bus, and you take advantage of that time as learning time. Another possibility is that it happens, one person remotely monitor several buses, and they have some system for protecting things, but you still need some, at least given our understanding the technology today, you\u2019re going to need someone to do that \u00a0protection job even as driving becomes automated.<\/p>\n When you break down the workflow, you can see some things in some jobs are going to become completely automated, but lots of jobs, yes, certain tasks within the job are going to become automated, but there\u2019s lots of other things that they do that our prediction tasks that a human needs to do. And potentially over time if it becomes cheap enough and easy enough, you can see even more jobs and more interesting jobs and better use of time because the mundane prediction part of the task is gone.<\/p>\n CURT NICKISCH: And that\u2019s a really simple example, but you\u2019re saying that you could do that with any job \u2014<\/p>\n AVI GOLDFARB: You can do that \u2014<\/p>\n CURT NICKISCH: Or any business process, business unit even.<\/p>\n AVI GOLDFARB: Absolutely. So, Goldman Sachs has broke down the steps to an IPO into over 100 small tasks. And when you look at each of those small tasks, you can identify which aspects of them are prediction, which aspects of them might be automated through some other technology, and which aspects require a human judgment or input of more data or an action done by a human or an action done by a machine. And so, once you break down the workflow into specific pieces, you can identify the opportunities to insert prediction machines as a tool for a particular task.<\/p>\n CURT NICKISCH: Especially at high-cost, high-value workflow like that one.<\/p>\n AVI GOLDFARB: Especially a high-value workflow like that one, absolutely. And the immediate results you might think is productivity enhancement in the sense of some people who used to have jobs won\u2019t be doing those anymore and it will be lower cost with machines, but at the same time, other aspects of the workflow \u2014 once the bottleneck potentially becomes cheaper, we can use other aspects of the workflow more and get more value out of the humans there and even create more jobs in that part.<\/p>\n CURT NICKISCH: I mean, just going through that task example makes you think a lot of tradeoffs, and with tradeoffs we think of strategy. From a business unit, from a corporate strategic point of view, how do you apply this simple economics model to, to what you\u2019re doing?<\/p>\n AVI GOLDFARB: The way to think about it is this, these are prediction machines and so what does prediction do? It reduces uncertainty. And so, if there\u2019s aspects of your business that you can\u2019t do everything you\u2019d want to do because of uncertainty, because you have to hedge against the risk of uncertainty, then prediction machine could be transformative.<\/p>\n Since there\u2019s so much uncertainty, but what an end consumer might want from a retailer, from Amazon, that Amazon has to wait for you to tell them what you want before it can ship to you. But if that uncertainty is resolved, and they know what you want, now, Amazon\u2019s business model can change.<\/p>\n And so, the business model moves from a shopping-then-shipping model to a very different model, which is, they send me things I want, and they\u2019re almost always right; and if I don\u2019t happen to want them, then I can in principle send those very few things back.<\/p>\n We can do a similar example. We\u2019ve talked a little bit about machines being better and better at, at hiring and HR. We can take that to the extreme similarly, which is that the reason we have to go through this onerous hiring process of posting a job and waiting for applications and screening through applications and interviewing and hiring is because there\u2019s uncertainty at every stage on which of those \u2014\u00a0who\u2019s gonna be interested in applying. If they apply, who\u2019s going to be above some threshold and we to interview to see if they\u2019re going to fit with culture and interest and we\u2019re going to get along and all these things. But if we have a great prediction machine, all that uncertainty could go away to the point where we could look at the database of people who might be interested in the job, who exist in the profession \u2014<\/p>\n CURT NICKISCH: And make it \u2014<\/p>\n AVI GOLDFARB: And make an offer right away and skip all those intermediate steps and totally transform the industry because we\u2019ve resolved that core uncertainty.<\/p>\n CURT NICKISCH: Avi, this has been great. Thank you so much for coming in and talking.<\/p>\n AVI GOLDFARB: Thank you very much.<\/p>\n CURT NICKISCH: That\u2019s Avi Goldfarb. He\u2019s a professor of marketing at the University of Toronto\u2019s Rotman School of Management.<\/p>\n