What did the expert say?
Well, OK, I was talking to Demis Hassabis [cofounder of DeepMind], and he was quite interested in this idea. We talked about it over a number of conversations, and I think the key question here is: Can AI actually get to that empathy, by understanding human emotions, controlling them through something like a work of art?Once it gets to the point where an AI program, AlphaTolstoy or whatever, can actually make me laugh and cry, and see the world in a different way, I think we've reached an interesting point, if not quite a dangerous point. Nevermind Cambridge Analytica. If it can do that to me, then it understands human emotions well enough to be able to run a political campaign. It can identify the frustrations and angers and emotions in the nation, or in the world at large, and know what to do with that.
The novel also considers how a person’s personality might be captured and re-created algorithmically. Why are you interested in that?
Klara and the Sun just accepts a world in which big data, algorithms, these things have become so much part of our lives. And in that world, human beings are starting to look at each other in a different way. Our assumption about what a human individual is and what's inside each unique human individual—what makes them unique—these things are a little bit different because we live in a world where we see all these possibilities of being able to excavate and map out people's personalities.
READ ALSO:
Is that going to change our feelings toward each other, particularly when we’re under pressure? When you actually face the prospect of losing somebody you love, I think then you really, really start to ask that question, not just intellectually but emotionally. What does this person mean? What is this loss? What kind of strategies can I put up to defend myself from the pain?
I think the question becomes something very very real then. It's not just an abstract philosophical question about, you know, the ghost in the machine, whether you have some sort of traditional religious idea of a soul or a more modern idea of a set of things that can be reduced to algorithms, albeit vast and complicated one.
So it becomes a very human and very emotional question. What the hell is a human being, what's inside their mind and how irreplaceable is any one human? Those are the questions that, as a novelist, I'm interested in.
Artificial intelligence isn’t yet close to this. Should we still worry about what it can do?
In general, that question, about human oversight, is one that we need to be thinking about right now. In the popular discourse, the thing seems to revolve around whether the robots are going to kind of take us over, a kind of crazy zombie vampire kind of scenario, except featuring kind of sophisticated AI robots. That might be a serious concern, I don't know, but it's not one of the things that I'm particularly worried about. I think there are other things that are much more on our doorstep that we have to worry about.
The nature of this generation of machine learning, which I understand is called reinforcement learning, is quite different to the old forms [of AI], never mind just programming a computer. It's just about giving an AI program a goal, and then we kind of lose control of what it does thereafter. And so I think there is this issue about how we could really hardwire the prejudices and biases of our age into black boxes, and won't be able to unpack them. What seemed like perfectly decent normal ideas a few years ago, now we object to as grossly unjust or worse. But we can go back on them because we can actually see how they're made. What about when we become very dependent on recommendations, advice, and decisions made by AI?
