Main navigation
Main content area

Watch a primer on AI to help you get the most from this summit.

Presenter: Bill Simpson-Young, Chief Executive Officer, Gradient Institute.

Transcript

(MUSIC PLAYS)

DESCRIPTION:

The Australian Government Coat of Arms. Black text reads Australian Government Department of Industry, Science, Energy and Resources

TEXT: TECHTONIC 2.0 – Australia’s National AI Summit 18 JUNE 2021

Below, blue text on white reads ‘This session will commence shortly’.

On the right, colour images arranged in tiles: a woman operates a drone over sun-kissed wheat fields; a robotic arm grips a small computer chip; a robotic arm in a factory welds; a large yellow mining truck; the Mars rover; a dark-haired man wears a grey coat in a factory and holds a tablet computer; a man wears a yellow hard hat and sits at a computer.

On webcam, a middle-aged man has short brown hair. He wears a dark suit jacket over a grey jumper.

TEXT: Bill Simpson-Young, Chief Executive, Gradient

BILL SIMPSON-YOUNG:

Good morning, everybody. I hope everyone has managed to join OK. Welcome to the AI Primer. So this is the AI primer that is on the morning of the National AI Summit put on by the Australian government. I'm Bill Simpson-Young, the chief executive of Gradient Institute. We're thrilled to be given the opportunity to run this AI primer. And hopefully for those not familiar with AI, you can get some general understanding of AI as appropriate for getting the most out of the summit. And then for those who already know a bit about AI, hopefully you learn a little bit more.

So just briefly, I'll say a little bit about who Gradient Institute is for those who don't know. We are a not-for-profit independent research institute, we’re a registered charity working specifically on trying to help make AI be used and deployed more responsibly. We focus on four main areas, research into responsible AI systems, technical research or background and machine learning. We try to guide our system design in business, government and other organisations to be more responsible, and that includes both from a governance point of view but also a technical point of view. We do work like system assessment, AI algorithm assessment, system assessment for companies around the world.

We also do a lot of education and training of people who are responsible for AI systems. Today is just a primer. We actually do much more detailed technical training for data scientists and software engineers, as well as governance training for people responsible for AI systems and organisations. And we actually work quite a bit to provide input into policy, government policy, to try to help ensure that government policy is technically informed from our technical understanding. We’ve…grade two, we're very happy to exist with thanks to organisations like IAG, University of Sydney and CSIRO Data61. And we've been operating since 2019. Now I'll like to hand over to Simon, who will tell you more about the Primer and kick things off. Simon.

DESCRIPTION:

Bill stands up and a young man, Simon, takes his seat. He has short brown hair and wears a pale knitted jumper over a blue t-shirt.

SIMON:

Great. Well, yeah thanks, Bill. Yes, so let's just get straight into it, I guess. So, our aim for today is to leave you with an understanding of the kinds of applications, areas where we might be best suited to deploy AI, the basics of how modern AI systems work, and some of the technical challenges and ethical considerations in designing and deploying these systems. And we'll also look at maybe some ways to try and mitigate these problems.

Just in terms of the format for the next two hours, roughly speaking, we kind of have a section for each of these like three points.

DESCRIPTION:

Black text reads ‘Aims of the Primer’.

Below, black text reads ‘By the end of this primer, we hope that you will understand:’

Black text dot points below read ‘the kinds of application areas that might be suited to AI systems’; ‘the basics of how modern AI systems work’; and ‘some of the technical challenges and ethical considerations in designing and deploying AI systems’.

In the top right, the Gradient Institute logo is a three-quarter blue circle of differing shades.

SIMON:

And there'll be some level of kind of interaction with demos as well on each of these. And I'll be splitting the presenting duties between myself, Finn and Al who are both here in the room with me. And maybe depending on how we're tracking sometime around the hour mark, we might take like a five minute break. So, we're very keen to have this session as interactive as possible. So if you have any questions, you can just post them here in the Q&A box to the right of the screen and then others can upload it too.

AI is a very broad area, so we won't have time to cover every topic, even on a very high level in this primer. But if there's something specific that you'd like us to address or something to clarify, just pop it into the Q&A box on the right, others can upvote it and then on occasions we'll cross over to our colleague in Dan who will be, he's in Canberra, he'll be monitoring the question box. And we'll cross over to him and he'll ask us some questions and we'll see if we can do our best to answer it. And there's also a whole section here that we should be activating momentarily that will kind of let you, kind of see, you know, let you fill in what you're kind of level of experience is or connection with AI. That'll kind of give us an idea as to what level to pitch some later slides at. That should be opening now, we'll look back at that later.

DESCRIPTION:

White text on black reads ‘What is AI?’

A black and white image has a cross section of the human head and brain. A sphere around the head has branches of interlinked dots.

SIMON:

OK, well, like let's start with what exactly is AI? So I've seen the term AI applied to everything from like fictional super intelligent systems that for some reason deem it necessary to eradicate the human race, right down to electric toothbrushes that harness the power of AI to provide unparalleled levels of cleaning. So this primer is focused on AI systems in use today. So it's very much closer to the toothbrush end of the spectrum than the fictional super-intelligent end. We're talking about systems that are trained using data to perform well at a very specific application. Often when we see representations of AI in the media, we're presented with something that looks a bit like this image here you see in front of you. I should stress, like be very careful not to anthropomorphise these systems, because it implies that these systems use a similar logic to how humans solve problems. Right? Modern AI systems do not think the same way as to how humans think.

DESCRIPTION:

White text on black reads ‘What is AI?’

On the left, the previous black and white image now has a red cross emblazoned across it. Alongside the image is a line graph tracking upwards that has a green tick emblazoned across it.

Below both image and graph, black text reads ‘AI is a system that adapts its behaviour based on data to achieve some goal.’

SIMON:

So it's less of the image on the left, which seems to kind of imply some level of sentience, and much closer to this image, like, less exciting image on the right. Right? An algorithm that uses math and statistics to find patterns of data. Modern AI systems are dual systems that search for correlations of data, and then tune their behaviour to meet a very specific narrow goal. They have no context for anything that isn't captured by the data or its goal. The learning part of AI is like when the algorithm adapts itself to achieve its objective based on the data that has been shown. And this can kind of work uncannily well at times. So the behaviour which gives people the impression that the system is somehow intelligent or sentient, is when it becomes able to kind of detect or recognise some quite sophisticated patterns that even a person might struggle to identify.

DESCRIPTION:

Black text on white reads ‘What are AI’s benefits?’

Below, black text dot points (outlined by Simon).

SIMON:

So, like recent improvements in AI have come largely due to this explosion in the quantity of data out there available to the people who make these systems, but also kind of significant leaps forward in the easy access of huge amounts of computing power. However, AIs like widespread adoption in industry, comes down to a number of key factors. So one of which is scalability. Right? AI allows organisations to make decisions at a scale that's just not possible with human operators. You could think of say, YouTube for example, which I think is something on the level of 30,000 hours of video for every hour. Right? You can have an army of moderators trying to vet these videos and it just wouldn't be possible for them to properly screen all these. So instead, they offered a lot of this kind of decision making to an AI.

Decisions can also be personalised. Right? So an AI making a decision about you will use whatever information it has on you. So even though they are scaled globally, they can make decisions that are kind of factored to each individual customer. They're also, in many cases, accurate if used correctly. Capable of kind of ingesting huge amounts of data and referring to historical records to make a very informed decision.

Also in industry, they can be, in some cases, a cheap option. Right? So AI systems can work 24/7, seven days a week without getting tired. So in comparison to, say, human operators. Right? Which could suffer from fatigue or whatever. But it's important to note that these aren't like set and forget technologies. OK? They learn patterns and data, like I was saying. If the environment changes or the people interacting with the system change, the patterns that it's learned in the past might become stale and like they're susceptible to making large errors. So it's important to remember, don't underestimate the cost of maintaining an AI system.

And then lastly, another application that's important for industry, is that they can often be a safe option, too. So systems operating in environments where it's hazardous to send a person, an AI system can operate in these regions like you can think of say, robotics in mine centres or in dangerous parts of a plant. They are often like a better solution than trying to send a person into some areas.

DESCRIPTION:

White text on black reads ‘Where is AI?’

Six colour images: a person holds a smartphone; a red ‘Netflix’ icon on a small black tile sits atop some DVDs; an x-ray of a human chest; a hand stamps ‘APPLICATION REJECTED’ in red ink on a job application form; a driverless harvester on a row of crops; a yellow truck drives between shipping containers as a jumbo jet flies overhead.

SIMON:

OK. So now let's look at like where do we find AI? So, AI is becoming ubiquitous. If you've checked your phone today, more than likely you've already interacted with some element of an AI system. Like the phone itself is using like numerous AI models to do things like predicting your usage so that it can save on its battery. Or predict your intent when you're using predictive typing. Or, you know, improve your photos through filtering on its camera. Right? The apps in your phone, too, like the social media apps, are using AI systems to decide what articles you get and what articles you don't get. Online shops make like product recommendations tailored specifically to you and your tastes.

So in some cases, like not only do systems decide what products to show you, but how to show them to you. So like large online streaming companies, they not only decide which movie to show you, but which ten-second clip of that movie will most likely pique your interest in it. And that's targeted specifically at your tastes and what it thinks you'd be interested in.

Of course, then there are many AI applications that are non-customer facing as well. So supply chains are becoming more and more interconnected. And optimising their performance requires predictions on supply and demand, for example, travel times, price fluctuations. This is a problem that's growing ever more complicated and nowadays, large companies outsource a lot of this to AI systems. You can think, too, like ride sharing companies. They have like this big kind of demand prediction problem, which they address with AI systems and then they use those AI systems to kind of redeploy their fleets in certain positions.

This session that I'm talking through right now is probably hosting on something like the AWS Cloud, which use AI to decide, you know, make predictions about load balancing and which computers to bring online where and when. Even this session, I think, also has like a closed caption system. So there is a speech-to-text AI that's probably struggling with my accent a little bit, no doubt. But it's trying to interpret my audio signal and converting that into text.

So in Australia, too, the agriculture and mining sectors have experienced huge benefits from AI. We can think of these like enormous farms where they have AI vehicles that are deciding like what's a weed and what's a crop. Or in mines for them to identify bits of minerals. You also have autonomously driven tractors or hauling trucks. Or also now these days, more and more are using AI to make decisions about people, right? From who gets picked for an interview to who should get released on bail, who gets medical treatment, who gets a mortgage, who gets matched with who on dating apps, for example. Like the fact that we're relying on AI to make more and more of our decisions, means that it's important that we understand its capabilities and its limitations.

DESCRIPTION:

Black text on white reads ‘Natural language processing (NLP)’

Below, further black text (outlined by Simon).

In the top right, three rounded squares have face emojis inside them. From left to right, a broad smile with green text that reads ‘Positive’ and black text that reads ‘I can’t recommend this product highly enough’; a flat mouthed face and grey text that reads ‘Neutral’ and black text that reads ‘It’s okay, I guess’; and a sad face with red text that reads ‘Negative’ and black text that reads ‘I’ve had trips to the dentist that were more enjoyable than this’.

SIMON:

So, as I mentioned, like AI is a very broad topic, right? So what we'll do know is we'll just look at like a couple of areas that have experienced like significant progress in the last couple of years as a result of innovations in AI. Right. So the first one we're going to look at is natural language processing or NLP. Right? So, computers have kind of understood certain languages since their inception. Right? We call them programming languages. Right? They're typically kind of rather limited in their vocabulary and they follow very strict conventions. Right and the code must be 100% error free in order for the computer to understand it.

When humans communicate with one another, we draw on like this very diverse vocab that varies from location to location. Some words have multiple meanings. And without context, meanings are often extremely ambiguous. And that's before we even kind of think about mispronunciations, grammatical errors and all these kind of things. Right? But the human brain is quite adept at handling all of these issues. So there's been a large amount of work in trying to enable computers to understand human speech.

So AI has contributed a lot to this field in the last few years and as a result of this, we have applications such as, say, auto correct. Right? Where there's an AI system predicting what it thinks you meant to type, for example. Or speech recognition, where there's an AI system listening to an audio signal and then attempting to convert that audio signal to text. And it does some of this based on like a language model, which I'll talk about in a second. There's also like sentiment analysis, right, where you can think of these huge corpuses of text, like say that an AI is trained on, and then it'll predict what the general sentiment is of a topic. So you can look at, say, reviews for a movie or about a politician or whatever, if you look at Twitter, you can kind of use a sentiment analysis message trying to assess whether there's, you know, they are pro a certain product or against a product.

You also have chatbots, obviously, which are customer facing machines that help deal with certain queries or complaints, like hopefully more queries, I'd imagine, that will help direct customers and avoid them having to use the phone altogether. And that's also kind of engaging in a conversation on a computer to human level. And also, I mentioned it previously, but automated moderation. Right? The kind of content that's been generated by people and put online now just isn't, it's not possible to moderate these kind of things with an AI system. So… Sorry, with humans. So what we use our AI system to decide, what gets passed in the gate and what doesn't. So we're offloading a lot of kind of what people see and what people don't, to an AI system.

So I mentioned language models, and these are things that try to predict what word comes next in a sentence. OK, so like start a sentence and give you three words, can you predict what the next word I'm going to say is? Right. And AI systems are trained to do this by looking at huge amounts of chunks of data from the internet and over time learning how humans speak, right, and what the kind of connection is between words. The most or one of the more recent ones of these, I think it's called GPT-3, took something like $12 million of computing power just to train it. Right? And has 175 billion variables. It's very powerful. And we'll have a look at one of these now, actually. I think so, what I might do is might jump to one of our demos.

So let's just jump out here. And you should be able to, there might be a window that lets you click on pop-out mode if you want to continue seeing me. But if you click on the link here below me called techtonic.gradient.org it should take you to a, like a screen where we're kind of linking to a bunch of various demos that we'll be referring to throughout the day. So let me just go over to it now myself. I will jump here.

DESCRIPTION:

Black text on white reads ‘Gradient Institute Ai Primer Resources’ along with the blue three-quarter circle Gradient Institute logo.

Below, a black text header reads ‘Machine Learning Demos’ along with five hyperlinked headings: ‘Natural language completion’; ‘Speech-to-text translation’; ‘Image-to-image translation’; ‘Small neural network classifier’; ‘Complex neural network classifier’.

At the bottom, a line graph with scatter plots and red circles of differing sizes along it. Black text above reads ‘Model Selection as Optimisation’. Further black text at the top of the graph reads ‘Model Loss: 223.45’. Black text on the vertical ‘Y’ axis reads ‘Hospitalisation (Days)’.

A mouse pointer clicks on the first of the five headings – ‘Natural language completion’.

A subsequent screen has black text at the top that reads ‘Talk to Transformer’. Below, a text box and a blue ‘button’ with white text that reads ‘Complete Text’ (outlined by Simon).

SIMON:

So if you click on the link here, and let's jump through. And let's just click on ‘natural language completion’. Let's open this up here. So, shift, click. OK. Alright. So what we're seeing here is like an interface to a language model, which you can prompt with a sentence and then it will try to infer or predict what it thinks the next word is, and the next word, and the next word, and the next word and so on. So this is just a default sentence that's been put in here. You can change the sentence in any way you want or open up some other ones that suited us. I'll just use the default here for a second and say complete text. And what we can see now is an AI system is, based on patterns that it's learned from reading text on the internet, it's inferring what the rest of the sentence is, the rest of this article is.

So it's talking about, ‘while not normally known for his musical talent, Elon Musk is releasing a debut album’. That's our prompt. And then it continues with, ‘of original rock music, with the band, The Boring Company. According to Rolling Stone, these albums, the album will be called the boring company, over 15 tracks.’ OK, so we can kind of see here that it's kind of pulling in different information about music and about Elon Musk and combining it into something that's somewhat understandable. Right? Like I mean, each word seems to make sense following the previous one, but oftentimes we find on a like a… if you kind of take a step back and read many paragraphs after the other, it kind of starts to lose its flow a bit.

We can do even something small here, like, let's just say change, ‘releasing a debut album’. Releasing a debut, let's just say a ‘hip hop album’. And let's run this again. OK, what's… ‘next month, Variety reports’… ‘Musk’… so it's put in some more things here about Tesla and talking a lot about his car company this time. It's kind of moving away now from anything about… Yeah, it’s moving away from any mention of his album. Or we can see here something about, yeah, brings back in the hip hop album towards the end.

So if I just… this is a stochastic process, so if I actually could generate again with the same prompt, it will give me something completely different. But it's interesting. You can imagine how this can be used to just generate news articles or whatever. It's quite a powerful technology and needs to be kind of used appropriately. I'll just go back here. Let me just jump back. So I think I'll do, back to this and then I've got a question. OK. Alright. So I think we have a question here that we might jump across to if we can bring up Dan for a second, perhaps. I see you in the chat.

DESCRIPTION:

A second webcam window appears alongside Simon’s window. A man, Dan, has dark hair and wears a red jumper and black headphones.

DAN:

We have a couple of questions.

SIMON:

Great.

DAN:

First is, ‘will there be a recording of these?’ And the answer is yes, there will be. It'll be available on the industry.gov.au website. But I believe you have to ask for it. And the next question is a tricky one. ‘So what do you think are the opportunities and threats for symbolic systems? Or more specifically, how important is neuro symbolic AI?’

SIMON:

OK, great question. And I'm going to bring in Finn to answer that question here. She's in the room with me. Scoot across.

DESCRIPTION:

Simon’s webcam pans across. A woman, Finn, has short red hair and wears a black t-shirt.

FINN:

Alright. So we know that the current generation of AI systems are really powerful at certain types of problems, specifically recognising these really complicated patterns. So that's what the neuro or the deep learning networks are really great at. But we also know that there's a huge gap between the kind of set of problems that modern AI systems consult well and the problems that people solve. So these systems still require, you know, vast amounts of data to learn. They are not very good at transferring something that's learnt on one problem to a problem that's even slightly different. And so we can kind of see that they're not, they're not anything like a general artificial intelligence at the moment.

So if we want to get to a more human-like intelligence, there is some missing component. And this kind of symbolic reasoning combined with the pattern recognition that we have from neural networks, is kind of one path towards that, that people think is quite promising. But because we're still so far from this, we really don't know whether that will be the path or there will be some completely different approach that ends up getting us to artificial general intelligence. So absolutely, we know that what we have now is not the, the end goal or if we are aiming for general intelligence, but which specific approach is going to get there is something that's too early to say at this point.

DAN:

Thanks Fin. That's all the questions we have now.

DESCRIPTION:

Simon returns to the webcam.

SIMON:

Great. OK. We'll check in on that in a second. OK, so thanks very much. OK, so I'll just go back to what I was saying about language models there. So we saw an example of one being used to generate text. We can kind of imagine too, how that might be useful for... Let me just bring back the slides here. Bring it across. Here we go. OK.

DESCRIPTION:

The shared screen returns to the ‘Natural language processing (NLP)’ page from earlier.

SIMON:

OK. Yeah, so we can kind of imagine how this language model might be used for speech recognition. For example, if I say a word like limb or lamb, like they sound pretty similar. It can be very hard maybe for an AI system to try and work out which one I've said, especially with an accent. But if I said, if the first, kind of, few words before that were, Mary had a little, it's more likely that that word would be lamb. You can try it out yourself. There's another link there about speech recognition or speech text. If you say Mary had a little limb, it'll predict it's probably lamb. I think it's (INAUDIBLE)…

So it relies heavily on context to try and disambiguate any kind of confusion it might have in these systems. OK. And also out of interest as well, I was wondering if I could simplify my job and just auto-generate the presentation. So I fed like a starting line into this language model. I got something like this.

DESCRIPTION:

Black text on white reads ‘AI-generated presentations.

Below a text box has black text generated inside it (outlined by Simon).

SIMON:

So you can see my prompt up the top there. ‘Natural language processing systems are AI systems’… et cetera, et cetera. Here's what it replied with. ‘These systems have the ability to process and process human language, and the capability of processing human language is the feature of natural processing systems to provide the human machine interface with the system. Natural language processing systems are both behind and ahead of text mining. Natural language processing systems, by definition, are capable of processing and processing human languages over text mining is the best at processing textual content.’

So, I mean, there's, it sounds a bit like technobabble. I think my job is probably safe for maybe like at least one more iteration of this language model. Yeah. Although, I mean, to be fair, this one is slightly cherry-picked. There were other ones that I drew from that were up close to what I was saying now. So I (AUDIO CUTS OUT) a little bit. OK.

DESCRIPTION:

Black text on white reads ‘Computer vision’.

Below, black text and dot points (outlined by Simon).

On the right two colour images one above the other. The top image has a city street with people, cars and features highlighted by yellow or red box outlines and a descriptor such as ‘car’ or ‘traffic light’ etc attached to each one.

The image below has a picture of a man and white text that reads ‘Results for this detection: – Man – From 36 to 43 years old’.

SIMON:

The other area of technology that we're going to look at here is computer vision. So this is an area similar to how language models focus on trying to understand language, computer vision looks at trying to get computers to understand what it's seeing through a camera. And until very recently, computer vision engineers used to often like hand code features in an attempt to help AIs identify objects and scenes. Things like edge detections, like circle detectors, texture detectors that we use to kind of put up a lot of the stuff and hope then that we can combine these to understand what a car is or what a person is or what a tree is, right?

Now, like there's enough labelled data out there and enough computational power to just build like this incredibly flexible model and just let an AI develop its own features. We'll have a look at this in a lot more depth later on. But just some of the applications in this area are things like face recognition, where it's kind of not only trying to infer where a person's face is, you know, to help the camera focus, but also things, attributes to know about that person, right? Like whether they're happy or sad or paying attention or not.

I ran one there on myself last week, I think when I was preparing these slides. So it said I was male and I was 36 to 43 years old. I was a little disappointed in that because I'm 36. So I tried it again after the long weekend. Maybe a bit more refreshed and I got, now I'm a male 29 to 36. So both are right but I'm glad that I'm on the upper end of the second one. I got a haircut too. So maybe that's making a difference. It shouldn't. But you can see how these models shouldn't be affected by certain things. Like my mood, (INAUDIBLE) but they're still susceptible to these kind of features, right?

Also computer vision has been used in quality assurance, as well as a medical diagnosis for identifying tumours in images, security, for example, for not only detecting people, but also perhaps their intent or their activity. Autonomous driving is one where I think it's Tesla are just relying completely on vision to drive their cars, unlike others that are using RADAR and LiDAR as well, and visual search. So we'll just jump over to our last section here. If you go to Techtonic again, it'll just open this up.

DESCRIPTION:

Simon returns to the ‘Gradient Institute Ai Primer Resources’ page from earlier.

A mouse pointer clicks on the third of the three hyperlinks – ‘image to image translation’.

SIMON:

So let me jump across myself and go here. Great. And let's open up ‘image-to-image translation’. Click that myself.

DESCRIPTION:

Grey text reads ‘edges2cats’.

Below, two squares side by side. The left side square has a black text label that reads ‘INPUT’ and inside the square is the partial plotted outline of a cat in black lines. The right side square is labelled ‘OUTPUT’ and has a colour image of a grey cat inside.

Between the two squares, a smaller square linked by two arrows that point to the right on either side. Inside, black text reads ‘pix2pix’ and a pink rectangle below has white text that reads ‘process’.

SIMON:

OK, so we're just looking at the first one here. There are other examples that you can look at yourself. This is an AI system that's, they generated this data by taking lots of cat images and running an edge detector over it, and then training the AI to go from the edges to back to the original image. So what we can do here is draw a cat and then it will try to infer what the picture of that cat we've drawn would look like. So let me just clear it first, and I am using, let me (INAUDIBLE) my mouse here.

DESCRIPTION:

The two squares are cleared to white. A cross-hair pointer begins to draw a cat outline in the left ‘INPUT’ square.

SIMON:

OK, you have to bear with me from a very crude, I probably should have practiced what a cat looks like before trying to… Give it a face, maybe a body and a tail. OK, shoulder here, maybe some paws. Oh God. OK, and a pair of eyes. Prepare for a nightmare here. Let me just process this. So what it's doing now is it downloading the model and it's going to run the model over this edge here. Not completely happy with this, but we'll see what it comes up with. And then it's going to kind of compare these patterns to what it's learned when it was comparing edges to I think...

DESCRIPTION:

A colour image of a small grey kitten appears in the right ‘OUTPUT’ square.

SIMON:

OK, right, it's pretty cute. OK. so we can see here, this is kind of the images inferred here, but it's like not completely robust. Like it doesn't have a real understanding of what a cat is. It's just kind of our correlations of data. Like I think if I draw, let's just say if I draw another circle here, which could be part of a collar maybe or something. Right? Let's see what the AI kind of person looks like. OK. It looks like it's given it like another eye. And it can kind of do the same. So it's associating the circles with eyes and it doesn't really care about where in the body it is.

DESCRIPTION:

The shared screen scrolls down to some similar two square arrangements. One is labelled ‘facades’ in grey text and another below it is labelled ‘edges2shoes’.

SIMON:

So you can play around with the ones below those two that do similar what they've been trained on, different training sets. So this one's been trained on buildings, one on shoes. You can make your own shoe there, et cetera. Right? OK, let me jump back now. I think that's enough from me. And we're going to now move over to how AI works. So we're going to leave the application side and begin focusing on... maybe just go back here. Begin focusing on the nuts and bolts, what's actually going on under the hood in these models? I'm going to pass over to Al for this.

DESCRIPTION:

The shared screen returns to the ‘Computer vision’ slide from earlier.

Simon moves off camera and another young man, Al, takes his place. He has short blonde hair and wears a dark t-shirt.

On the right, white text on black reads ‘How does AI work?’

AL:

Alright. Hi everyone, so in the previous section. We've just seen lots of exciting applications of AI to solve real problems. And in this section, we're going to take a look under the hood at how these technologies actually work. And even if you're not going to be developing or deploying an AI system yourself you might still benefit from understanding the design decisions that go into building one.

DESCRIPTION:

Black text on white reads ‘A paradigm shift in software development’.

Below, two black text dot point headers have text that reads ‘Traditional software:’ and ‘Machine Learning:’ respectively. Further black text dot points accompanying each header are outlined by Al.

AL:

So, AI represent a paradigm shift in software development. So suppose we need to produce a piece of software that examines a data record and takes some action based on it. The traditional approach to this has been the developers manually specify how to choose the action. They could, for example, write down a set of rules or equations based on their experience and knowledge about the domain. But with an AI system, you approach the development differently. So, we don't specify how to choose an action anymore, instead, we set the goal that we want the system to achieve and then we provide data containing the patterns that we want the system to recognise. And from this point on, the computer creates and executes the actions that the data suggests will solve the problem automatically.

So, let's put this into a more concrete example.

DESCRIPTION:

Black text reads ‘The ingredients of a modern AI system’.

Below, five numbered points read  ‘1. Goal: Identify candidates to bring in for an interview’; ‘2. Data:’ (small images of  five documents/CVs); ‘3. Labels: hired – hired – not hired – not hired – hired’ (below each previous CV); ‘4. Loss function: Twice as bad to exclude a good candidate than to interview a poor candidate’; ‘5. Machine Learning Algorithm:’ (an accompanying diagram of coloured circles in vertical rows and wavelengths crossing through them).

AL:

So, these are the ingredients that go into designing a modern AI system. Let's imagine a scenario where we're building or we're using AI to filter the CV of job applicants. So, the first thing you need to do is specify what the goal of the system is going to be. In this case, we'll say we want to identify candidates to bring in for an interview based on their CV. Next, we define the data that we'll use for decision making. So, we have options here in terms of what kind of features we want to extract from their CVs. We would, for example, at least want to extract features, encoding things like employment experience and qualifications, and represent these numerically in a way that the algorithm can adjust them.

The next thing we have to do is find a way to pass our knowledge onto the algorithm, and we do that by labelling example data. So, the construct that we're trying to convey to the algorithm, is whether a candidate should be brought in for an interview. But that's not something we can easily measure in data. So, we typically have to choose a proxy (AUDIO CUTS OUT)…

..things such as we could record, whether, with historical CVs, the candidate received an interview or even whether the candidate was hired, for example. The next thing we need to do is decide on a loss function. That is a mathematical objective to determine, given a choice between two alternative models, which one is the better model. And so that means we need to do things like weigh up how important different types of errors are for the model. For example, it's twice as bad to exclude a good candidate than to grant an interview to a poor candidate. And each of these steps changes the design and the ultimate behaviour of the model. The last thing you do once you've formulated your problem, is you throw it into a machine learning algorithm which searches for the pattern that best fits the data as defined by our problem formulation.

So, each of these steps an important design choice with consequences to how the system will behave and more to it than just picking your machine learning algorithm, which arguably is the least consequential decision here compared to setting your goal, defining your data, making your labels and then defining your loss functional objective.

DESCRIPTION:

Black text reads ‘AI Systems Today’.

Below, a thin black rounded-corner rectangle encircles six small colour images. Black text on the right reads ‘Training Set’.

Inside the rectangle, the top row has black text that reads ‘Dog:’ and has three pictures of different dogs alongside it. The bottom row has black text that reads ‘Cat:’ and has three images of different cats.

Below the rectangle, an arrow points down to a black funnel shape. Below the funnel shape is the earlier algorithm diagram of circles and wavelengths.

AL:

Let's switch tasks, so instead of analysing CVs, we're now going to be thinking about a classification task of identifying whether a picture is of a cat or a dog. So, we would start by collecting data containing (AUDIO CUTS OUT) of dogs. And then we need to label that data. Now we all know the difference between a cat and dog. So it'd be really easy for us to specify labels for the training data. So we put all the dogs in the top row, all the cats on the second row.

However, although we're really good at doing this task. It will be very difficult for us to write down the specific rules that we're using to recognise cats and dogs, especially when you consider that these images are arrays of pixel values, like we would need to be writing rules into the element in a position in an array. So here, machine learning, we can leverage machine learning as a really great way of encoding our implicit knowledge. So, we specify the objective, which could be something like accuracy and then use machine learning to find the patterns and rules that can make this prediction for us.

So, if we feed this data set and this objective into a machine learning algorithm, it produces a model, a piece of prediction software that we can then use on new data. We can feed a new image. And I’ll just flag here, like in this case, it's easy for us to label the data. But the process might even work, if we didn't know the difference between a cat and a dog but the answer was recorded in some way. For example, a vet could write the label on the back of each photo. So the AI developer doesn't need to be good at the patterns they're using. They just need to be good at setting up the problem and passing it on to the machine learning algorithm.

DESCRIPTION:

At the bottom of the screen, a colour image of a brown and white beagle replaces the funnel and diagram. Alongside the beagle image, an arrow points to the algorithm diagram. A second arrow points from the diagram to a horizontal two-row bar graph. The top row has black text that reads ‘Dog’ and a brown bar on the graph that reads ‘94%’. The second row reads ‘Cat’ and has a blue bar with ‘6%’.

AL:

OK, so then once we have this, we can set the training data aside, but we need to keep in mind that was the data the model was trained on. But we can then invoke this model to run a query on a new data point. So in this case, if we feed a picture of a dog in, the algorithm might say, or the model might say, that is 94% chance that that's a dog, which is what we'd want to happen.

DESCRIPTION:

The colour image changes to one of an elephant. The bar graph now reads ‘9%’ for ‘Dog’ and ‘91%’ for ‘Cat’.

AL:

But… So the algorithm is doing computer vision, which is something that seems like quite an intelligent task. It has no broader view of the context. This is what we call a narrow intelligence, because it can only solve the problem it was specifically trained to do. So, I can change the context on it by substituting in a picture of an elephant as the input to the model. Now, you might hope that the model says it's not a dog and it's not a cat. So like 0%, 0%. Except that we've never, in our problem formulation, everything is either a cat or a dog. So, it has to say some combination. And hope it says 50/50, suggesting it doesn't know.

But in this case… it's not guaranteed that that'll be the case. In this case, it might say, 91% chance that's a cat, which to us seems senseless, but this stems from how we set up the problem. So, the algorithm doesn't know what it doesn't know and things like common sense, like saying that's an elephant, is domain knowledge that we haven't given it. And if we intend to query it with like animals other than cats and dogs, that's our job to set up that elephants are in the space of answers it can give.

DESCRIPTION:

Black text reads ‘AI is both brilliant and senseless’.

Further black text below reads ‘Some tasks that are difficult for people are easy for algorithms, and vice versa.

Below, two colour images. The left colour image is of the board game ‘Go’ – black and white ‘stones’ on a grid. Black text below it reads ‘AI Algorithm is reigning world champion – the patterns are subtle; data is plentiful; the state of the board is known’.

In the right image, a young girl brushes her teeth. Black text below reads ‘The boy is holding a baseball bat’. Further black text dot points below read ‘Common sense is domain knowledge in disguise – how big is a baseball bat?; what is it used for?’.

AL:

So, AI can be both brilliant and senseless at the same time. Being a narrow AI doesn't mean that it's not useful. So, a narrow AI can achieve superhuman performance on some problems. But it can make errors if we set up the problem in a way that doesn't contain the context, that can seem very stupid. So there's no single, uniquely human ability that determines whether a problem is best solved by an AI or a human. But here are some aspects that might lead to it being better or worse.

So you might have heard of AlphaGo. An AI algorithm was trained and became the reigning world champion in the difficult board game of Go. And the reason it was so effective here is because the pattern (AUDIO CUTS OUT) are really subtle and human players struggle or need a lot of experience to learn them. But, playing to the algorithm's advantage, the data is plentiful, in fact, essentially infinite because the algorithm can play against itself, and the state of the board is fully known. So it sets up a really clear problem for doing pattern recognition.

On the other hand, we have an example of a computer vision algorithm to caption an image on the right, and it's… it's a girl brushing her teeth and it's labelled it as a baseball bat. So in this sense, the common sense… this is not a sensible answer, but it's because the algorithm doesn't know things like how big a baseball bat is or how you hold it or what it's used for, it's just detected the face of a child and something that is shaped like a baseball bat in their hand.

Alright, so we're about to jump into an interactive, but before we do, let's go to the poll results.

DESCRIPTION:

TEXT: What is your connection with AI?

Below, four differing horizontal bars on a graph have green text and a percentage amount that reads ‘I work in AI – 33%’; ‘My organisation uses AI – 11.1%’; ‘I want to know if AI is appropriate for my application – 0%’; ‘I have a general interest in AI – 55.6%’.

AL:

OK. Sorry I can't see that.

SIMON:

Maybe we'll go to Dan.

AL:

Dan can we chat to you please? I can't see the poll results.

DESCRIPTION:

Dan on webcam.

DAN:

Yep, sure thing. We have one question that's come through and it is, ‘how do you account for bias, especially in these examples where past hires were based on a certain group of people?’

AL:

Yeah, that's a really great question. So certainly you can, you can unintentionally capture historical disadvantage in the way you set up the problem, in this case, if your labels come from whether someone was historically hired or not hired. And this is a real problem where you need to either rethink the way you're labelling the data, for example, you could (AUDIO CUTS OUT)…

..a panel who you believe is not biased to assess the training data and produce their own label instead of just drawing it from the historical data, cause there's always a risk that we fall back on the most accessible data, which may be more biased than the…the best data we could get. But the other thing to watch out for here is that, that could still capture current disadvantage. There may be some correlations between protected attributes such as age or gender, or sexuality and the labels that you end up taking. So later today, we will look into responsible AI, of which this is an element, and specifically we can look at concepts like algorithmic fairness. But I’;; actually defer this to a later part of the discussion. Alright, and in addition to the questions, Dan, can you just run us through the poll results there?

DAN:

Yeah, sure thing. So the majority seems to be, I work in AI at 33%, actually sorry no, that's not right. The majority is I have a general interest in AI and that's just under 50%. Then the next is, I work in AI, so that's just over a third. And then we have my organisation uses AI at about 10%. And then lastly, we have I want to know if AI is appropriate, and that's at about 5%. Yeah. And that's and that's what we have so far.

DESCRIPTION:

The earlier bar graph, updated with the values Dan read out previously.

AL:

Alright, excellent. Thanks very much.

DESCRIPTION:

Black text reads ‘A simple example: Curve fitting interactive’

Below a dot point link reads ‘Go to techtonic.gradientinstitute.org’

Below the link, the line graph from earlier with scatter plots and red circles of differing sizes along it. The ‘Y’ axis reads ‘Hospitalisation (Days) and the ‘X’ axis (now visible) reads ‘Age’.

AL:

So continuing on here, we'll now go to an interactive, which is a curve fitting one. So if everyone can pop out now, and we'll go back to the same web page, the techtonic.gradientinstitute.org And on the links page, we actually have a couple of interactives embedded. So these interactives are about setting up machine learning regression as an optimisation and then looking at how we actually select a model.

So the task here is that, imagine we have people coming into hospital with some disease and our task is to predict how long they will stay in hospital so we can schedule them a bed. And we've simplified the attributes we have describing them down to one dimension for this illustration, which is their age.

So on this plot here we have the horizontal axis is their age, which is the feature that describes them. And the vertical axis is the hospitalisation in days, which is the target that we want to predict. Now, we have some historical data with which we want to calibrate our model, and this data is shown by the black dots. So it's noisy, but there's an underlying trend. And then we have a model, which is this blue line, and we want to learn this model. And it has three parameters just to keep things simple. We've got an intercept, a slope and a curvature, and we've given you sliders to control each of these. So the first slider is intercept, next slider is slope, and last one's curvature.

DESCRIPTION:

The blue horizontal line moves up and down, tilts and curves, and the graph values change accordingly. i.e. the red circles expand and retract.

AL:

Now, to choose the model, we're not going to just eyeball it, what we're gonna do is actually optimise a particular objective. So we specified a model loss function that's visualised by these red circles. The size of the red circle relates to how far the model is from each data point and how much, therefore, that data point is contributing to the total model loss. So a large circle means a poor fit that we should be trying to address, where it's a smaller circle means a relatively a better fit. And we can, we can sum the contributions to get a total model loss that was given a title.

So, for example, this model has a loss of 43 and this one has a loss of 38. So by our objective, it is the better model that we should choose. But we can now… So let's have a go. I'll give everybody a minute or two to try and optimise this manually by adjusting the parameters until you find a good model fit. And then after a minute, I'll walk you through the process. But I think it's good to have a go yourself before we, before we do that.

OK, hope you're all doing better than my loss here of 23.5. So the process that we're taking here, is something that would normally be automatic, but it's helpful to gain an understanding of what's happening when you run the learning algorithm. So what I can do, one strategy I could take is I can pick a lever. Let's pick the intercept lever, which is currently set to six, and then I can move it around, to figure out which direction improves my loss. So I'm reading the loss of title here, and as I move the lever, it decreases and then it starts increasing again. So I can say that, that is a good position for that first lever. Then I can move on to the next one, of the, the slope and I'll optimise that. OK, so this is a good position, and now I'll do the last one.

Now, are we done? I've optimised each lever once, but I think we can do better than this. And the reason for that is that every time we adjust one lever, the optimal position for the other levers has changed. So if I was to go back and re-optimise the first one. It's now in a different position. And so we can keep iterating this until we get to a solution where we just can't reduce the loss by moving any of the levers. That's the point where we're just gonna have to stop.

But this is rather tedious, and especially when we consider that all we're trying to do is get a very simple model with three parameters. Now, realistically, some of the models, some of the state of the art models you'll see in the literature are training billions of parameters. There's no way we could do this manually with billions of levers. So instead, what we're gonna do is look at what would happen if we threw the task to an automatic optimisation algorithm.

DESCRIPTION:

The shared screen moves down to two smaller graphs side by side.

The graph on the left is the ‘Model Loss’ line and plot graph from before. The graph on the right has black text that reads ‘Loss Landscape’ and has red concentric circles on it.

Above, three more slider bars have a black text header that reads ‘Automatic Model Selection’.

AL:

So if you scroll down. Now, instead of it controlling each parameter independently, we have a slider that just advances the system, advances the optimisation by one step than an automatic algorithm would take. So, as I…as I advance the steps forward, you'll see on the left is our model fitting the data, which is how we generally think about the problem. But on the right, you're seeing a loss landscape, which is how the optimisation algorithm is seeing the problem. So generally, an optimisation algorithm just sees the loss function and it wants to minimise that function.

Now, a good analogy here is that you're trying to descend a foggy mountain into the valley below. Now, the algorithm knows where it is now, but it can't see the destination. So if you look at these red lines in the loss landscape like a contour map, we're sitting on a big bowl and we want to get to the bottom. Now we can see where the centre is. But the algorithm can't, it just knows where it is now and it knows which direction is downhill. And so all it has to do is keep moving downhill and it will eventually get to the, to the bottom. At which point we will have an optimised model.

DESCRIPTION:

A black plotted point line on the second graph zig-zag’s up from the zero value on the X-Y axis to the middle of the concentric circles.

AL:

And so, these points, the steps it took to descend and now it's actually reached a model that was better than the one I chose before and was much, much easier. It's fully automatic. And so this is how we, we would use an automatic optimisation algorithm. So I believe there's a question that's come in related to this demo?

DAN:

Yeah, that's right. So they just wanted clarification. Is the model loss on the train or validation test set?

DESCRIPTION:

Dan on webcam.

AL:

Yes. So in this case, we haven't, we've just got the one data set. And the reason, well, partly just a simplified explanation. But secondly, because we have such a simple model, it would be very difficult to overfit this data with so few parameters. But certainly if we had a complex model with hundreds of parameters, we would have to be concerned about overfitting and then we would have held out data to evaluate the loss. But in this case, just for clarity and simplicity, we have just used one data set for essentially train and test.

DAN:

Cool, and there's another question here as well. ‘The axioms and assumptions in and of problem definition and design are clearly critical. Is algorithmic bias as much a reflexive question of embodied human intellect, aptitude and analytical insight, as it is of our…of our recursively amplifying historical inquiry in the data?’ It's a tricky one. We can always come back to that, if that's a tricky one in the next section.

AL:

I think that would be a good one when we're talking about responsible AI. But certainly at every step in the problem formulation, we can encode…well, we're implicitly encoding objectives and values as we go. And like, for example, one really important thing, is that it would be very easy for a developer to inadvertently make a decision in a line of code, that actually sets the strategic direction for the whole organisation developing the AI.

DESCRIPTION:

The ‘Ingredients of a modern AI system’ slide from earlier.

AL:

And so one thing we often teach in our data scientist course, is that instead of making a call about how to balance different outcomes. For example, excluding a good candidate, versus interviewing a poor candidate or disadvantaging a minority group, is we need to expose the behaviour of the system in a way that the stakeholders who are responsible for it can make an informed decision about whether to deploy the system, about whether they need to revise their goals, their data, whether they need to set an ethical objective to include in the loss function, for example. These are all strategies you could use to try and control for these problems.

Alright, so at this point, I will hand over Finn to talk about, more about AI classifications.

DESCRIPTION:

Black text reads ‘Classification problem: Child or Adult’

Below a simple plot graph has a ‘Y’ axis labelled ‘HEIGHT’ and an ‘X’ axis labelled ‘WEIGHT’. On the left of the graph, a grey arrow points from three small colour images of children to a group of three orange plots. To the right, a grey arrow points from three small colour images of adults to three blue plots higher up on the graph.

Finn on webcam.

FINN:

Alright. So you've just done the exercise where you've tweaked some model parameters and drawn a line through some data points. That might not seem anything like the applications that we talked about earlier, you know, in terms of recognising images or predicting which sentence will come next. But actually, as we'll see, they are very similar problems.

So we're going to jump into another slightly, very simplified problem, but where we are gonna try and figure out what category something belongs to. So here we've got children and adults, and we're now imagining we've got just two pieces of information about these people. We've got their weight and we've got their height. And so in this case, the orange dots represent the children who are both shorter and less heavy. And the blue dots represent the adults who are taller and heavier.

And now you could imagine if you wanted to build a model to figure out when we had a new person and we knew their height and weight, whether they were a child or an adult, you could sort of imagine drawing a line through…between these two points. And then we say, well, if we're on the left side of that line, we're gonna say, that's probably a child. And if we're on the right side of that line, we're gonna say that's probably an adult. So let's jump in and we'll actually train a neural network in the browser, so we're gonna go back, put me back into pop-out mode and we will go into the Techtonic page again.

DESCRIPTION:

Black text reads ‘A simple neural network’.

Below, a dot point has a hyperlink that reads ‘Go to techtonic.gradientinstitute.org’

Black text below reads ‘Machine Learning Demos’

Four blue links below read ‘Natural language completion demo’; ‘Image-to-Image demo’; ‘Small neural network classifier’; and ‘Complex neural network classifier’.

A red box encircles the ‘small neural network classifier’.

FINN:

And we'll go back up here and we're gonna go to the small neural network classifier examples, so just click to that. And you should see a page like this appear.

DESCRIPTION:

Four columns have headers that read, from left to right, ‘DATA’; ‘FEATURES’; ‘1 HIDDEN LAYER’ and ‘OUTPUT’.

One small square sits below the ‘DATA’ header; two small squares sit below the ‘FEATURES’ header, the first has one half orange and one half blue split vertically, and then horizontally in the second; a single square below the ‘1 HIDDEN LAYER’ header; and below the ‘OUTPUT’ header is a plot graph with numerous orange points on the lower left and blue points on the upper right.

Further information below each of the headers is outlined by Finn.

FINN:

So this problem is very much set up to mirror that child vs adult classification problem. We can see we've got our data on the right hand side here. So imagine these yellow points are the children and the blue points are the adults. And this is our neural network, which is a type of machine learning algorithm that is behind many of the advances that we've seen in AI in the past decade. And so this one is an incredibly simple one, with just one node in the middle here. So all that it's doing, is it's taking in these two inputs. So this is X1, you can think of this as the weight of the child lower to the left and higher to the right. And then we've got X2, which is the height or the height of the person I should say, which is again, lower to the bottom and higher to the top, and then both of those feed into this middle one, which just combines with some weight those two inputs.

So what we'll do is we'll just go up here and click the play button on the upper left, and that will make the algorithm run its optimisation and find a curve that separates those two points. So this thing is doing under the hood exactly what you did in the last example where you were drawing a line through the points. It's finding this curve to separate these points. And you see it did so very quickly and it's found a pretty nice diagonal line.

And you can actually jump in and you can see here the model has these weights. So these are just parameters. And if you want, you can click on one and change it. So let's say I change the weight on this parameter here. Of course, so it doesn't keep adapting for me. And I'll just ramp this way up and you can see now the model is just using that second feature. So it's just using the height of people to figure out whether they're a child or an adult now. And if I take it the other way, then we'll see it will start to flip over, oh it went beyond zero. Now it's just using the weight to decide whether someone's a child or an adult. But the…in this case the optimal solution is to use a combination of both of those things to, to get the best predicted model.

Alright. So that's a, a very simple problem where we can separate these two groups with just a straight line, but in reality, problems are gonna be more complicated than that. So let's jump up to, we're gonna switch data set here so if you go over to the left you can see we can select this circular data set and we'll just run that model again. Right. So we can see here that it has learnt a straight line again, but a straight line is not a very good way of separating these data points, because in this case, we kind of have a cluster of blue points basically in the centre. And the orange points are around the outside. So we want, we want the model to do is kind of learn of this centre area, is likely to contain, if we've got a point that's in the centre area, it's probably a blue and if we've got a point that's in the outer area, it's probably an orange.

Now, the problem is that our network here is just too simple. The model is too simple. It can only combine these two inputs and come up with some kind of a diagonal line. So we need to make it a bit more complicated. So let's just add another two nodes here. So if you go to that central neuron and you click plus, do it twice.

DESCRIPTION:

Finn adds two more small squares below the single square in the ‘1 HIDDEN LAYER’ column.

FINN:

Alright. Now, we've made the network a little bit more complicated. Let's run that again and see what happens.

DESCRIPTION:

The columns link together with blue and orange overlapping lines. The plot graph at the end has a cluster of blue points in the very centre surrounded by orange.

FINN:

Alright, so now you can see that it has done a much better job at learning the underlying relationship that separates these two groups. So if we were going to try and predict whether a new point that came in was blue or orange, this would do much better than the previous one. And we can also kind of visualise how the model is doing this. So if we look at these neurons in the centre here, we can see they've each learned a kind of a diagonal line by themselves. And when we combine those together, we get this kind of triangular shape that is the final outcome that's figuring out which category a given point belongs to.

So we've seen here that adding complexity can really help us solve more difficult problems, but too much complexity can cause problems. So we'll just go back to our Techtonic page, and we're gonna open up the ‘complex neural network classifier’ example. Click through to that one.

DESCRIPTION:

The Techtonic page from earlier. The link for the ‘complex neural network classifier’ is selected.

A similar page to the previous ‘small neural network classifier’ but with more squares in each column and more intersecting lines between them.

FINN:

So here I've just set up a much larger neural network, still small on the scale of modern AI applications, but much larger than the last one. Let's just run that on the same circle data set and see what happens. So first of all, it was looking pretty good. It's picked up the circular pattern. I can see there's far more nodes. So the first set of nodes or neurons have learned these kind of diagonal lines. And then as we go along, further neurons are starting to learn these much more complicated patterns. But as time goes on, you start to see that the model has learned some really very complicated and peculiar shapes. So it seems to be picking out, for example, on my screen, that there's a blue dot way down here. So it's saying, OK, this entire area down here must be blue along this line. And we can also see if, if your model is starting to sort of jitter around like mine, you can switch the learning rate. Slow it down a bit. So it's just a bit smoother.

The other thing we can see here is, we've actually now separated the loss on the training data and the loss on the test data. So this refers to that question which was asked earlier. What we want to achieve is that when we get some new data that the model was not trained on, that the model gives a good prediction. So we can actually click this little button down here and show the test data, on top of the old data, the test data is kind of some new points that come from the same underlying distribution. And what we see is that, yes, the model has learned these kind of little quirks that occur in the training data. Like my point down here is blue, but that's not helpful when it comes to new data, because that's just a quirk of the training data. And when we look at these losses that we're seeing here, although the training loss keeps getting better, the longer we let this network run, the test loss is actually getting worse. So we're not doing as well on new data as we want to be. And how well we do on new data is the only thing that matters.

Alright. So how can we fix this? Well, machine learning models will have some kind of a lever that controls their complexity. In this case, the lever is called regularisation. And we're gonna turn on L1 and run that again. So what this is actually doing under the hood, is it's telling the model, “Look, I want you to fit the data as best you can, just like before you, we'll penalise you if you don't fit the data.” But it's also adding another penalty saying, “Look, I want you to fit the data, but I want you to keep as many of these weights really small as you can and ideally zero.” And so what we see now is that the network has still learnt this kind of basic circular pattern in the middle, but it's done so it's automatically turned off most of the neurons and all of these waves and zeros. So it's automatically found a model that's just complicated enough to solve the underlying problem without overfitting. Alright...so we'll go back to there.

DESCRIPTION:

Black text reads ‘Model complexity is driven by data’.

Below, a line graph has its ‘Y’ axis labelled ‘Error’ and the ‘X’ axis labelled ‘Model Complexity’. A brown line and a blue line move down in a curve from the upper left down to the right. The brown line curves up again and is labelled ‘data not used for training’. The blue line continues on to the bottom right of the graph and reads ‘training data’. Black text in the middle of the graph reads ‘Too Simple’.

On the right of the graph is a blue half circle meter with a black needle. The needle angles to the left near text that reads ‘Simple’. The other end of the meter reads ‘Complex’.

FINN:

My slides...alright, so we've done the... we've seen that example, now essentially a key component of all modern AI systems is that they have this kind of a complexity lever that we can tune to make a model of just as complicated as we need it. So if we have a model that is too simple, we find that we get pretty high error on both the data we use to train the model on and on new data.

DESCRIPTION:

The black text in the middle of the graph switches to ‘Too Complex’. The black needle on the meter below is now angled over onto the ‘Complex’ side of the meter.

FINN:

On the other hand...on the other hand, if we use a model that is too complicated, it will fit the training data really, really well, but we'll get a high error on new data, which is a problem because new data is the data that you're deploying on. You might have trained the system to decide if someone was a child or an adult. You've trained it on data where you knew the answer, you're gonna deploy it on data where you don't know the answer. So it's entirely that the new data that we care about.

DESCRIPTION:

A third version of the graph reads ‘Just right’ in the middle. The black needle is about two-thirds to the right, closer to the ‘Complex’ side of the meter.

FINN:

So we wanna find this spot that's just right where we have our model. It's not too complex and not too simple. And the great thing is that we can do this automatically too just the same way as we can automatically tune the model parameters. And the way we do this is through a technique called cross-validation, where we just repeatedly try different complexity settings with the complexity lever, and we check on new data, how it's doing.

Alright, OK. So we're just about to try and give you guys a five minute break. After the break we'll come back and see what this looks like in realistic systems where we have, you know, thousands of billions of parameters and how this like simple neural network that you just train in your browser is similar to the modern AI systems that can solve all of these amazing problems.

And then we'll also jump in to talking about responsible AI. So we'll go to a brief break now and we'll see you shortly.

DESCRIPTION:

TEXT: Break 00:10:00.

The ‘Techtonic 2.0’ home page from earlier. Countdown clock begins.

(MUSIC PLAYS)

DESCRIPTION:

Finn returns on webcam.

FINN:

Alright, welcome back everyone, I hope you've enjoyed the first half of the session. So I'll just remind everyone that, yes, there will be a recording of this session and you will be able to access it. A couple new questions have come in around filtering or modifying datasets to deal with complexity or issues around values and ethics. So coming to the first question, if you've got datasets that don't encode the values of your organisation, can you kind of solve that bias problem by filtering or doing a check on the data set? In principle, yes, when you know that there is a data set that contains bias or doesn't affect your organisation values, you can help mitigate that by modifying the dataset or improving the dataset. Or you can deal with that by modifying the model, the algorithm that uses that data to handle the fact that we know the dataset itself is biased. And which of those options is gonna be best depends very much on the context of the problem and what access to data that you have.

A particular thing to be cautious of is, one approach that's often suggested, so for example, if you don't want your model to be sexist, you just remove anything that is a gender feature from the model size, so don't explicitly tell the model somebody's gender. Unfortunately, that will not ensure that the model can't be sexist because in these modern systems, we tend to feed in so many features, you know, potentially thousands or tens of thousands of features that the model can infer that information from something else. So if it was a résumé, even if gender has explicitly been removed, it can infer it from the pattern of writing in the résumé, it can infer it from which school you attended, a combination of many other features. So just removing a protected attributes generally doesn't work.

Similarly, if we're worried about the model being too complex and over fitting, can we deal with that by changing the data in some way again? Yes, we can deal with that by reducing the kind of amount of information we feed into the model, but we don't - that kind of decision is really part of the modelling, the algorithm. We don't typically do that manually, we just let the algorithm figure that out because then it can find the optimal much better than a human could.

Alright, so now I am going to hand back over to Al, who is going to talk to you about some of the more complicated, deep learning systems.

DESCRIPTION:

Al takes Finn’s place on the webcam.

On the right, black text reads ‘Classifying with more features’.

Below, the ‘Height-Weight’ Adult-Child graph from earlier with three orange plot points and three blue plot points.

On the right, two black and white images of a dog, above, and a cat, below. A small red square on each image is joined to a magnification on the right. Black text below the dog magnification reads ‘Pixel values = […, 178, 94, 91, 160, ..’. On the cat image it reads ‘Pixel values = […, 2, 13, 22, 15, …]’

AL:

Alright, thanks Fin. So before the break, we looked at some simple illustrations of how an AI system could work. For example, we were looking at that adult child example where we had a two dimensional problem, height and weight. For example, a child might be 90 centimetres and 21 kilograms, which is the labelled Orange Point, or an adult might be 180 centimetres, 78 Kilograms. So the system used these numbers together to make a prediction.

But in real systems, we're likely to have a much richer feature set on this, which means that we're going to need more sophisticated models to pick up the patterns. For example, let's go back to the original cats versus dogs example. So this is a task that's easy for us to do, but it'd be really hard to write down the rules in terms of pixels. And if we think about an image of a cat and an image of a dog, we're not looking at two dimensions anymore. So an image would typically be represented by hundreds to millions of numbers, even the small 100 by 100 grayscale image like this would have 10,000 values in it. And if we switched to colour, we would have to triple that again because we need red, green and blue channels.

So the concept here is, we need to think about images as a really rich data source where it is an array of numbers. And once you start thinking about images as an array of numbers, we can realise that the basic principle of machine learning applied to images is the same as machine learning applied to height and weight, in that the algorithm is just looking for a boundary or a region in that space that separates, in this case, dogs from cats in the data.

DESCRIPTION:

Black text reads ‘Classifying with more features’.

Below, a sideways flow graph-block diagram has various blue, green and red rectangles that are joined by black arrows and lines. Three orange rectangles and a white elliptical circle form three different end points along the graph.

AL:

So although the principle is the same because the data is so high dimensional, it calls for more complex and sophisticated models because they need to have the capacity to learn these complex relationships. So this figure, this block diagram depicts a structure used for a model called GoogLeNet, a 22-layer deep convolutional network. So each of these boxes represents a different processing step and they're coloured by the type of operation.

So obviously, this model is much, much more complex than the Toy Neural Network that we were looking at in the interactive exercise. In fact, it has seven million parameters on this behemoth structure, is trained on 150 gigabytes of images in the image in that database, which means that the training of this model is an enormous computational task involving clusters of computers. Then once it's trained, it's a very versatile model that can be used in all sorts of computer vision tasks to distinguish between thousands of different types of objects and creatures.

DESCRIPTION:

Black text reads ‘Fireboat or Streetcar?’

Below, two rows labelled ‘Fireboat’ and ‘Streetcar’. The top row has the colour image of a red firefighting boat on the left and the bottom row a streetcar. To the right, four circles on each row, orange on the top, yellow on the bottom, are labelled from 1 to 4 and have various small tiles of colour images and blank spaces inside them.

AL:

Now, people often say that neural networks are a black box. Inputs go in and predictions come out and that we can't possibly understand what the seven million… sorry… ten million parameters are doing under the hood. But increasingly, strategies are being developed to get insight about what the neurons are encoding under the hood in these models. So the examples I'm showing on this slide come from a tool called an activation atlas that visualise intermediate representations that the classifier has learned. So it's the interpreter ability tool to help us understand what's going on inside that complex model.

Now we can get a rough idea of how the model is learning to categorise images into thousands of categories by looking at, using this atlas, which images the neurons in each layer are responding to or which features in the images, I should say, and also, which neurons are being activated for any particular image. So in this example, we're looking at the response of the network to a picture of fireboat and a picture of a streetcar. And we see that there are parts of the network that are relevant to this problem that respond to windows on vehicles, crane-like structures, water and buildings. So if you were to zoom in on the columns here, these little image patches, you would see the textures and the features that that neuron is responding to.

So we can see that the fireboat on the top activates windows, crane-like structures and water, whereas the streetcar below activates windows, a little bit of the crane-like structures and lots of buildings, but not for water. And this further suggests to us that to some degree, the algorithm is using the background scene of these images to gain context. So we can hypothesise this model here, a fireboat with buildings in their background, it might well misclassify it for streetcar.

DESCRIPTION:

Black text reads ‘Whale Shark or Great White?’

Below, staggered columns of small colour images. On the right, three images of a dorsal fin above the water with statistics below each. The middle and far right image also have a baseball ball and mitt in the corner of the image, small and large respectively.

AL:

We can use this insight gained from the activation atlas to predict what a model might be going to do for some synthetic inputs. Though we've noted that the context in which the object is placed might be more important for a computer vision system like this than for a person with their general intelligence. So the part on the left shows us that there are neurons in this network that respond to both baseballs and great white sharks. And if we examine the features that that neuron is responding to, we recognise it as a pattern of red spikes on a white background. Basically, it's responding to both the teeth of a great white shark and also the stitching on a baseball.

So a classifier to predict whether a creature is a grey whale or a great white shark seems to be using this feature. So we could…we could change the decision of the classifier by superimposing a baseball over the head of a grey whale and… So in the first example on the left, we haven't doctored the image at all and the, and the algorithm detects correctly a grey whale. But then if we put a baseball where its head would be, we're now adding a response to... that resembles the teeth, this neuron that detects the teeth of a shark and it has actually tricked the model into saying with high confidence, it's a great white shark. And of course, if we start making the baseball take up most of the image, then it'll switch again to classifying the whole image as a picture of a baseball.

DESCRIPTION:

Black text reads ‘Adversarial Inputs’.

Below a large colour image of a ‘STOP’ sign has two small white rectangles and two black rectangles stuck on it. A grey arrow points to a second image on the right of a white speed limit sign with a red circle around the outside and black text within that reads ‘LIMITED TO 45 MPH’.

AL:

So in the previous slide we talking about modifying the image, but it's also possible to modify the object itself to confuse the algorithm. Now usually, little modifications like adding stickers wouldn't do anything because the algorithm is fairly robust. But in the picture here, you can… by adding these stickers in the precise configuration shown there, you can trick the algorithm to think this stop sign is a 45 mile per hour sign.

So these stickers are precisely designed to confuse a particular algorithm, and we call these inputs that intentionally confuse an AI, adversarial inputs, which you are only able to create if you have access to the model and it's internal representations. Which then allows you to design inputs that activate particular internal representations that are, in this case, are important for distinguishing between the stop sign and the 45 sign.

DESCRIPTION:

Black text reads ‘Prediction to Decision’.

Below, a colour image of a wet road at night. Arrows point to the right to an algorithm diagram (from earlier) and then on to a bar graph. The top row of the graph reads ‘Plastic bag – 94%’ and the bottom row ‘Pedestrian – 6%’.

Below, another arrow points to a red ‘Stop’ sign and a green ‘Go’ sign.

AL:

To continue the driving example, another important thing to keep in mind when we make predictions is that we have to consider what they're being used for. It's certainly not always the case that you should take the most likely answer from a classifier. Suppose that we have a self-driving car that is using a computer vision system and it has just made a detection of something on the road, it predicts 94% probability that it's a plastic bag with six percent probability that it's a pedestrian because it's at night and the lighting conditions are poor and it's hard to tell.

So, I mean, the most likely outcome is that it's a plastic bag. But should the car brake or continue? Given that it’s a six percent chance that it's a pedestrian, clearly the car should stop, because that's not an acceptable risk.  So the careful consideration of the risk and consequences of the decisions that come from the predictions made by an AI system is the foundation of responsible AI, which is a good point to hand over to Finn to discuss this important aspect of the technology.

DESCRIPTION:

Finn returns to the webcam in the top left.

On the right, black text reads ‘ Unintended Consequences’.

Below, a colour image has a circular robotic ‘Roomba’ vacuum cleaner on a polished wooden floor. Black text below reads ‘Go faster and avoid (detecting) crashes’.

FINN:

Alright, great. So this last section of today's primer is going to be all about responsible AI.

DESCRIPTION:

White text on black reads ‘Responsible AI’.

Below, the blue three-quarter circle Gradient Institute logo.

FINN:

I’ll just remind you we're very happy to take questions so, if you have a question please put it into the Q&A box. Right, so when we are thinking about responsible AI, one of the things we have to be really careful about is unintended consequences following from AI systems.

So here's a silly example of this, so an engineer, they had a Roomba, but they were annoyed how slow it went, so they decided they would use AI to kind of reprogram it and they, they set the objective of the AI to go as fast as possible cleaning the room, but to avoid crashes. Now, unfortunately, the model of Roomba that they had only had a crash sensor around the front of the machine. And what the AI figured out was, it could satisfy the objectives that the engineer had set by driving at top speed backwards. And although it crashed into all of his furniture, it never detected it was doing so because it didn't ever trigger the bump sensor.

Now, that might seem hilarious, but when you have AI systems making decisions about whether or not to release someone on bail or what ads to target to people, you can have much more serious consequences and you may have seen kind of headlines around this, you know, there is software that's used in the US to help decide whether or not to release someone on bail, but is biased in a particular way against black inmates. We have seen cases where disadvantaged people have been algorithmically targeted with ads for high interest credit cards, or poor value, for-profit-colleges, for example. There have been issues with algorithms that recommend candidates for jobs and also issues around representation. So if you search for something, for example, and you do an image search for CEO, there was a period when, for a while, Barbie was the first female result. And, you know, this is kind of perpetuating stereotypes that exist in the world and potentially reinforcing them.

DESCRIPTION:

Black text reads ‘Most problems are Hidden’.

Below, a vector image of an iceberg floating in the water. The white section above the water is labelled ‘Obvious headline grabbers’ in black text. The blue section below is labelled ‘Insidious, long-term harm’.

FINN:

Alright, so those kind of problems are really just the tip of the iceberg in that they are the obvious headline grabbers. Often we see cases related to kind of tech companies because it's easier to get data and understand what's going on in those systems, whereas in many other systems, in government or in finance that are really still making decisions that impact people's lives, it's a lot harder to actually check and understand whether or not these types of bias or problems or unintended consequences are happening or not, because it's hard to see it at the level of one individual, you really only detect it when you look at distributions about, across groups of people.

DESCRIPTION:

Black text reads ‘Why does this Happen?’

Below, black text dot points (outlined by Finn).

On the right, a colour image of the ‘Genie’ from Disney’s ‘Aladdin’.

FINN:

So why might we have kind of unintended outcomes from algorithmic systems? Is it, you know, it seems unlikely that there might be cases of, you know, malicious intent. There have certainly been AI systems that have been developed... for settings that very clearly don't have social licence. But a lot of the cases that we've seen really seem to be unintentional, unintended consequences. And the reason that this happens is that AI systems, first of all, they obey the letter, not the intent of any instructions we're giving them. So it's a bit like a genie, you get exactly what you asked for, but probably not what you want, although in the case of the algorithm, there's no malicious intent behind that.

Secondly, they can only view the world through the data that we give them. So if that data is biased, then the algorithm will reflect that bias. And they have no inbuilt minimum moral constraint. They just don't have the context of broader understanding to know that if something comes in, that they should override that decision with some other piece of information.

DESCRIPTION:

Black text reads ‘An example’.

Further black text below reads ‘”Care management” program enrollment’

Below, an algorithmic diagram labelled ‘Model’; a ‘Risk Score’ of ‘7.2’; and a colour image of two people holding hands labelled ‘Care management’. All three are linked by two arrows that point right.

Below, two black text dot points, plus an additional point added later (all outlined by Fin).

FINN:

So here's a kind of example of that. This was a real system in the US that was used to help drive health care decisions for over 70 million people. And the goal of the system was to identify people who had ongoing chronic health conditions, who needed additional support to manage those conditions so as to be able to reduce hospitalisations and problems in the future. And so, they were going to enrol those people into a care management program.

So what they instructed the AI system to do was to predict as accurately as possible what the future health care costs for a whole set of patients would be. And then they were going to basically take people for whom the predicted future health care costs were high and enrol them into this chronic health care management program to help them manage their conditions.

And unfortunately, what turned out to happen was, a group of researchers analysed the system and discovered that actually black people were being really disadvantaged by it. So when they compared a white person and a black person who had very similar health indicators, the black person was much less likely to be enrolled into the care management program. And the reason that that happened was because the system had been told to predict future health care cost rather than future health care need, it had actually detected that black people had poorer access to health care, and so even if they were equally sick, on average they cost less. And so, the system decided, therefore, they didn't need support. And so, this meant that the very problem, the lack of access to health care was perpetuated by the algorithm by then denying those same people health care into the future. And, of course, the algorithm can only see the world through the data that it is given, it doesn't have access to any additional context on systematic disadvantage.

DESCRIPTION:

Black text reads ‘Should we use AI?’

Green text below reads ‘Do we have the legal & social licence to do so?’

Below, a colour image meme has actor Jeff Goldblum and white text that reads ‘YOU WERE SO PREOCCUPIED WITH WHETHER OR NOT YOU COULD YOU DIDN’T STOP TO THINK IF YOU SHOULD’.

FINN:

Alright, another question that we really need to ask when we are building an AI system is, just because we can, should we? Do we have the legal and social licence to do so? So one thing that AI has done a lot is really reduced the cost of certain types of things. So surveillance is a really good example of that. Surveillance used to be incredibly costly because we would have to have people to watch cameras all the time, whereas AI systems can potentially really speed that up. And so, it might be possible to use surveillance in a cases where, you know, we're really looking at minor crimes, like jaywalking, for example, crossing the street when the light wasn't green. You know, it's feasible to do that, but we may very well not have the social licence to do so.

DESCRIPTION:

Black text reads ‘Should we use AI?’

Dot points below (outlined by Finn).

FINN:

If we do have an application where we feel like we have social licence to do it, there are some additional technical questions we should ask that really come down to what is the risk if the system is not going to work well, is going to have unintended consequences? So, first thing is really critical, we have a clear objective. Because AI systems are not general intelligence, they need a very specific goal to follow. So, we need to have a clear goal of the system. And we need to be able to quantify that goal. So really measure performance towards that goal because that's what we're going to then feed back into the system. Then we need there to be a kind of well-defined sandbox within which the algorithm can operate, where its behaviour is constrained so that we're not going to end up in a system where it does something we hadn't even considered, that is clearly outside of kind of what is reasonable, ethical behaviour. And we need to be able to detect if there are unintended side effects occurring.

The next set of things are things kind of we’d hope to have or we need to be cautious about. So, we need to have quite a lot of data for AI systems to work. Extremely rare events can be a problem, because as we've seen, AI systems don't generalise well to things that they haven't seen before. So, if extreme events are an issue, then the AI system will not know how to respond to them. If we're going to change, or select policy changes, then we're really changing the system underneath the AI. And as we've seen before, what AI systems are great at doing is we've got some problem that's kind of fixed, and it then predicts what's going to happen into the future. It's not so good at predicting what's going to happen if we change everything.

And the final point is very similar, does the environment in which this algorithm is sitting change very rapidly over time? So, for example, I'm sure that the COVID-19 pandemic through many AI systems kind of briefly haywire because demand for certain products fluctuated in ways that had never occurred before in the kind of data sets that they were trained on.

And so, finally, you kind of think, well, ask what is the worst that could happen from this system, whether we're using an AI or whether we've got a human driven system, and think about how you would mitigate that case.

DESCRIPTION:

Black text reads ‘What’s being done?’

Below, four words in black text, one above the other, have arrows linking them downwards – ‘Principles’, ‘Frameworks’, ‘Standards’, ‘Regulation’.

On the right, a circular diagram with coloured concentric circles has coloured dots on them in different segment clusters – yellow, green, red, blue and pink. Small black text labels around the circumference.

FINN:

So that might seem like a lot. What's being done to kind of help people developing or interested in using AI systems deal with these issues. Many organisations and groups and countries have developed principles or frameworks, which are gradually, kind of, moving towards more concrete standards and very slowly filtering down into regulation.

DESCRIPTION:

Black text reads ‘Australian Government’s AI Ethics Principles’.

Below, the principles read ‘P1. Human, social and environmental wellbeing’; ‘P2. Human-centred values’; ‘P3. Fairness’; ‘P4. Privacy protection and security’; ‘P5. Reliability and safety’; ‘P6. Transparency and explainability’; ‘P7. Contestability’; ‘P8. Accountability’.

On the right, a small screenshot of the Australian Government ‘AI Ethics Framework’ webpage.

FINN:

And the Australian Government has released its AI ethics principles. And there's going to be a session this afternoon on that if you are interested. And we're just going to talk briefly about two of those kind of principles here just because of time. So, we're going to talk about transparency and fairness, just a brief taster into these things.

DESCRIPTION:

Black text reads ‘Transparency’.

Below, four black and white icons – a stick figure joined to a three-way branch of two crosses and a tick; a woman sits at a computer; a magnifying glass; a stick figure torso. Accompanying text with each icon (outlined by Finn).

FINN:

Alright, so transparency is really about understanding how these systems are operating, or understanding how a decision is being made. But there's no kind of one thing that represents transparency. It's not really a single thing, because different people use - having different roles, will need different things. So, if we think about this, who might want a system to be transparent? Well, firstly, we've got the people who are being impacted by a decision. And they might need to contest that decision if they believe that it was incorrect or made on unreasonable basis or they might want to adapt to that decision. So, maybe they were denied credit and they'd like to know what they need to do in order to qualify for credit in the future.

Then we've got people who are designing or responsible for the design of an AI system. And they really need to be able to detect cases where the system is not meeting the objectives that they intended it to.

External regulators or researchers might also want to detect whether the system is meeting its objectives. They might also want to understand exactly what objectives are encoded in some particular system. You know, maybe there is a system that appears to be producing negative outcomes for a disadvantaged group. Is that because the objective is explicitly encoded in the system, has an overemphasis on profit, for example, and has ignored concerns around fairness or bias? Or is it due to technical issues that mean that the intended objective is not encoded?

And, finally, the public as a whole rather than just those who are impacted by decision want to know whether this system is kind of something that has general social licence, and to do things like prevent corruption more broadly.

DESCRIPTION:

Black text reads ‘Transparency’.

Below, two colour images side by side of a wolf-like husky dog. The second image on the right is predominantly greyed out.

Black text below the left image reads ‘a) Husky classified as wolf’. The second image ‘(b) Explanation’.

FINN:

So, here's an example of a transparency tool that can help people who are developing AI systems understand whether or not they are doing what is intended. So, this case, this system is supposed to be separating huskies from wolves. And on the left we've got an example that's gone wrong, a husky has been classified as a wolf. And on the right, there's the explanation as to why we got this classification. And so the areas that are not grey are the parts of the image that the neural network, that made this classification, cares about it. These are the parts of the image that the model thinks are important in predicting that this is a wolf.

And the thing that we notice here is that actually the dog is hardly in the part of the image that the network is caring about. It seems to be looking just at the background here, which is snow. And what's actually happened here is that on the data that was used to train this model, the dogs were all on grass, or in urban landscapes, and the wolves were all on snowy backgrounds. And so, what the model has learned, it's not actually a wolf detector, it's a snow detector.

And this will be problematic if, for example, you wanted to deploy this camera in a national park, where this background is going to be constant, and you want to know if you're seeing a wolf come past or somebody's dog come past. So, this will be something that would warn developers or designers of a system that it's not actually learning dog versus wolf, it's learnt snow versus not snow, which was not the intended objective.

Alright, the other key principle that we're going to talk about, I'm going to hand over to Simon to round up today's session.

DESCRIPTION:

Simon takes Finn’s place on the webcam.

Black text on the right reads ‘Fairness’.

Below a line of blue car and green car icons in a line. Black text reads ‘11%’ on the blue side and ‘9%’ on the green.

Black text below reads ‘Police have resources to stop 1/5 cars’.

SIMON:

Thanks very much, Fin. Yeah, so this other area that we'll finish off with is algorithmic fairness, which, I mean, we've seen throughout the day that algorithmic systems or AI systems can kind of have huge benefits and huge harms as well. And algorithmic fairness kind of looks at how these benefits and harms are like distributed across the population.

So, we're going to just start with like a motivating example where you can imagine, like a case where you're trying to optimise some kind of a short-term objective, right. Suppose that the police force are randomly breath testing drivers and they have limited resources. So, we’ll consider a population of like two groups of drivers, right. We've got people who drive blue cars, blue drivers and green drivers, OK. So, the blue group have like a slightly higher prevalence of drunk drivers, let's say like 11% versus, say, 9% of the green drivers. And the police have got the resources to stop one in every five cars with a breathalyser.

So, if we gather historical data and pass this on to an AI system to try and maximise the accuracy of these tests, and the AI system would produce a policy that looks something like this.

DESCRIPTION:

A round cornered pale yellow rectangle has black text inside that reads ‘Ranked Selection: all those stopped are Blue. 22% of drink drivers are caught’.

Below, a row of blue car icons and a black stick figure icon has a label that reads ‘Policy’. Two additional rows below are labelled ‘Caught:’ with a row of blue bottle icons and ‘Undetected:’ that has a longer row of blue bottles and more green bottles added to it.

SIMON:

OK, so using the one feature that it has and knowing that there's a higher prevalence of drunk drivers in blue cars, and it's going to pull over as many blue cars as possible, right. And with that, we seem to catch about 22% of the drunk drivers. But I think a lot of us would agree that looking at this, there seems to be some kind of there are other issues here, right. Well, first of all, it seems to like unfairly or disproportionately, let's say, target blue drivers. So, they might feel a little bit hard done by this kind of a policy. There's also kind of long term impacts of green drivers eventually realising that they never get pulled over. OK, so no doubt that 9% prevalence of drunk driving might actually change in the green driver population.

DESCRIPTION:

A second rectangle is labelled ‘Random Selection: half of those stopped are Blue. 20% of drink drivers are caught.’ Below, an even split of blue to green icons in the three rows.

SIMON:

If we look at something like say, let's just a random selection. We don't catch as many drunk drivers. But I think that we get a much fairer looking policy, right, and it mitigates against those kind of harms that I suggested previously. So, there's obviously like kind of a spectrum here where on one side we have like a very accurate system, and another side we have a very random system, right. And it's not always the case especially when we're applying our system to people that we want to maximise accuracy at the cost of everything else. There are often competing objectives at play here and fairness is one of these objectives.

So, like, we can kind of look at… Just because an AI system is driven by data and driven by algorithms that are a matter of statistics in no way implies…means that the decisions it produces are like these kind of objective truth, OK. And we've seen that already today.

DESCRIPTION:

Black text reads ‘AI pipeline’.

Below a flow-type diagram has a circle with a cluster of stick figure torso icons inside labelled ‘People’. An orange arrow mores to the left and curves down to a grey funnel with black cogs on it and five icons jumbled above: a DNA strand, the Twitter and Facebook icons, a plane and a barrel shape. A black text label reads ‘Data’.

A second orange arrow points down and curves to the right to an algorithm diagram labelled ‘AI System’.

A final arrow points right and curves up to a stick figure torso that has a tick and a cross on either side. Text reads ‘Predictions and Decisions’.

SIMON:

But let's maybe look at some ways in which kind of disparities can kind of creep into like a (AUDIO DROPS). So, here's a very simplified version of an AI pipeline, right. We can kind of think of people that generate data, right. The data kind of gets sucked up from all these different kind of sources. Whether you're buying things online, filling out forms, uploading photos, whatever it is, OK, these get ingested by AI systems, OK. These...processes the data, and make some kind of a decision, right. So that's a very simplified pipeline. But let's look at maybe places where disparities between groups can creep in here.

DESCRIPTION:

Black text reads ‘Sources of disparity’.

Below, a second diagram has the ‘People’ icon labelled in red text ‘Societal Inequity’. The arrow that points to the left and curves down to the ‘Data’ funnel is labelled ‘Over/Under representation’ in red text.

The second arrow that points down and curves to the algorithm diagram is labelled ‘Records what did happen, not what should have.’ A red text label below the algorithm diagram reads ‘Mis-specified intent’.

The third arrow that curves to the right and up to the torso ‘Predictions and Decisions’ icon is labelled ‘Discriminatory treatment & outcomes’.  An additional arrow now points up and curves to the left to the first ‘People’ icon and is labelled ‘Feedback effect’ in red text.

SIMON:

So, just in this very simple model, we can kind of think for quite a few, right. We can kind of see that we live in a world that isn't necessarily fair by itself, right. It's just the reality of the world we live in. People come from different upbringings, different circumstances affect their lives. So, inherently, we're going to get differences between people, or between groups. Often we find, too, in data that some groups are over represented or underrepresented in the data. And as a result of this, the AI system, if a group is underrepresented, might perform quite poorly when we try to make predictions about that group.

It's also important to remember that, like, the labels that we give the machine, like we tell it to try to match these labels, right, like, try to predict whether this person should or shouldn't get a mortgage based on the people that we've given mortgage to in the past, right. Those labels refer to what did happen and not what should have happened. So, any kind of biases that were inherent in previous systems, whether it be a person or another AI system, will be copied and replicated if we naively apply AI to this, right, and also ingrain these kind of issues.

There's also mis-specified intent. Often we see when we apply an out of the box AI system, it's interested in maximising accuracy. And that comes at the cost of everything else, right. So, if we care about maximising accuracy at the cost of everything else, then you know, OK, go for it but you're taking a lot of risks there, right. If there are other things you care about like the well-being of your customers, or the AI's impact on disadvantaged groups, you need to explicitly encode this in your system and monitor it in order for your AI system to have - to care about it, right. It has no common sense and no context outside of anything that isn't just in the data or in its code.

And then, of course, then we have this feedback loop as well, right. So, the decisions that the AI system makes, especially if it's an AI system that affects millions of people, right, can inflict a change on the population. And this change, if it's kind of some kind of systemic bias, can ingrain certain disadvantages. And then over time when we retrain the system, it'll see that this difference exists and then continue to perpetuate it. You can think of like a hiring policy that has a slight bias towards hiring one type of person over another. Over time there's been plenty of models that show this difference, which might have been slight at the start, and any AI system continuing to follow this route will just exacerbate this.

Also you can look at, say, the previous example, where we're pulling over green and blue drivers. If we only stopped blue drivers, and we retrain an algorithm on data that only consists of blue drivers, it's gonna have no reason to want to pull over green drivers. Now, there are many ways that we can try to intervene in this process, and try to mitigate some of these problems.

DESCRIPTION:

Black text reads ‘Mitigating harms’.

A third version of the previous diagram has blue text labels replacing the red ones. They read, moving anti-clockwise, ‘Gather Better Data’; ‘Data Debiasing’; ‘Modified Loss Function’; ‘Adjusted Decision Thresholds’ and ‘Corrective Actions’.

SIMON:

Some are much easier than others, right. I can talk about a few of them here at a very high level. So, one is, like a nice one to do is to get better data. That improves a lot of things that's downstream of that. Not always possible, though, right. Like, ideally, you can solve a lot of problems if you have a group that's underrepresented, like gathering more data about that group. Sometimes, though it's just not possible. You can imagine your training system on arrests, OK. What does gathering more data involve about that group? Does it require you to arrest people that you wouldn't normally have arrested? Or if you're in insurance, are you hoping that there's going to be some kind of a cyclone in a certain region so you can gather more data about that region. So, we're often just at the mercy of the data that we're given as opposed to being actively able to acquire new data.

You can sort the data in such a way that you can kind of try to mask sensitive attributes from the system while still maintaining the signal that you're interested in. It's kind of like a data pre-processing step. There's a lot of work to be done in encoding what we really care about into the AI's objectives. So, instead of just maximising accuracy, which we saw earlier, where Al was talking you through about like the loss, in this case, where loss was just your distance from the data points. You have other factors too that contribute to the loss. So, you look at, like, you also include how different is the accuracy between different groups, OK. And then if it's a large difference, that adds to the loss. And then as the AI system begins to learn and train itself, it'll be rewarded for producing fairer outcomes.

We can also kind of adjust the threshold of the system after it produces its results to try and compensate for biases that we know we've somehow introduced by training on say... like under- or over-represented data. And then, of course, if your system, you know, if you have a certain social licence, if you were like a big enough organisation, maybe the government or a bank or like an insurer or something large like this, you might have some kind of obligation to kind of introduce corrective actions where you can actively start to change existing societal inequities so that over time by rolling out a policy on such a large scale, we can kind of correct some of the existing inequities in society and then feed through again in this feedback loop and improve things for people rather than trying to ingrain disadvantage.

There's a lot of ways we can address this, but it requires us to rethink about what we care about, right. This isn't like a purely technical solution by any means, right. Accuracy is a nice metric because it's easy to kind of quantify and say exactly what we mean when we talk about accuracy. It's the fraction of number of correct answers compared to the number of incorrect answers. Something like fair is very hard. Right. It's like some kind of an ethical consideration we have to make. We have to make calls like what do we mean when we say fair, right? There's no like universal answer for this. What are the things we care about and who are the people that we care about being impacted by these systems? And there's a lot of work being done on who do we bring in, like shareholder or stakeholder engagement. Sort of like these are kind of the effected groups here. In what ways is the system effecting them and how can we kind of alleviate this? And how do we balance these kind of trade-offs between maximising accuracy, but while also producing like a fair system.

DESCRIPTION:

Black text reads ‘Summary’.

Below black text dot points (outlined by Simon).

SIMON:

So, we just kind of briefly touched on some very general topics in AI at a very kind of high level. We could obviously go very deep into a lot of these, and we'd love to do, like, at a later date. But just to summarise maybe what we (AUDIO DROPS) you know, in the session today.

So, we know that AI systems are extremely powerful, and are becoming more and more ubiquitous, and are changing how industry operates, right. But an AI system at its core, is actually a very simple system, right. It's just a system that finds patterns in data and changes its behaviour to meet some kind of a measurable goal, a very specific narrow goal as we've seen. As soon as we stray any way, or introduce kind of data that isn't really representative of the data it's seen in the past, we can see how they can fall over. They have no kind of common sense.

Used carefully, an AI system can unlock huge benefits. Right now, I think it kind of speaks to the fact that we're seeing more and more of them today kind of, you know, the proof is in the pudding, I guess, that they do offer huge benefits to us, and yet no common sense. So, it's up to us to try and ingrain our own kind of moral values into these systems, because without them they will just actually always maximise accuracy.

DESCRIPTION:

Black text reads ‘Thank you’.

Below, contact information for Gradient Institute.

On the right, a black text header that reads ‘Contact us for:’ Below, dot points (outlined by Simon).

A colour image at the bottom has three men and a woman using a whiteboard inside a room.

SIMON:

So, just before I wrap up, I'd just like to say that we're all here working at Gradient. And you can contact us if you have obviously any questions about what we covered today. But we also kind of provide training for organisations that are across the kind of different levels of organisations from board, management, technical courses where we look a lot more kind of topics we've covered here today and dive into actually getting stuck into the details and the algorithms themselves. And a data scientist course too. These are like one or two day courses, and the board courses are shorter.

We also provide technical consulting for a lot of organisations, both public and private. And we do algorithmic impact assessments too. So, organisations that are concerned about the potential effects of their algorithm, we are happy to give you advice as to how to monitor the different impacts that your algorithm might be having and ways you can mitigate it. Obviously, too, we also do some research. And we have also contributed to numerous policies, OK.

Well, thanks... (AUDIO DROPS) Should we pass to questions? We have some time for questions. So, if anyone would like to add anything to the question box, we're happy to discuss them here. I realised that we covered a lot there on a very little time, but hopefully it gives you a flavour for the kind of stuff that's out there. And we'll jump over to Dan there maybe, and to see if there's anything too that we can elaborate on.

DESCRIPTION:

Dan on webcam alongside Simon’s window.

DAN:

Yeah, definitely. We have a good one here. So, they asked, ‘there have been debates about bias stemming from data. But more recently, inductive biases propagated by the choice of model architecture is also highlighted. What are your thoughts on that?’

SIMON:

Alright, reading the question there off of a bit. Oh, it's about the choice about model architectures? OK. Yeah, trying to think...

DAN:

So, I can think of an answer. So, we've definitely noticed in the past where definitely by choosing different models, you'll have different errors. So, one example is if you have some under representative population in your data, sometimes choosing a simple model that sort of can't necessarily model or capture the patterns in that particular cohort in that portion well, it'll make more errors than, say, a slightly more complicated model that has the flexibility to model kind of the relationships between that particular population and the outcomes. So, yeah, we've definitely seen that model choice can also sort of introduce particular biases potentially in the form of different types of errors on different cohorts.

SIMON:

Yeah, also someone previously was asking questions. Someone previously asked a question just about the kind of axioms and assumptions we're making about the AI approach altogether, rather than just biases in the data. And whether that's introducing some form of bias. And, yeah, we've seen that too, right. Like you can think about in order for the AI system to try and optimise some objective, we need to have to quantify exactly what we mean by the objective. And even just the process of turning reality into data itself is like this kind of ethical solution, right. Does - things get lost at each step of the way, right. Like, even making a problem a classification problem is like a big simplification of many instances. Like, if you're looking at who to intervene on, and you want to make some kind of a classification problem out of that, you have to kind of turn the training data into like people who we correctly intervened on, people that we didn't correctly intervene on, and then you're turning into like a binary problem. Whereas there might be like a spectrum of possible solutions here. But often we see that just by framing the problem as a machine learning classification problem, we're introducing our own sets of biases there too.

DAN:

We have another question here. ‘How does AI accommodate variables dependency on others? That is, the ripple effect of change to one variable impacting another by perhaps, at a varying degree over time?’

SIMON:

Yeah. So, like, AI certainly, I mean, I guess it depends on what you mean, like variables that are all being captured by the model or not? AI certainly attempt to learn correlations between different variables. And to an extent that, if one variable is missing from a particular source, and AI can kind of impute what that variable might be just by learning correlations.

Things changing over time gets tricky, and requires you to retrain your model if it's not designed to handle that. So, many times you'll see that the patterns that the AI has learned become stale. Finn mentioned examples like in the COVID case where patterns that existed in society of February last year suddenly changed in March, OK. There was a huge demand for different products that AI's wouldn't have been used to training with, right and everything that, the key thing that AI is leveraging is its ability to find patterns of data. If those patterns change, then it can take…and if the AI doesn't adapt to that, you're going to get this kind of strange behaviour, alright.

DAN:

Cool, thanks. Another one’s just come through. So, ‘in theory, the more extensive and complete the data, the better. But there'll always be limits on what data can be known and discovered. Does that mean more extensive data makes the unconscious bias from what cannot be known collected even more dangerous?’

SIMON:

Can you just read that last sentence? The last part of it there?

DAN:

Yeah, does that mean more extensive data makes the unconscious bias from what cannot be known or collected even more dangerous?

SIMON:

Yeah, interesting. So, I guess, yeah, Finn can speak to that.

DESCRIPTION:

Finn on webcam.

FINN:

So, I can have a go at this one. So, the more data that you collect, the more likely it is that the algorithm will be able to infer potentially sensitive attributes from other features. And so, this idea that we can just remove sensitive attributes is really not a solution once we come to these complex systems that train on many, many features. However, because it's possible to kind of take the algorithms and explicitly encode in the algorithms that we would like to avoid certain types of bias or that we want certain types of outcomes… as long as you're doing that, actually it becomes possible to do better at removing bias, the more information you have. So, as long as… The more information you have, you know, perhaps you have a group of customers who are less well represented, you want to do something differently for them because they need a different response to other populations. If you can detect that, then you'll be able to serve those customers better as long as you tell the model that you would like to serve them better, rather than necessarily just to maximise profit.

So, really kind of cutting down on data, it's usually not the optimal solution. It might help in some circumstances, but usually it's better for the model to know as much as possible. But for the designers of the system to encode what they want done with that information. The flip side of that is that collecting data, particularly about personal attributes, brings in privacy concerns. And so, that's just another thing that you need to balance in the system, is what is the benefit of bringing in this extra data to both the company or the organisation and to individuals, and how is that weighed up against privacy concerns?

DAN:

Cool. We have another related question. So, ‘what does AI do to detect spurious false and misleading data input by bad actors?’

DESCRIPTION:

Simon returns on webcam.

SIMON:

Hmm… (UNKNOWN), is it? Yeah. Yeah, so, this comes actually to like robustness and there's a whole field on this on adversarial attacks on AI. And there's no easy solution, if anyone wants to jump in. But in many cases, like if someone has knowledge of how the AI system is working under the hood, then it's very hard to protect it against these kind of spurious attacks, right. You saw there, where AI was talking previously about someone maliciously changing a stop sign knowing exactly what a sticker is in order to think it was a go sign.

So, we were talking about, say, transparency there a while ago, and how that's important to kind of enable trust in some way in systems and all that. But then there's this trade off with the more you expose about an AI system, the easier it is for an adversary to come up with these kind of very sneaky attacks that a person might kind of wave through but that can really like scupper an AI system. There have been ways of trying to identify like these kind of unusual and unnatural sort of attacks. You can certainly do like anomaly detection and things like this on your inputs to see if there's anything that's looking unusual, but there's no hard or fast solution yet.

DAN:

Yeah, that's right and I think basically the short answer to that is nothing unless you build it into the system.

SIMON:

Yeah, yeah. Naively deployed there's no like in-built defence against these kind of attacks.

DAN:

I don't think we have any new questions come through yet.

SIMON:

Right. We're just kind of pushing up on the two hour mark. And I'm sure everyone's kind of keen to get themselves a cup of tea or coffee to get away, right. But, yeah, so once again, thanks very much for making this so interactive considering that it's an online session. Really appreciate the kind of the comments that we'll be getting. Hopefully, you found this useful and you've got a better idea as to how an AI works and where to use it and where not to use it. And I'll say again, if you want to reach out and talk us about any of these things, or want to learn more, then our contact details I think are up on the screen there. But you can find us on gradientinstitute.org and you might be able to click that if want to reach out to us.

OK, and thanks again. I'll sign off now. See you next time.

DAN:

Thank you.

(MUSIC PLAYS)

DESCRIPTION:

The ‘TECHTONIC 2.0’ page from earlier. Blue text now reads ‘This session has now concluded – Thank you for joining’.

(END OF RECORDING)

Hide publication menu: 
Show menu