-: Good afternoon, good morning or good evening.
I'm Andy Lippman at the MIT Media Laboratory
And I wanna thank you for joining this session
of Media Lab Perspectives.
Perspectives is a periodic series
we hold at the Media Lab where we invite
distinguished speakers to present, in this case, virtually,
although I will admit that it's a lot more fun
when it's actually there
and when most of you are in-person.
The series is noted for bringing to us and you
a series of speakers who are stimulating and provocative,
and I'm sure that today's will be both.
Let me begin by introducing Joaquin Quinonero Candela,
who is our guest and speaker for the day.
I've come to know Joaquin over the last several months
and so I guess I think we can call each other friends now,
so, you know, like, let's for the moment say that.
But just to get things going,
until September of this year, 2021,
Joaquin was the distinguished technical lead
for Responsible AI at Facebook,
where he led the strategy for areas like fairness,
inclusiveness, robustness, privacy, transparency,
and accountability.
Before that he built and taught
a new machine learning course
at the University of Cambridge,
worked at Microsoft Research
and conducted postdoctoral research
at three institutions in Germany including
the Max Planck Institute for Biological Cybernetics.
He got his PhD from the Technical University of Denmark
and the Max Planck Institute for Artificial Intelligence.
Post Facebook, he serves on the board of directors
of the Partnership on AI,
an organization interested
in the societal consequences of artificial intelligence,
and is a member of the Spanish Government's Advisory Board
on Artificial Intelligence.
And among other things include learning the guitar,
which you might well see in the background there.
Well, I gotta admit it's a very yin and yang guitar,
and Joaquin is considering what the next steps will be.
And your questions will bring that out.
I should remind you that,
I welcome your questions,
we're going to have a discussion
and hopefully that's going to be led and include all of you.
So please type your questions in Slido,
the link for which is at the bottom of the screen
on which you're watching that.
Joaquin has prepared a few opening remarks,
and then we'll begin our conversation,
then we'll end promptly at one o'clock.
So over to you.
-: Thank you, Andy, and welcome everybody.
Very excited to be here.
I think the format we're gonna use today
is that I'm going to talk for about 20 minutes
and then Andy and I will engage in a fireside chat,
hopefully reflecting some of your questions as well.
If you see me look down,
it's because I prepared some bullet points,
I'm gonna be using this to guide my introduction.
The first thing to share with all of you is that
about 15 years ago, I took my first fork in my career
when I decided to join industry.
I joined Microsoft Research in Cambridge, in the UK,
a vibrant group with people like Chris Bishop,
some of whom some of you might know.
And then what happened almost immediately is that
2007 was a time where computational advertising
was really taking off, right?
And this was a model where
machine learning was used to predict
whether someone would click on an ad,
and then that prediction was used to rank ads
and show them to people in a personalized way.
And the consequence of that change
is that advertisers were only charged
if someone clicked on their ad,
which decreased the cost
of acquiring customers dramatically,
and it was also one of the most large-scale and,
almost like immediate impact applications of ML.
So I started to work on that as a researcher at first
and long story short,
the algorithms we built ended up being the ones that shipped
with the Bing search engine when it launched in
summer of 2009.
And I was asked to leave Microsoft Research
and join the Bing organization as an engineering manager
to lead the click production team, which I did.
And I think that was a pivotal moment because I became
completely passionate about shipping impact at scale
through ML and actually putting ML to work in practice.
In some ways reflecting back,
I think I became a little bit of an impact junkie.
I know that's a bit of a sell to their arbitrary term,
but it's addictive when you ship things to production
and you see results,
it's exhilarating in many ways.
So that was 2009.
Then I joined Facebook in 2012
and we relocated with my family from the UK here.
And the two reasons why I joined Facebook were,
the first one is the Arab spring in 2011,
where platforms like Facebook and Twitter
were being used for society to mobilize
and overthrow authoritarian regimes.
Having lived and grown up in Morocco as a kid
when my parents moved there from Spain,
I was and remained very connected emotionally
to the Arab world.
So this was a really significant
positive impact of social media from my perspective,
the second thing is I visited a friend in March, 2012
here in the Bay area where I live
and I hung up with him at Facebook,
And I remember sitting there talking to people
and I was sitting next to this engineer and I asked her,
what are you doing?
And She says, well, I'm redeploying the site.
And I'm like, what do you mean you're redeploying the site?
Facebook was mostly a website at the time.
You are in who's army?
Who's helping you?
Where is the quality assurance team?
It's like, no, no, we have this way of doing it, BitTorrent
And we have this Canary system to check
for errors and so on.
And I'm like, wow, that is incredible.
And I thought, I just need to be here, right?
Like the, the purpose of it is amazing,
and then the culture where individuals are trusted
with something as important
as redeploying the entire site is incredible.
So I thought, I belong here.
So I joined Facebook initially to work on
machine learning in ads,
and there was the beginning of a pretty amazing story.
For context, when I joined Facebook in 2012,
there was mostly two big applications of ML.
One of them was feed ranking,
and the other one was ranking ads.
But when I joined the ads team, what I found is,
first of all, that the team was minuscule.
It was a couple of people doing ML only,
and both the technology used,
I used to refer to it as a so use,
you know, it's robust and reliable and it works,
but it's not fancy at all.
At the time neural networks were not being used to rank ads.
And also the processes to go from applied research
to shipping a model to production
were extremely fragmented, painful, and slow,
and it would take way too long
to go from research to production.
So I was faced with the dilemma and the dilemma was,
should I invest on building much fancier
and advanced models
with the existing infrastructure we have,
or should we actually clean the kitchen,
press the stop button,
realize that as I would often say,
wizards don't scale, right?
If you don't have the right infrastructure and tools,
you can hire as many ML experts as you want,
but it's gonna be a slow process
to go from research to production.
So I made the bet to clean the kitchen
and what I would call,
build the factory and the vision behind this
was that we're working to build a full ML stack
that would allow everyone in the ads organization,
whether they were predicting clicks
or whether they were predicting conversions,
whether they were looking at ads on mobile or on web,
another interesting thing,
and I'll talk about that later again is that,
when I joined in 2012,
most of the ads were shown on web only
on the right-hand column,
and when I joined, there were no ads on mobile, right?
But of course, usage on mobile was spiking.
So we had multiple surfaces,
multiple types of events to predict
and multiple groups of individuals
predicting these different things,
building models that didn't talk to each other,
and that were very difficult to reuse.
And I thought, we need none of that.
We need a universal,
incredibly flexible platform and set of tools
that allows everybody to collaborate by design,
where you can actually see any model
that anyone else has built trivially,
you can clone it, replicate it,
you can look at what input features are they using,
you can look at the learning curves, right?
You can actually, most importantly know who is doing what?
And then you have a record,
you have almost like exquisite experimental bookkeeping,
so you don't have to redo
the same experiment someone else did.
That was a vision, and that's what we'd built.
And he'd worked really well.
It really helped us speed up
research to production dramatically.
We went from many weeks,
maybe up to two months down to shipping
many models every single week.
And that paid off, that paid off in spades.
So what happened then is that other teams like
feed, ranking, search,
looked at what we had done and said,
well, can we adopt this platform?
And we started to work, we were very selective,
we thought, well, feed seems to be the most
impactful next application of course,
so we teamed up and we joined forces and
we ended up building the platform instead of tools
that are used, still today across the entire company
for every single application of ML at Facebook.
That success led to a change where I left the ads team,
and I was asked to build a centralized team at Facebook
called Applied ML or AML for short.
And the charter of this team was indeed to
almost democratize the use of ML,
and the strategy that I thought we should follow
was a three-pronged one.
The first one was, make it really easy for engineers,
even if you don't have ML background
to put ML in their products,
through automation, through building services where
you just provide a data set
And what you get is you get a trained predictor
and you get a hook point that you can hit to get predictions
on demand just to make it very easy.
The second aspect of the strategy was,
well, we need to innovate, right?
Let's remember we still have this so use, right?
The so use were great in boosted decision trees
as a way to almost generate
categorical features, if you will,
and then followed by real time online trained
logistic regression, which was, you know, pretty sparse
and massive capacity, right?
Very robust, it works,
but also very limited compared to the
deep and wide neural nets we have right now.
So the second aspect of the strategy was
we need innovate in model architectures
and learning paradigms.
And then the third aspect of the strategy was
a little bit that, that quote that is attributed, I guess,
to a combination of Steve Jobs and Henry Ford,
if I had people what they wanted,
if I had asked people what they wanted,
they would have said I want a faster horse,
we needed to also create demos and prototypes of things
teams across the company weren't asking for
because they didn't know it can be done.
So right now I'm using the Facebook portal
device for videoconferencing.
It's the first video conferencing device
I've shipped with a camera,
with a smart camera that tracks you,
and that does cinematography.
That was one of the demos that we built.
So the third pillar was demos.
So AML was widely successful in the company,
over 25% of all engineers in the company
were using and are still using these tools.
I left Facebook on September 7th.
Any statistics I have are old from when I was there.
And you can think about the scale of this,
if you wanna do a backup the envelope calculation,
imagine that any service like Facebook, YouTube,
Instagram, Twitter, Amazon for shopping,
or even when you use ML for content moderation,
you're looking at the order of 1 billion users a day,
it depends on who you are, right?
But just order of magnitude.
And then typically you have in your list of a thousand
pieces of content to rank.
And then if people come back to the service
10 to 100 times a day,
you're looking at 10 to 100 trillion ML predictions a day.
So this is a pretty unprecedentedly
massive scale application of ML.
The last thing that I'll say about the applied ML
era in my career at Facebook is that
it reminds me of the transformation Facebook went through
in 2011 and 12 where Facebook realized
that we needed to become,
I should say, we all don't work at Facebook,
but I feel emotionally still there,
that we needed to become a mobile first company, right?
In 2011, Facebook was a web company,
and so a centralized team was created
to drive that transformation that hired masses of engineers
who were experts at building things for mobile.
And in a way with applied ML, we did the same thing,
we created a centralized team
that first served the entire company,
but then very importantly,
one part of our mission was to try and make ourselves
as obsolete as possible,
And make sure that we scaled sublingually
compared to the growth of the company, right?
And to do that, we helped product teams across the company
build their own ML teams
to make technology transfer easier,
and also to get ourselves out of the critical path.
And so we ended up exporting over 150 experts
over the period that I was leading the AML team
to create things like the AI infrastructure team,
the AI camera team,
the assistant team embedded in product teams.
In 2017,
after an incredible period of building
and scaling the applied ML team,
I was pretty burnt out,
this was both exhilarating and amazing,
but also fully exhausting.
And I was thinking of leaving and taking some time off.
And then at the 2017 Europe's conference,
for those of you who were there or remember,
I think it was a people told (indistinct) conference where
there was a big presence of AI fairness talks.
There was a tutorial given by Solon Barocas and Moritz Hart
that was amazing on AI fairness.
And then Kate Crawford gave a keynote on bias in AI,
I think it was called a problem with bias.
And I think that got a lot of light bulbs
that had been in my mind for a long time to lit up.
Back at Microsoft
when I was working on computational advertising,
I remember being concerned about
what features we were using
as inputs for the ad rankings models.
And for example,
I had this open question,
is it okay to use gender as an input, right?
Even today, I think you can argue both sides,
but what was notable is back in 2009 or 10
when I was asking myself that question,
I didn't really find,
I didn't really find any guidance or anyone to talk to.
In 2017, the situation was completely different.
It was clear to me that we needed to urgently
invest massively at Facebook on the ethics of AI in general,
but on responsible AI and more specifically,
and even more specifically on fairness in AI.
So that's what I decided to do.
In early 2018,
I transferred leadership of applied ML
to a dear friend, colleague who was reporting to me,
and then I became an individual contributor for a while
and I started to learn a little bit more about AI fairness.
And it was a kind of an interesting story because at first
I saw the work that (indistinct) were doing
on computer vision and facial recognition and biases.
And the reason I'm smiling is that,
is that the portrait camera that I'm using right now
suffer the executive from the same problems,
and I have a colleague who is no longer at Facebook,
(indistinct) she's a black woman
and she was one of the engineers building this thing.
And she would test it sitting on a couch
together with colleagues
and the camera would often just crop her out
and focus on the white dude sitting next to her.
And it's one thing to read a paper about this problem,
but to be in the team
and see it actually exclude someone outrageously like that
makes a big impact.
The other thing is I have a colleague Isabel Coleman
who was working on fairness for job rankings.
We have a product called Jobs Near You.
And in there she was asking the question
is the accuracy of the ranking equally good for people
over on below 40 years old
and for binarized gender male and female.
And when I looked at this two examples, I thought, okay,
I've been building platforms and using math and code
to solve problems at scale,
I think we can do the same for fairness,
and we can probably get this sorted in a couple of months.
And at the Fairness, Accountability
and Transparency conference in New York in 2018,
I was at the bar before the conference started,
And I was talking to people
to a community that I didn't know,
And I just mentioned that vision,
I'm like, hey, we've just built tools
and platform for AI for an entire company,
we're gonna do the same for fairness,
and we're gonna solve this.
And then this person just turned around,
turned their back to me and stopped talking to me.
And I was perplexed and they said,
oh, that's just another techno optimist engineer
that doesn't know what they're talking about.
And I was really perplexed.
And that changed when I started to look
the ProPublica study in 2016 of the use of ML
to produce risk scores, assessments for risk of recidivism,
to helping the criminal justice system
for judges to take that risk score as an input to decide,
or to inform the decision
of whether to hold someone in jail while they wait for trial
or whether to let them go on bail.
And we can talk more about that,
I realize that I'm gonna be running it off time soon,
we can talk about that later,
but what blew my mind there is that they were
completely reasonable mathematical definitions of fairness
that were mutually incompatible.
And at first I thought, well,
that's why I dropped down menus were invented, right?
Like I just can't, I can say, well, individual fairness,
or maybe, you know, some group calibration
versus maybe the demographic parity,
we can equalize false positive rates or whatever it is,
and it's fine, just thick.
But then I was talking to Ethan Zuckerman
at some point from the MIT Media Lab about these things,
and he said, well,
there is a mapping between those mathematical concepts
and moral philosophy definitions of fairness
and reasonable people will disagree,
And he showed me this picture
that many of you might have seen
of three people of different heights
trying to watch a baseball game behind a fence.
And in one picture it's labeled equality,
everyone has got one crate,
and then the shorts person can't see, right?
With only one crate.
And then in the other picture is labeled equity,
and the short person has got the crate of the taller person,
so he has got two crates and the tall person
has got no crate and then everybody can see the game,
so you're equalizing outcomes.
And then it hit me like a ton of bricks,
I thought, okay, well, now I understand
why that ethicists turned their back to me
because I was just missing the point.
This is not primarily an AI problem.
And some of these questions are not new,
they're exacerbated by AI, but not created by,
and people have been thinking
about these things for centuries.
And so what I then decided to do
is to build this responsible AI team
in a way that was very multidisciplinary
and we did hire moral philosophers
and political scientists who are still on the team.
And we build a framework
that starts with high-level questions
on what definition of fairness makes sense
for the product you're looking at,
is it gonna be fair value exchange,
or is it gonna be a minimum
or standard quality of service,
or should we be looking at equal treatment
or should we actually be looking at equity.
And then the AI and the implementation, you know,
whether you have bias and data and the algorithms
or interventions will come, right?
But you don't start there.
It's been a very humbling and amazing journey.
So I'm gonna take a moment to look through my notes
to make sure that I don't miss anything.
We can talk about examples later,
we made sure that the use of ML
to remove election interference
in India in the 2019 elections
was fair under some definition of fairness,
and there was fascinating work.
And then there's a lot of interesting fairness questions
when it comes to removing hate speech
or bullying or doing content moderation in general.
So to try and close,
as I was thinking of leaving Facebook for the second time
and this time for real,
I embarked on the last project,
and that last project was accountability
and diversity and inclusion.
So the thing that really, again, like, lit up my mind
dates back from 2018 at the New York's conference, 2018,
where Yo-Yo Ma, the famous cello player attended,
and I was having a conversation with Yo-Yo
and I was asking him,
what would help you trust AI?
And he looked at me and he said, well,
the most important thing is to understand
who's building that AI
and what are their intentions and motivations.
And I guess that, that helped me build
a very strong connection with
another discussion that was going on and is still going on,
which is whether we have AI teams
that are diverse and inclusive enough?
The answer is no, we don't.
And I realized, oh, my God,
frameworks are great,
tools are great, processes are great,
but we also need to focus on the culture,
and I spent a year working on this,
and the end result is that we changed
the performance review system for all managers
and leaders in the AI team at Facebook
to make sure that we rewarded people
when they went above and beyond in diversity and inclusion,
but also, you know, that we held them accountable
if they didn't.
And some of the myths that I discovered in a way,
personally, I mean, I know other people knew them before,
one of them is diversity and inclusion
is often seen as a recruiting problem,
mostly I think that's wrong,
I think it's mostly a retention problem
and a problem of creating a culture of equality,
of opportunity and a culture where everyone
can participate and contribute.
Another one is you focus on education
and people will do the right thing, I think that's wrong.
People do what they get paid for.
And then the third one is as a triple whammy going on where,
I was in many meetings working on these diversity
and inclusion performance expectations
where out of 12 people,
I was the only white dude
and everyone else was from an underrepresented group.
And I think the triple whammy is
people who are doing this work, if it doesn't get rewarded,
they're working on something that doesn't get rewarded,
whammy number one,
the whammy number two is a opportunity cost,
you could be working on something else
that does get rewarded
and just started one which is actually pretty dramatic
in many ways is that,
if you're trying to push for change and rock the boat,
you can be perceived as being annoying and disruptive
and so you might get actually even penalized for that.
I think I really need to close now
'cause I think my 20 minutes are pretty much up.
So why did I leave Facebook and what's next for me?
I have no idea what's next for me but
my wife stopped working 15 years ago
when our second child was born
and she's been wanting to rebuild her career
and we've tried to hire help
and find ways to do it while I was working like crazy.
And it just became clear to me
that the only way I could help her reboot her career
was by actually taking her job over,
by being a stay-at-home dad,
and almost like push her out over the house tasks.
So this is what I've been doing,
and this is what I'm doing right now
and that's my top priority.
And it's way harder than I thought it would be,
both because I'm not good at it,
and second because emotionally,
it's very difficult to go from identifying with a job
where you're very busy and you're getting recognized for it
to having none of that.
What I will do, you know,
I would definitely go back to doing work
'cause I actually enjoy working,
especially on meaningful things,
but I don't know if it's gonna be closer to academia
or closer to big tech again.
So yeah, thank you for listening.
That was maybe a longer introduction than you had paid for,
but I'm super excited now to get into a discussion,
or I didn't mention that I'd been thinking about
social recommender systems.
You might have seen the MIT Tech Review
article that came out in March.
I tried to focus on what are the questions
that that article asks that we shouldn't be looking at
and less on whether how reading the articles felt.
But anyway, to end on a positive note,
I'm extremely excited to see so much interest
for building AI responsibly.
And I know that I'll be engaged
in this field one way or another.
So I don't know how we get now, oh, here we go.
Now we get Andy back on screen.
-: Well, you sure said a lot.
And covered it in a short number of times,
and while we're waiting for questions,
I'd like to remind everybody,
type your questions in Slido
and there's a link to Slido underneath the window,
by which most of you are watching this.
There's two questions that just come to mind immediately
that are inspired by what you say,
and one of them is,
the sort of like deadly embrace of
at a company like Facebook
spreading the gospel of AI so that it becomes accessible
to all of the local teams within it
and a potential conflict with the global goals
of trying to actually make it
useful, functional, fair, and ethical,
because, you know, those are decisions
that may not be easily made at a micro level,
but even before we think about that,
there's something that's woven through your entire message
that I would like to kind of like
ask a little bit more about because of your experience,
and that is the magic associated with the phrase AI,
because the assumption is
by everybody that it actually works
and that, you know, like,
and then what's wrong with it is, well, you know,
we feed it the wrong data, or we assess it the wrong way,
or we don't have the right people designing it, or,
you know, the model may be wrong
and if we just fix the model, or if we just fix this,
then it's actually going to work,
but that's not,
I'm not sure that's true.
You know, like one of the key elements in a system
that does the kinds of things you did
is the issue of how,
whether it's over-trained or under-trained,
so in other words, but no matter how you slice the cake,
it's inbred,
when it makes a decision,
it's making a decision based on a set of nodes,
mathematical relationships that are abstracted away.
So the first question is
are the limits really that the wrong people are doing it,
or the limits in part that we've invested
more mental energy into it,
and I apologize for the background noise,
but we had a big storm here yesterday,
the streets are filled with leaves and trees
and the teams are out there cleaning it up
and there's nothing I can do about the weather or the teams.
Anyway, over to you.
-: No, thank you Andy.
For what it's worth,
I can't hear the noise, so whatever,
whatever noise cancellation Zoom is doing is working.
It's a pretty profound question.
First, I wanna share you know, for context,
that the reason for that frenzy
of trying to make an ML or AI,
you and I have had discussions on
the fact that we should probably call it ML and not AI,
to make it available to everyone,
it's just that you could see the impact of it, right?
If we showed newsfeed ranked in reverse chronological order
with the most recent story at the top,
and we survey people,
we sit with them and we look at the experience,
people like it a lot less than if you actually rank it,
so there's a difference between we then without.
The second part is the one that maybe we need to,
you know, we need to appeal up that on and off
little by little.
It's very tempting and easy to focus too much on the ML part
to treat it like a black box
and to forget that it lives in a broader context, right?
And when you build a product that is powered by ML,
the ML layer is, you know, one part of it,
but I think there's a lot of other decisions that you make
that are about what metrics are you building?
What are your goals?
But also design decisions.
I'd like to give you two examples of design decisions.
For the longest time in Instagram,
you couldn't really share external links, right?
So if you had a story
that came from whatever news outlet you want, right?
And imagine that it was false or fake or whatever, right?
It couldn't really go viral
on Instagram for a simple reason,
there was no share button, right?
That's not an AI thing, that's a design thing.
On Facebook, there was a share button,
and you could actually share
content that you were not connected to directly, right?
So stuff could go viral, right?
You need to understand that it lives in that world.
Another example maybe in YouTube,
do you auto play the next suggested video
or you actually force people to look at
and then pause for a second and think about what they want.
I think that has also a very, very big impact
on the experience and it may make ML, you know,
more or less critical or impactful in what you're doing.
And then maybe the third thing that I'll say is that,
just based on my personal experience,
I think maybe the human brain
or at least my brain is somewhat sequential,
I guess the first problem that I was confronted with is like
okay, it's extremely inefficient for people
to use and deploy ML, right?
It's a pain.
How can we make it easier?
That seems to be the next front
because most of the places I look it's the, you know,
going from not using it to using it, it's beneficial.
And then once you're using it at scale,
and once you have these closed loop systems
where you have both the designers, the PMs, data scientists,
building metrics, iterating on the models,
shipping the new revision,
and you also see how it may even like change human behavior.
That's where you start to think, okay,
now we need to move on to the next question.
But I couldn't think about the responsible AI question
that much, and again, this is my own personal story, right?
Until I had solved the first problem
that was in front of me, which is like,
we don't even have the infrastructure or the tools.
I don't know if I answered your question, but I tried,
I think you're muted.
-: I'm muted to avoid the noise of what's going on outside
from penetrating while you're talking.
When we talked about this before,
there was another point that came up and the point was,
almost no matter what we build,
society is gonna adapt to it in kind of,
kind of funny ways, and, you know, so, you know,
back in the very, very early days,
where are the ads that appeared on the web
were very rudimentary,
one of my dear friends would type deliberate search requests
in order to pull up the ads that he wanted,
like if you wanted a new phone plan,
he would type in things
so that AT&T and Verizon would appear as the ads.
And he wasn't exactly gaming the system,
but he was adapting to the fact that it did that.
So as you do these things
with ads or feeds,
that changes people's behavior in
how they respond and how they generate feeds,
how do you fold that into the kind of picture?
-: Yeah, there's a lot to say about this for sure.
Let me start with a simple example
to illustrate what you just said.
If you're building an objective function for feed ranking
where clicks get a lot of weight, right?
'Cause your objective functions
for context are often gonna be
the weighted sum of,
it's a linear combination of expected events, right?
You're gonna say, well,
I'm gonna give a click, five points,
I'm gonna give a share, you know, 20 points,
a comment is gonna get 100 points or something like that,
and if I build a new friendship with someone else,
that's like 200 points or whatever, right?
If you put too much weight on clicks,
you will get clickbait,
you will get like sensationalist,
I can't believe he said that click here to know more,
you just get a ton of that, right?
People will adapt because people wanna
get more distribution on the platform,
you got to think about people's incentives.
What we also saw is that if you put too much weight on
comments, then you would get outrage,
you would get statements that cause people to react and say,
well, that's not true, or I disagree with you,
so I think that's totally, that's the thing.
That's absolutely right.
Another aspect of this that I think is quite interesting.
I've been reading Stuart Russell's book 'Human compatible'
which I recommend very highly,
and he makes an interesting hypothesis
that I find fascinating which is,
if you're, you know, take any objective function
of your liking for your social media ranking algorithm,
if you have a closed loop system composed of humans,
data scientists who are looking at the metrics,
designers, product managers, and then engineers
who are shipping new iterations of the model,
if you think about it,
if you're trying to maximize the metric,
and this is a hypothesis, of course,
but it's an interesting hypothesis to try and verify,
you may end up trying to make people
more predictable somehow, right?
Like if I can create a set of users where the AI can,
or the ML can predict more easily
whether you would click come and show or whatever,
you're gonna be able to move your metrics better,
so if there's any way in the decisions you're making
to actually get there,
maybe create people who are more polarized or maybe show,
maybe make sure that you gravitate towards a world
where again, the content that gets created
is more binary or polarized or the extremes, right?
Then you might be tempting to go there.
And so what Stuart Russell says is like,
we need to spend a lot of time
thinking about the objective functions themselves
and maybe one of the key things
that we need to be working on on ML is,
is to have, you know, systems that have,
that are more humbled about the objective function itself,
and that are willing to sort of question it,
but I think it begins with humans,
like we're humans need to question it
and we need to look at what we've built
and how it's behaving in the world and of course correct.
And then one more point I wanna make on this.
I'm not an expert at all,
but many of these questions are old.
There is a professor of architecture
at Columbia called Laura Kurgan,
who has been looking at the parallels between urban planning
and social recommendation algorithms.
And I'm looking at issues like homophily,
what decisions in urban planning increase homophily
whereby people tend to flock
with people who are similar to them
and actually increase polarization
and reject people who are different,
and, you know, could we look at algorithms
for ranking in social media in the same way?
And I think, Andy, your saying
over-training or over-optimizing,
the way I hear that is, you know,
what obligation do we have to keep exposing people to stuff,
the algorithm might think you're not interested in,
but we think it's just healthy
for you to be exposed to the diversity of things
versus exploiting what do we know.
I think these are all open questions.
-: I think part of what I was trying to get at
is a little bit of the inbred
sort of nature of,
you know, internally how the algorithms work and that,
that it seems to me one of the problems is
we just seem to put too much faith in them,
you know, I'm reminded of the,
you know, the previous phase of AI
when everyone was enthusiastic about what it would do
and that was called expert systems,
and, you know, they were programmed with 50,000 odd rules
and indeed could recommend an antibiotic
perhaps even better than,
more effective than what a physician might've known about,
but you didn't know what it didn't know,
and so you didn't know what rules it had,
and so it was a little bit opaque,
but then you could,
you could add some mechanism to trace that,
but now the level of sophistication
of some of these systems is such that,
that's fairly difficult to do,
you know, I mean, people have been trying to understand
the layers of vision in the human brain
for a long time and we really, you know,
that's extremely difficult to do
and extremely difficult to do here,
so the problem is we don't know what we don't know,
and we don't know, you know, what it's telling us so
I think part of what (indistinct) thesis was all about was,
hey, look, you know,
if you get your recognition algorithm up from 83% to 93%,
have you done any good at all?
That's one dimension of it.
And you've certainly touched on that.
And the other dimension is, you know, but we,
how are we going to know even that we've done that?
Go ahead and I'm gonna
turn to some of the questions
from some of the people who are watching us in a the moment.
So back to you.
-: Yeah, no, thank you.
Two thoughts on that.
Funnily, I did my PhD thesis
in probabilistic machine learning in (indistinct) processes,
and I became quite militant about the need
for any algorithm to produce
a measure of confidence about the prediction
and the dangers of not doing that.
And if I decide to go back to academia,
somehow I might very well pick back, you know,
the whole probabilistic ML,
you know, idea 'cause I think it's essential.
But the second one is,
there's always gonna be humans involved in
these closed loop systems with ML and you know, and people.
And that just takes me back to both diversity and inclusion,
but also participatory governance, right?
So to give you a couple of examples,
you can be happily working on using ML
to find and eliminate hate speech, right?
And you can think that you're doing very well because
maybe the accuracy of the predictions keeps increasing,
you know, where the ground truth is,
you have a bunch of humans
that have some community standards,
some community guidelines that you're paying,
and then they labeled content and they tell you, you know,
what the accuracy,
what the precision recall of the algorithm
is being et cetera, right?
And you can feel good about yourself.
But then if you don't have the right people in the room,
you might not be aware that may be hate speech
against black people in the U.S
has a different meaning or impact
than hate speech against another group or across gender.
And maybe you need to zoom out a little bit and think,
again about like, oh,
well maybe my objective function isn't quite right here,
or maybe my policies aren't quite right.
So I think
one necessary condition not sufficient
is really, really to embrace the discomfort
of having multidisciplinary teams
with people from different backgrounds
who will ask annoying questions
that at first you don't understand,
but then you sit down and you think about them.
I think that that discipline, that hygiene is essential.
And then zooming out from that,
I think there's a question of participatory governance.
Facebook has got these external oversight board
that can overrule Facebook's decisions
on content moderation.
I think that's good.
I think it doesn't scale,
but I think it's good, it's a good first step.
How do you build scalable models
for participatory governance?
If you're Airbnb, Twitter, or LinkedIn, Facebook, YouTube,
and your concern, right?
Again, to bring the discussion back to
the U.S and to civil rights,
you're concerned about, is there a bias or racism
that pops up in the algorithms
and how would I measure it?
Well, I would need to measure by race,
you know, how do you do that?
Who should hold the data?
How do I engage civil society and civil rights organizations
in that conversation
to make sure that I'm not making the decision
about pros and cons on my own, right?
I know that this sort of deviates quite a bit
from the use of ML, but you know, an ML engineer like me
might say, well, I can infer it,
I can't just build an algorithm that predicts,
you know, race or ethnicity,
well, I got to be careful,
you're gonna zoom out and understand, you know,
what does that mean and why would you wanna do it or not?
So did we get questions, Andy, from the audience?
-: Yeah, we do.
We have quite a few and let me see if I can
put a couple of them together
because the topics that you're covering
cover much more than AI,
you said you you went to Facebook in 2012
because that was after the Arab spring?
-: Yeah.
-: And that was a period of time when that kind of optimism
that the technology really could
be a force for good in the world was pretty strong.
I remember it well and we all cared.
And then came things like the 2016 election
and the current discussion about Facebook now,
which is, you know, the flip side of that.
And so we have questions, you know, about
how you fit machine learning into that,
and one of the things, you know, you can do
is to say, wait, back up,
we're not going to make it any better
because we're afraid of what happens when it does,
people have a tendency to believe machines
and people have a tendency to,
as you've said,
you know, game the system and, you know,
and introduce a lot of inauthentic interactions,
you know, in the course of using it.
And where's the accountability associated
with the designers of AI?
Okay, themselves, maybe okay to have a board of oversight
see who's looking at the content, but you know what?
What is the board of overseers that we envision for AI?
So there's a slew of questions that are,
it's hard to distill them into a single thing,
but that's the tenor of concerns that many have.
-: Yeah, I know it's interesting that the January, 6
Capitol riots of this year in many ways
have a strong parallel to the Arab spring,
pretty much a decade earlier, right?
It's the same mechanisms.
And one of the first questions is what role does ML play
in amplifying certain content versus other content?
Twitter, a couple of days ago released a really nice study
that looked at content and publishers
across the political ideology spectrum,
and then asked the question, you know,
is there more amplification of a content
that is right-leaning or left-leaning?
And honestly,
I don't really care so much about the findings,
especially because Twitter also is very transparent
about the fact that they have no idea
why the results are the way they are, you know,
the results for those of you who haven't read the study,
say that counter to intuition,
or what people expected that right-leaning content
gets more amplified than left leaning content.
To me the interesting thing is the transparency precise,
much less than the results because
I suspect that the results could have been different
depending on what's going on right now in politics,
and what stories are coming, you know, more from the left,
so I could totally see it flip.
I think the most important thing here really is that
we absolutely need to look at outcomes.
It's essential to look at how is the product working
in the wild,
and imagine a world, let me just paint a vision,
imagine a world where YouTube, Twitter, Facebook, TikTok,
any social media platform has got
ways to share almost in real time
what content is going viral.
One thing that's interesting is that,
you know, every platform has different privacy rules where,
you know, stories that I wanna share with my friends
can't really go viral 'cause
I'm not that popular, I have like a few hundred friends,
but I don't have millions of friends, right?
So it tends to be public content anyway, right?
That goes viral.
But imagine if you could actually take it
to the public sphere, right?
The discussion of like, hey,
here's in real time a dashboard,
here's the stuff that was going viral right now
in each of the platforms.
And different people will have different ideas on
depending on where your bar for freedom of speech is,
you may have your own personal and also cultural, you know,
induced opinion on where you should draw a line.
But at least if you had those discussions
in the public sphere,
I think that would be a very powerful way
to then understand, you know,
what changes we wanna make if any.
Now, on accountability,
my opinion is that the accountability has to be end to end,
like you need to have someone accountable
for how the product is working in the wild.
As someone who used to be an ML engineer, or, you know,
trying to make the algorithm work
and trying to maybe algorithm predict
whatever we had all agreed it should predict,
I should be held accountable for
expectations of responsible AI,
If there are privacy guidelines,
I should a hundred percent respect them
and if I violate them,
because I'm the person who actually plugs the data,
then I should be held accountable.
If there are guidelines for assessing
whether there's bias into predictions or accuracy,
I should be held accountable for that a hundred percent.
But I think there's a place where the accountability
has to be higher up
in deciding what is the goal of the product,
in deciding even what are the metrics
we're gonna be optimizing overall?
I think it's unreasonable
to put all of the accountability on the ML engineer,
I think the organization as a whole
should be accountable for these things.
-: That sounds good.
It also,
well, I mean, potentially, maybe a little bit late,
because in order for them to understand it,
it has to go out there and have its impact.
Because there are other dimensions that I think
when we were talking earlier,
you made the point that there were two areas
that you worked on, one was feeds and the other was ads,
I have to admit, I don't know exactly what an ad is today
because an add is anything that somebody can pay to put up.
So an ad can be of a sort of political influence,
or it can be selling toothpaste or cellphones and,
if I get an ad that tells me to buy a toothpaste
I don't particularly want,
that's not particularly interesting or harmful
or distressing to society.
On the other hand, if there's a profusion
of paid content or ads that's related to
political things, that's a completely,
that's a completely different matter.
So one of the questions is,
is there a difference between
in the AI itself between these trillion transaction systems
like advertising and low volume systems?
Some of the people you worked with
must've been working on lower volumes,
like hiring, for example or things like that.
I knew that was a little bit of pause.
-: It's a shame that, yeah,
it's so difficult to do these things over Zoom because
if we were in a room together
than engaging in a conversation
with a person that has a question to try and understand
the intent behind the question,
so I'll do my best to,
to answer it with the first thing that comes
to my mind.
But I think it's gonna be
probably a disappointingly technical answer
and I don't know if the question was more about policy
then I need to know more about what lies behind it.
So the commonality is that a lot of the platforms
and tools are very similar and are shared,
and that's a good thing.
So for example,
if you think about low prevalence integrity applications,
these are low volume,
some of them are low volume,
removing particularly horrible or egregious content
from the site of which you don't see a lot luckily,
and imagine that that content requires analyzing a video,
for example, right?
And just to call the elephant in the room,
I guess we have the Christ Church terrorist attack
that was a livestreamed, right?
That's a video that you would wanna
be able to detect very quickly,
you don't have a lot of training data of that kind,
you just don't, right?
But, because you have a unified platform,
what you do is that you train universal embeddings
for texts, for speech, for computer vision.
And you actually are able to do a lot of transfer learning
where you can benefit from massive scale models
for feed or for ads
to learn representations that are extremely powerful, right?
Where then you only need a small handful of labels
to be able to learn for a new domain.
And an example that is less dramatic than terrorism
would be, for example, machine translation,
where not only in Facebook but across the board,
there's a lot of amazing work on cross lingual embeddings
and cross lingual learning where
if you want to learn, you know, a classifier,
actually we can go back to dramatic examples,
you know, to low resource languages, right?
Like Burmese, where we've also seen a lot of concerns
of integrity and misinformation in Burma and Myanmar
where with very little labeled data,
you're able to build NLP models that work quite well.
So that was more the technical side of the answer
and I'm sorry that we can't engage in a conversation
and I can understand the depth of the question.
-: Well, I'll pursue that further.
As people use video more and more,
it seems to me,
naively one would say that's an insurmountable problem.
I mean, like, you know, imagine just something simple,
forget a mass assault, but just something simple like
it's probably not a good idea to propagate
videos of people swallowing Tide Pods,
I mean, you know, that that's probably something we,
you know, responsibly would not want propagated too far
without at least some sort of warning or rejoinder there.
But I don't think any of those videos actually say,
watch him eat this Tide Pod,
you know, there are probably a bunch of people
standing around saying, look at this, isn't that amazing?
And, you know, and I don't know how you would do it
in a world of images and, you know,
and in the world of sounds
that's associated with the videos.
I don't even know how you would begin.
And so that even if you did manage to,
you know, do better at a company or organization,
having a pervasive end to end holistic view
of how it wants to position what's coming out of it
and the AI is one component of it,
the impact is on the other and so on.
But the fact that you are a little bit of a forum
and you're in a complex multi-dimensional world like video,
is there any chance of even understanding that?
And what scale would you have to do that at?
-: Yeah, and that's a good question.
So there's been quite a bit of work and successful work
at a multimodal learning
where you train simultaneously on the video content
on the audio, both the audio and if there's any,
any language that is spoken on that as well,
and also surrounding texts,
'cause often there are comments or description
or something like that.
And some of the interesting applications, for example,
have been on finding drug sales on Facebook
where the language never says, you know,
hey, do you want some meth?
Or do you want some pills or some,
you know, illegal substance, right?
It's always pseudonyms
and these language actually keeps evolving all the time.
And then the images themselves,
they kind of like are very suggestive,
but they're not exactly the same,
you know, they're not exactly the thing.
I remember our CTO had this nice illustration where
I think there was some broccoli tempura taken, you know,
at a certain angle that looked very similar to marijuana,
but it wasn't and to a human reviewer,
you would be confused between the two,
but you can actually train computer vision these days
to actually be more accurate than people,
and also you can learn models that
are able to discern whether the soundtrack
or the audio or the text matches
or doesn't match the images.
You do need large data sets and you do need labels, right?
Annotations are very expensive.
One of the things that keeps fueling progress
in complex tasks like these ones
or maybe even like deep fake detection and stuff like that
'cause you haven't gone there yet,
but you may be going there too
'cause that's a pretty tough one, right?
Is public data sets, right?
Few organizations have the resources
to label things at scale and whenever I think,
whenever large tech companies release to the world
labeled high quality label data sets,
I think that helps everybody.
So you could imagine if we wanna
take a bit of an optimistic spin here,
I think if we saw more of that, if we so,
you know, most of the, I don't know,
trillion dollar plus valuation companies contribute more
high quality labeled data sets, I think that'd be
for the common good for sure.
-: I think shining the clear light of scrutiny
on the datasets and the algorithms
and the way that we do things is certainly
at least one part of a productive path forward
and that kind of transparency would probably do Facebook
as well as everyone else a lot of good now.
We're at the end of our hour.
I wanna thank you for spending the time with us
and bringing that well for experience
and insight to what's a truly important issue.
And I'd like to thank everyone out there for
joining us and asking questions
and stay tuned again for the next
Media Lab Perspective Series.
We'll announce the date soon and we'll see you again.
So long for today.
-: Thank you, Andy.