Featured Session: Reclaiming our Humanity in the Age of AI

Speaker 1 00:00 It. Let's give a huge shout out and thanks to our volunteers. And Speaker 1 00:07 then too, I just want to encourage you all to check out the XR exhibition at the Fairmont and open today I'll be running through. Okay, today's session gets into the heart of one of the biggest things we've seen across the event this season, addressing the need to reclaim our humanity in the age of AI is essential and fundamental to our go forward. Timnit gebru is the founder and executive director of the distributed AI research institute, formerly the CO lead of the ethical AI research team at Google, and the co founder of Black and AI Dr Gibb let's give it up. Speaker 1 00:53 John Palfrey is the president of the John D And Catherine T MacArthur Foundation with deep expertise in how technologies are changing learning and education. Speaker 1 01:10 And Karen Howe is the New York Times bestselling author of empire of AI and an award winning journalist covering the intersection of AI in society. I Speaker 1 01:25 as a quick reminder during the Q and A open your South by Southwest Go app. You can ask and upload questions by selecting the Engage button in this session, please join me in welcoming the speakers to the stage. Applause. John Palfrey 01:49 Well, thank you, everybody. This is amazing to see such a great crowd. On a Sunday afternoon. I wasn't sure anybody would show up, but at South by, people show up. This is fantastic. My name is John Palfrey, and I'm president of the John D And Catherine T MacArthur Foundation based in Chicago, and we, for many years have been funding at exactly this intersection, trying to figure out how we center humans in the age of AI. And I am excited to be here with two leaders that I deeply admire, two intellectual leaders and practical leaders in the field. And you've heard their official titles, but Dr Tim, the founder of Dare and Karen Howe, known to many of you as a best selling author. And whenever I'm asked for one book on this topic, I say, it's true. It's true. It's amazing. And to me, it has a book coming out not very long from now too. So I have to say too well, I'd like to start, if I might, just by opening the discussion to ask you about the gaps in the conversation here at South by I know one of the big topics is, how is AI changing our society? And each of you is bringing a lens to this conversation, and I think, a critical voice to how we think about AI and its development at this moment. But maybe, could you help us think about what's the most important thing that we are missing in the conversation globally. If I would go to Karen, I would say, Speaker 2 03:17 the hype there is, there's a conflation of things when we say what AI is. It's very difficult to ground the conversation in real things. So for example, chat GPT is in the same category as an Automatic Speech Transcription system that is trained using curated data and that I like. So the things that I'm fighting against and the things I'm building are in the same category. So that makes it very difficult to have bad conversations in what's bad versus what we want to pursue, right? Because when you argue against all the bad things that are being done in AI, people will say, but what about, you know, breast cancer screening? Well, that's a very, completely different thing than some of the, some of the technologies that, for example, Karen, Speaker 3 04:13 Karen, what's the lens you bring into the conversation? Yeah, I would. I would absolutely add on to what Tony has already said, and say, also, we don't have very robust discourse right now about what is really the purpose of building AI and like, what are the goals that we are trying to accomplish? I think the companies in Silicon Valley has set the agenda as so called architect of general intelligence, and a lot of people have just taken that at face value and adopted wholesale as, yes, ultimately we are supposed to just build machines that are going to advance further and further and further and be duplicative of humans. And I am like, why? Because the entire purpose of technology development is to support people. And there's so many other categories of AI technologies that do, in fact, do that. So shouldn't we be instead asking first, what is, what are the goals we're trying to achieve, what are the challenges we're trying to solve? And then first, engineer which technologies would support us, including AI, but also not AI, and sometimes including non technical solutions that would support us. And then figure out from there which we should build, rather than just simply advancing AI for AI safe and then making it start having these huge ripple effects, like automating away a lot of jobs, because that is how they're being designed. But Karen, you may But Karen, don't Speaker 2 05:37 you know that we are building the machine box that will solve you? Know, I don't know if you guys, I'm not making up what they say. There was this one conference where, you know, Ilya suskiver, Chief Scientist, former chief scientist of opening Amazon, and someone asked him about, how do they bake in ethics into their models. And he said, In the future, the model itself will tell us what to do. That is ethical, like which model to choose. Because of ethics, the model will help us solve climate change. You're going to build the thing that's going to fix all of our problems. Why? I don't understand why that's how they're justifying all of their resource consumption and all of the labor exploitation in all of the data, because if we're going to pick a machine plot that's going to fix everything for us, clearly we should use all of those resources on Earth, and if we are run out of resources on Earth, we should go to space and mine Mars or something. You know, that's, that's what John Palfrey 06:50 we're buying. I'm going to come back to you, if I might, and push on reasoning if I'm Talking you. Speaker 2 07:00 Means that one is John Palfrey 07:14 determined Speaker 2 07:22 to Reis community. John Palfrey 07:29 What's the effect of that when we have the either MCA or doomsday as the two scenarios? Speaker 3 07:34 Yeah, I mean, a huge part of the reason is because Silicon Valley is the one that is seeding these narratives and perpetuating both of them with all of their marketing materials every time their executives come on stage and give talks. And the reason why they perpetuate these narratives, they're two sides of the same coin in that they when they're myth making about the Machine God that they're trying to build, and they're trying to justify all the consecutive resources and all the things that Timmy mentioned, they will paint this vision that ultimately we are doing this to bring us to utopia. But if we weren't the one doing this, someone else would be doing it, and it would bring us to hell. And it's this, this, like they're trying to lean into both sides of this, this narrative to then conclude, because in both versions, you then conclude, well, then it should only be a small group of people controlling the development of this technology. And they then say, like the conclusion is, in fact, we are the only small group of people that should be controlling this technology. And so that's why we end up with these Boomer and tumor narratives that really polarize the conversation in these ways, with a lot of lack of nuance that to meet originally said, which is, like they never talk about the fact that there's an array of different AI technologies. They're just fixating on this, this overall concept of the Machine God, or the everything machine that is going to bring abundance or bring us to a hellish state. And I think because humans, we gravitate towards these types of simple narratives, then it becomes really evocative, and it becomes dinner room conversation, and it becomes cocktail party conversation, and trickles into all other parts of how people will start before you may what AI ultimately is. John Palfrey 09:24 I want to hear what your preferred narrative is in a second. But to me, it's been referenced a few times by Karen that the narrative is typically coming out of a relatively small number of Silicon Valley based companies, which, of course, is true, one of which is Google and one of which is your former employer. And so you have a sense of how these technologies, in fact, have been developed and are being developed also by you and your post Google experience through dare. What if you could just give the perspective of a technologist who's been involved in building these technologies, and again, why is this? Is it Boomer or Doomer narrative that Karen has introduced that? Why are we advancing that? And how does your technology background inform Speaker 2 10:01 that, well, I was very confused for a long time, and it's like, why are we where is the entire field of AI moving in the singular direction of attempting to create a machine God in the field? These people used to be ridiculed. Actually, they were. It wasn't always the mainstream. Back in 2012 2013 when Elon Musk called as we're summoning the devil with AI in your book, you talk about how I went up to him and I asked him, Why you this was in 2015 I believe. Why are you so? Why do you think AI is an existential risk? Isn't climate change an existential risk? He told me that with climate change, every single person is not going to die, but with this so called fictional, super intelligent machine can wipe us all out, you know? And so that's why he thinks climate change going to be super bad. But so I was very confused for a long time, and I actually understood that it was an ideological thing. It was almost like a secular religion. That's, you know, there are true believers, and then they exploit that true believer kind of, Oh, my God, we gotta either build the machine God, but we might accidentally build the machine devil, but we still need to build the machine God. So it's the same people saying this thing in 2015 there was, you know, I don't know if you guys know about the pause AI letter that just came out a few years ago that got a lot of play. Those same people were saying a few years ago that if we don't build this kind of technology, we're doomed. That's literally what they said. And if we do build it, because Earth is just our room, our room, we're supposed to colonize the cosmos. And they call it paradise engineering. I mean literally, this is eschatology, you know, typical religious kind of talk. So when I was at Google that I think your book does a great job of showing how OpenAI really grow this bigger model? It's better. We have to have a drive towards AGI. So when I was at Google, they saw that OpenAI was getting a lot of press for building a big model. And then the Google people were like, Why are we not the ones who are building the biggest model? And then I was like, why do we care about building the biggest anything? What are we trying to solve? It seemed like a pissing contest to excuse, you know, my John Palfrey 12:26 language. You're okay, Speaker 2 12:31 you know. So so this, this is an ideology that's been pushed very conservatively by a certain group of billionaires. I mean, I know the effective altruists are in there. We call them the test squirrel bundle in my in my paper, with my collaborator. So it's a combination of kind of the secular religion that's very convenient, because you're now centralizing power and getting all the money by convincing people that you're building both building a machine gun and preventing the machine devil from being built. John Palfrey 13:08 So to me, that's just to put you on the spot here as someone with a PhD in intellectual engineering and having done all this work at Google and all this work at dare it does not keep you up at night that there's an existential risk that these robots are going to Speaker 2 13:21 take over. Let me just, let me just John Palfrey 13:22 tell you something I'd like to hear. I'd like to sleep better. Speaker 2 13:25 I just want to tell you, you know, when I was doing a PhD, I had to work with 1000 data workers. So my PhD was to create an algorithm that detects and classifies cards in Google Street View images. We use 50 million images, and then to figure out, like to see if we can figure out certain characteristics of cities from those images, wow, like all these we can create a computer vision algorithm to do this. Well, there was a lot of grunt work involved. I almost I was looking at cars and classifying them to the point that I thought cars were people like I was, you know, like the bumpers looked like a smiling face or a sad face. Like this is how much I was looking at cars. And then I had to hire data labelers. Because when you're training, some of you who's who are artists or whose data has been stolen. Know that all of these things run on people's data that are painstakingly labeled, right? If you want to classify oranges in an image, you have to have a whole bunch of images where someone has labeled them as oranges the data, right? That that is how you train these models. So I had to my biggest task was to assemble a huge data set that was labeled and we had 1000 data workers that we were working with. But if you look at the headlines of my PhD work, first of all, I wasn't complaining, because once you've been working through a PhD, you're going to be happy of any news coverage. But what they were saying was, this algorithm knows how you vote based on the car you drive. That was my work. I was like, Well, I don't, not exactly, but if that's what you saw, you might be worried that they're the Machine God is coming. So except for Karen, I'll say a lot of journalists are not helping us because they're kind of parroting the propaganda of the companies. Hollywood is not helping us because all of the visuals that you see is kind of like Terminator scenarios, right? So I'm not, no, I'm not worried. I know it's based on a lot of exploited workers who we don't see, and stolen data. And so if you know what's under the hood, you wouldn't think that anything is super intelligent. John Palfrey 15:50 Karen, I teed up before this point about the potential two narratives that are out there, but you and your book, I think, are pursuing another narrative. You were seeking the media who are assembled here and the world watching you to tell a slightly different story that's maybe not as scary as some and maybe not as promising as other. What is that compelling and accurate narrative from your perspective? To me, the Speaker 3 16:11 biggest story that's happening is that these companies have consolidated a profound, extraordinary amount of political and economic power, and they are distorting democracy as we speak. And the reason why I call my book empire of AI is because I think Empire is the only metaphor to understand the scale and scope and style of how these companies operate. So I draw upon different parallels in the book between the empires of AI and the empires of old, and for example, they lay claim to Reis that are not their own, which to me has mentioned. They take the data of artists and writers. They take the data of private individuals that are posting on social media. They're literally land grabbing in order to build the computational infrastructure for supporting the training and development of their own models. They exploit an extraordinary amount of labor, just as empires of old did as well. They monopolize knowledge production, where they tell the public this narrative that if you don't like our technology, it's just because you don't know how it works. We're the only people that knows how these models work, and we're the only people that can tell you what is going to come to pass. And they do this to the public. They do this to policymakers. They also bankroll most of the AI researchers so that they can censor the research that is critical of their work. In the case of your experience, that's exactly what Google tried to do to you. And so they become like the only source of quote, unquote truth about these technologies when they're actually trying to distort the truth. And then they have this religious like narrative where they frame everything as this existential race between a good empire and an evil empire. And they, of course, are the good Empire that's bringing us to heaven, and the evil empire would bring us to hell and and then they they're building a technology that they sell to us as a one size fits all solution. It doesn't matter who you are in the world. Doesn't matter what language you speak, what you look like, how your life is like, your history, your context, your culture. We have one model designed based on what we think are our worldviews, our values, our language, and we're going to project all of those things to you through this model by convincing everyone to adopt it, and so that like, like, the parallels to Imperial ideology are endless, and we haven't even talked about now the fact that it's used literally in warfare. So the Empire is not just an analogy now, so it's becoming literal. But one of the things that I do in my work is i, i co created a program with the Pulitzer Center called the AI Spotlight series that trains journalists on how to cover AI. And this, Speaker 2 19:00 it's a great John Palfrey 19:02 just to pause on that. This is at the Pulitzer Center and can be found online, right? Speaker 3 19:06 Yes, and it's completely open source. We've recorded videos for free that are online. You can search this resource available even if you're not a journalist. You can search it and watch the videos and hopefully find some some usefulness out of it. But this was one of the things that I realized was a lot of journalists who are being called upon to now call to cover this huge sweep of change that's happening. They don't. The only narratives that they are exposed to relentlessly are the narratives of the companies, and then it ultimately gets reflected in their work, and then it misinforms the public. And so as part of this work, what, what we do is we provide all these free resources for journalists to wrap their heads around this technology, the history of this technology, the way that the companies co opted a scientific field to then serve their agenda, the way that they're consolidating this power, and how, ultimately, journalists need to play a role in holding these companies accountable the same way that they have always held great power to account. John Palfrey 20:09 I want to drill down on one of the additional concerns that some of the people in the audience may well have been reading about in the last few weeks. So there probably are parents and grandparents and others who care about young people, and of course, it's been in the news that companies, including meta, have been on the hot seat for the effect of social media plus AI, which I don't think is going in the right direction in this way, in terms of their effects on kids and their learning and their well being. And one things you mentioned, both of you have the effect of this concentration of power in a relatively small number of companies, and the effect then on research and so forth. Wonder how you might advise parents or others who are watching the story how to interpret what's going on relative to social media and AI relative to kids, and what you might have learned from your research that might help people to make sense of what's going on I was gonna Speaker 2 21:02 have you very hard, but so, you know, I was just deposed in one of the cases against character AI and Google and Google, I'm not sure how many people have heard of this case. There was a teenager who committed suicide after interacting with this chap. There's many cases now, but this is, I think, one of the first cases, and Google was trying to distance itself from it, because character AI is a company that was started by a researcher at Google who used to just sit right across from me, and right after I got fired, they just left, Google started this company, and within a year, almost, basically, kind of got bought back and came back a billionaire. So this one guy left started this company that how many of you have heard of character? Ai, actually, I'm just curious. So, you know, it's basically impersonating different characters. That's what it allows you to do. You get to interact with different characters in this Chatbot. And so they left, started this company, went back to Google. Now he's actually Jeff Dean's only employee direct report. I believe Jeff Dean is the guy who's firing me, who is chief scientist at Google. So this guy who started character AI, is his sole direct report in charge of doing a lot of things, like working on Gemini and things like that. So Google's claim is that this was not their product. They had nothing to do with it, but they basically financed it right. And now the guy is back there, billionaire, doing the same things. And so the argument is that they knew what they were doing. I mean, this is part of what the issues were, part of what me and other people were warning against, and they tried to suppress our work, and then ultimately fired me and my colleague, and the mom, the parents could not have known. They didn't even know that their kid was interacting with this with this Chatbot. So you know, my my advice to parents is really be very careful with your kids time with these chatbots, because we're just starting to understand the effects of social media, and I feel like this is actually worse. It's highly addictive, and people don't even know that their kids are on these chat bots, interacting with these chat bots. And it's unfortunate that we have to talk about advice to parents, because parents should not be in this situation. It's kind of like saying, you know, hey, crack is everywhere in school, but you don't even know that there's crack and there might be this thing that your kid is using. So parents be careful. You know? I mean, parents should not be in this kind of situation. It shouldn't just be legal to create technology like this and put it out there. Speaker 3 24:07 Yeah, I actually three. Basically met Megan Garcia, who's the mom of the 14 year old boy Sewell Setzer, the third who died by suicide because she was explaining to me that character Reis chaplain was impersonating Daenerys Fernan from Game of Thrones, and was sexually grooming her son, and basically convinced her son, 14 year old child, that he was genuinely in love, and that this was his love, the first love of his life, and that he would be able to join this persona in heaven if he killed himself. And, yeah, the thing, I mean, I guess to broaden it out from just chat bots and just talking about, like, AI, more generally, I think a lot of parents are just confronting this challenge of, like, do I make my kid use it, because then they think that the kid would fall behind if they don't or like, do I do I tell them not to use it, or whatever it I would say, just like, have a conversation with your kids about establishing, like, a personal AI policy, and also maybe, like family AI policy, because you want, ultimately, like, one of the things to understand about AI in this moment is that it's a political project, and the central feature of this political project is taking people's agency away. And so whatever you can do as a parent to basically imbue your child with maximum agency, not just in the AI conversation, but just in general, is already helping reframe and equip them for this, this era that we're in, but when it comes to AI, particularly, I think the best way to give them that sense of agency is by having these discussions where you help them navigate how they're going to integrate or not integrate these tools into their lives, and also help them understand that there's choices of different types of AI tools, and help them research like, learn to research what types of tools are out there, and then how to select them based on, like, their values and the family's values. And through this process, they will then be able to learn how to apply this to their school when their teachers might force them to use AI to their workplace one day, when their boss might force them to use AI like you're equipping them to push back and say, Wait a minute. Let's have a conversation about this, and collectively make a decision about the right way to govern the use of these technologies in this context, John Palfrey 26:42 goes really well. But I think one of the things I've been obsessed with in funding and MacArthur Foundation with this idea of humanity at the center of AI is how do we put agency back in human hands in the age of AI, as opposed to having all the agents outside? And I think that communication, and there are lots of great researchers and resources for parents on the side that have young people realizing they do have agency in this period, as opposed to being only acted upon. But that's, I think, a little against the grain, as you're describing. If I might go one more, that could go in a dark direction as well. But I think is relevant to this audience, particularly is around creativity and humans. Because, of course, this is a celebration of all the forms of art and creativity at the moment. And I think for some people, there's a word that the creative arts are going to be taken over by AI. They're obviously, again, real Reis to worry about this. But likewise, given what you've seen in research, how would you have creative people think about this moment in AI? What kind of advice would you give them to maybe I'll just go Karen first, and then Speaker 3 27:39 to me, one of the things that I was just in Baltimore at a writers conference where I was also talking on a panel about AI and creativity, whether AI can create art. And I was on a panel with Ken Lou, the science fiction writer, and bahini Bara, a Bloomberg editor, and we all included no and all for slightly different but complementary reasons, and what for me, like art, is that one of the crucial purposes of art and creativity is to push moral and social progress like we this is, this is like a very like, a gift that art and the creative community gives us, like in movies, in in in music, in all in writing, in all of its forms, it's always like pushing and critiquing the current version of society and asking, how can we do better, how can we do better? How can we do better? And the thing about AI is that it only regurgitates and remixes the things that came before and sometimes those remixes look novel, but in your paper and the stochastic Harris paper, you specifically point at this point that there there is no ability for these models to actually embody new values from what came before. And it's not all it's not able to rework language in ways that actually reflect the like, the direction of travel that we want to go, like we want to continue advancing and moving forward, rather than simply using the same exact language that we've used to entrench power, entrench inequities in the past, and so like in this moment, I think when Artists are thinking about, like, what do we do in the context of AI? Should I be using AI? Should I not be using AI? Like, how do I protect my work? First and foremost, I think, like, be clear about why you ultimately do art and keep that as your North Star. Like, if you do art in part to have to challenge society to move these conversations forward, like that should be your North Star, and then you should think about how engaging with AI is helping or hurting that North Star. And so if you you know, you decide that you don't actually want to partake in using AI, and you also don't want to support the development of AI. You know, there have been some extraordinary movements of artists and writers that are now suing these companies to challenge the fact that their data was taken away. There are movements where artists are using the tool night shade, developed by a researcher at the University of Chicago, where they shield their artwork so that from the human eye, you can't actually see when they're posting their photos on Instagram or, like at an online portfolio. You can't actually see the difference. But when a machine then trends on it without the consent of the artist, it begins to break the image generator down. And so that's what I would say, is like, first figure out, like, what is your purpose for, why you do artistic and creative work, and then figure out how to orient all the ways that the AI complex is intersecting with your work, like, how to then challenge and the things that aren't working in service of your project and support the things that John Palfrey 31:02 are I love this answer in part because it reminds us that people enjoy making the art. There's a reason we get fulfillment from it and engaging with someone else's art, who also might sit across from the festival, or you might see them. There's something in that that's really, really profound, and I realize there are forces that are intermediating here, but I also think the core act of creating and enjoying and connecting through art is going to persist. But to meet How do you think of this one? Speaker 2 31:25 Basically, very similar. I was just going to give a couple of specifics, like Carla Ortiz, who you also write about in your book. For me, she's like my go to in learning about these issues, and I basically support anything she does. She's at the forefront of a lot of these lawsuits. She's working super hard on the side of artists. I also think that I find that there are different levels of conversations that happen depending on who's on the chopping block in terms of work. So some artists are more established, they have more freedom. They're not as worried about income. And so those artists are, and it's very few of them, not that many, and they might want to experiment with AI or, you know, things like this. They're not as worried about it, whereas, like things, like illustrators, for instance, working at various companies, they've been they've been seeing firsthand how their work has been they're just being laid off. Or they're seeing how students at new art schools are very depressed because they're trying to understand whether this is even worth it. So we have to also understand the economic dimension of it. We all benefit from art. And if you know the phrase starving artists is already a phrase, and so why are we engaging in something that is further disenfranchising artists. I love art. I mean, I I play piano for a long time, I dance and I, I love art. It's part of it's kind of gives you meaning in life. It's why you want to live. It's how you make sense of the world. It's how you communicate. And you know, even if you tell me that some robot can play like some complicated piece over there, I'm not interested in watching that robot. I mean, like a computer can play chess, right? Like I'm not a car can drive faster than a marathon runner, right? So is that what we're interested in? No, we're interested in the marathon runners journey and things like that. So so it's a political project, like you said, and so if we care about art, we should fight for art, and we should support the artists who are using every single thing at their disposal to fight. So sometimes, some people have issues with copyright laws, but if, if that's the tool at their disposal that artists are using, then let them use that tool until we have better tools to help them fight back, absolutely. John Palfrey 34:09 And those of us who work in the law related to these topics, I think a lot of people are rethinking the copyright statutes and how they work and and our points of view. And I think that that work is going to be necessary just noting, in about 10 minutes, we're gonna go audience questions, and I have a few more I'd like to ask. But I do wonder, would either of you like to ask a question to one another? They come in hoping, like only Karen, how would if only to me again, I want to know. Speaker 3 34:35 Okay, so, so to me, John Palfrey 34:38 to me, how's this? She does no idea Speaker 3 34:41 to me, has this incredible organization now that she founded and runs called distributed AI research institute, after your experience of being ejected from Google, and I want you to tell the story of what made you decide to found this organization and how it operates fundamentally differently from Silicon Valley, because I think sometimes in the public imagination, as we've sort of talked about a couple times already on this panel, like when we critique what's happening with the way Silicon Valley is developing AI, people then bucket us into, oh, they don't like technology or AI at all. And that's absolutely not our orientation. And you then created an organization in which you are putting out a new orientation for how we should think about developing technology in the human in a way that supports human forging. Speaker 2 35:35 So I'm gonna say, I'm gonna answer this question by first telling you that I don't you know you had that AI colonialism series that you did was that in 2019, or 2020, is around when I got fired from Google. And so it's kind of like your book, where you talk about different regions and the Empire project, and then you highlighted tehiko media. I don't know if anybody has heard of tehiko media, just a couple of people, okay, and then I also heard about them on Wired. So this is one company, and this is why, for me, I think change doesn't happen with this whole humongous scale, one model for everything it happens where with small organizations that you are inspired from and that you kind of adapt whatever they're doing to whatever you're doing. So this small organization to ecomedia. They started out as a radio station, but they do a whole bunch of stuff now. They are focused on Mari language, the language of the Maori people in New Zealand, who are colonized. And so they wanted to create a tool for language revitalization, because they're well, their language was beaten out of them by colonizers, so now it's dying, and so they wanted to revitalize it. And so they thought about creating language type tools for that, but they needed data, training data, and so the way they had tanning data is they had a competition, I think, and a lot of people were super excited. They gave them data for the competition. They were able to create super cool speech recognition tools and other tools. But then this American company wanted to license this data from them, and they said no. And they put out a video of why, and they said, you know, you want to, you beat the language out of our grandparents, and now you want to sell the language back to us as a service. We're not going to do that. Whatever we do should serve the Mari people first and foremost. So I was very inspired by whatever they were doing, their whole thing, their whole approach, the tech they were building, the reason they were building it, how they were going about data protection, etc. And so my point was that I didn't get a PhD in electrical engineering to talk about Sam Altman every day, like, that's just not I didn't go like, I know what's gonna be fun. Let me you know that was just not in my purview. But I found myself constantly being like, Don't do this. That's terrible. Oh, my God, let me tell you know. And then I say, I can't do this my whole life. I want to pursue my vision for what I think a different text should be, just like Tinker media. And I want to resource other people's vision, because otherwise we're always behind. We're always letting them build something and then standing by the sidelines and telling them not to do it, and then whatever they ignore as they're building the next thing. And that's not how I want to live my life. So that's kind of how, how the story of the year started. I mean, we do the critiquing, but we don't only do that, we also put forward our vision for building tech, but also it was very huge thanks to MacArthur, because I don't know if I told you this, but while I was in the thick of finding Google, Eric Sears from MacArthur's, was like, MacArthur was like, Oh, what are you gonna do next? I'm like, Dude, I'm fighting this, you know, company that has more money than, like, most governments. I don't have time to think about what I'm doing next, you know. But after a while, I'm like, oh, I want to build this thing, you know, and and the fact that I have people supporting me and resources to start doing that, and like, resourcing my vision is why we're I was able to start there, and I want to do my small part to resource other people's vision. Can I ask her a question now? John Palfrey 39:40 But I would just like to say to support both dear and Pulitzer Center, but Yes, over here, Speaker 2 39:47 so, you know, I think a big portion of your book was about agency. And I'm like, I told you, like, I'm very impressed by how that theme of agency was, you know, kind of just very present throughout every single chapter of the book, and even chapters where I thought it was hard to tie it in, you had that agency, but I know you covered a couple of examples of tech that you like, approaches that you like. I'm sorry I just talked about teeth media, and you're not allowed to talk about dare, but I'm I'm just wondering, like from all of your work and all of the people that you've met, who who are the aspiring what are some inspiring stories that didn't make it into the book, but you really wanted to put into your book. Speaker 3 40:42 Oh my gosh. Okay, this is an unfair question, because the two main examples that I use are tahitu and dare, and also, specifically one of the dare projects that so this is okay. So one of the things that we have been actually working on is this project called the AI resistless where we're actually trying to document more stories of people that are doing really amazing things, either resisting the current Imperial vision of of AI development, or proposing new visions. And so I kind of want to come back to this question once we finish that work, because we've already found some, like, really incredible projects and where we want before we like actively platform them. We actually want to reach out to them and make sure that they would be feel comfortable being part of being platforms. So it's, yeah, I'm not, I'm not, John Palfrey 41:46 stay tuned. Answer that. I don't think she does that to the rest of us. I don't think you Speaker 3 41:58 said about dares approach that I think is so crucial to it that you said to me was Silicon Valley gets to influence the world, and the world doesn't get to influence Silicon Valley. And so one of the reason why you chose the word distributed is because of this, this thesis that the team needs to be embedded within Speaker 2 42:22 the communities that you're actually supporting. Yeah, the first word that came to my mind was distributed, because it's the opposite of whatever the empire building project and decentralized has been, has been co opted. Use decentralized. Distributed has not been co opted yet, but that's the idea, you know, I don't want to build one model for everything. I want to build many models for many different kinds of people in the world, because there's no one way of being human. There's no one right way of being. There's no one language everybody has to speak, or, you know, something like that. So I love, I'm going to go back to that tahitu idea. But one thing I love about tahiku is that they they weren't like, let's focus on the biggest language and get the biggest, you know, profit and then trickle down economics. They're like, let's focus on this particular language that we care about and do the best we can, you know. And if we resource many such groups around the world doing that, that's how we would come up with great tech, not resource open AI. John Palfrey 43:29 I was in India a few weeks ago for the global AI Summit, and it was really fascinating the way they did it, which was there were a million people from India, literally came into the there's so many people interested in this feature and building it from different angles and doing things, then I think this distributed vision can be part of what's positive. I can use my moderator probably, to ask one last question before you go to questions, it's going back to you. Karen, now, so let's go back to this sort of backyard barbecue. Maybe you're having great Austin tacos or something, and somebody's saying, Okay, I hear all the real concerns, which are legit, and I honor those. But what would it take for us to reclaim a human vision of AI that's really positive, like, what is it that it would take to have a future and a vision here that's really positive, not just the singularity one, but actually one that you think would actually represent a diverse view from humanity? Speaker 3 44:17 I think it comes down to two things, which we've already touched on. One is we have to resist the current paradigm and break up the Empire, and the other one is to then propose new alternatives and build those alternatives so that people can still benefit from technologies that do take a humanistic approach in their development. And we're already seeing on people doing both things with the resistance piece. I mean, there has been a blossoming of grassroots collective action movements all around the world where, you know, dozens of communities are now protesting the data centers that these companies are building. At a pace these companies are trying to build massive supercomputer facilities that are hundreds of times larger than the data centers that power our modern, digitally mediated lives, like pre AI, and there are all of these movements now mobilizing to actually stall and block and sometimes ban These projects, which is throwing a huge wrench in the ability of these companies to just simply continue being Imperial and top down and exploiting these communities. The artists and writers that we talked about that are using litigation or using these creative tactics like the nightshade tool, the fact that you know Megan Garcia, the mom of the 14 year old boy that died by suicide, she then sued, and led to many other parents to sue and started a global conversation about the ways that these models can psychologically prey on kids and harm those kids. And all like all of these are different movements in which people are actively asserting the fact that the vision that Silicon Valley is currently providing us is not one that works for them. It is not one that is broadly benefiting from and in the last like couple months, there was this new poll showing that 80% of Americans now think that the AI industry should be regulated. First of all, when was the last time 80% of Americans were on the same side of the issue? Like, that's crazy. But also, this is such a dramatic turnaround from like, when I remember when I was I was writing the book, the hype cycle was out of control. And I was like, this book is going to come out, and it's going to be so different from people's frame of reference for what AI is there they're being told AI is in their lives that I'm going to just sound like a conspiracy theorist. But by the time the book came out, the conversation inflict 180 because people started observing the impact in their communities, people started educating themselves about even further impacts, and then they started doing something about it, and actually actively, often not just change the public conversation, but also to start holding their elected officials accountable for introducing bills that would actually work for them, for containing the rapid proliferation of Silicon Valley's vision. These companies are now really being squeezed in many different ways, where they are spending hundreds of millions of dollars now on the midterm elections to try and continue forcing their agenda through. And those are hundreds of millions that they would not have had to spend if they didn't have to try and overcome the fact that 80% of Americans aren't agreeing with their approach now, and so yeah, like we were ready, like the movement to bring down the Empire and to create new, beautiful, community centric, ground up visions of technology is already here. It's already happening, and all of you can be part of that too. John Palfrey 48:11 We are going to turn to the slido app, and they've got some questions, and there's one that's been mostly uploaded. But to your last point, Karen, it was occurring to me recently that, you know, one of the things that companies are showing we're going to reduce the number of jobs available to another is we're going to take up all the natural resources to run this and others that maybe the robots will take over the earth. Another one is, perhaps, you know, your kids will actually have these harms to them. It's not a great deal that's being offered right now. You think about it, I think many people in America at least, and maybe elsewhere around the world are thinking that. Here's the upvoted number one question. It's got a celebrity slant. Ethan Hawke said he's very bored with AI. He believes it's going to change the world, but he's open to rebellion. Do you consider yourself to meet and Karen as rebels? Yes or no, yes. Speaker 2 49:01 All I ever tried was to be a good student, a good immigrant. I like I'm an accidental rebel. If I can say that was not the plan, John Palfrey 49:14 you're gonna ask them more afterwards about that. Okay, second most upvoted one, this is near and dear to my heart at MacArthur Foundation, you may know we've been funding public media for decades. It's one of our core activities, and we believe that local news in particular and the media needs support, as does the Pulitzer Center. But if AI becomes the primary interface to knowledge, how can public media and their questionnaire is talking about local PBS and NPR play a role in reflecting the public interest versus platform incentives. Speaker 3 49:44 This is, yeah, I spend a lot of time talking to journalists about this question we adopt, do we strike partnerships with these, these AI companies, or do we refrain? Do we build our own AI technologies? And what I always say to them, going back to this concept of a North Star like we first need to articulate, what is it that we're doing here? Why is it that we're doing like what we do and with the media, the North Star should always be that like we are here to fortify democracy, and so anything, any action that we take when it comes to the AI conversation should ultimately be still, how are we fortifying democracy? Like, how are we keeping the public informed? How are we giving them the tools to then engage in the democratic process, to then push back when elected officials aren't actually doing the things that they want, and when it comes to that question, then I'm always like, like, sometimes journalists will argue, well, all these surveys are showing that a lot of people are starting to get their news information from Chatbot. So we should, in fact, strike the partnership and just make our news available to on on chat, GPT, or on cloud, or Gemini and so forth. And I'm like, But did that solve is, is that actually helping with supporting democracy? I don't think so, because first and foremost, you are partnering with an empire that is actively also using its full force of capital and influence and power to undermine democracy. But second of all, like you are ultimately putting news that is meant to be highly accurate into a machine that comes out with where the information comes out. The other end like inaccurate. So are you actually informing the public? And third of all, the goal is actually not to just meet people on their consumer behaviors and just go with the winds of how which platforms people use. The goal is to make sure you ultimately deliver people the high quality information. So maybe we should be thinking beyond simply, like, do we use the AI platforms or not, and broaden our understanding of innovation to also include social innovation, like, one of the things that's been amazing about writing a book and going on book tour is this the first time that I've actually gotten to meet readers face to face, which I think is actually an indictment of journalism, like, why is it that throughout my career in journalism, it took writing a book to actually talk to people face to face about my work and then listen to them about what kinds of stories they want reflected in my future work? That is an aspect where we could innovate and have, you know, pop up events in communities, instead of simply continuing the work of divorcing ourselves from our readership and then trying to shove our stories through the stochastic machine to try and reach them because, because We didn't actually do a good job beforehand of building a robust direct relationship with our audience. John Palfrey 52:46 That's a very good prompt to meet. I know the next one's going to be up your alley, but you want to weigh in on the public media one though? Okay, no, she doesn't. This is a little bit of an inside baseball one for our field. But I do think that with a little explanation to me, can make it really interesting. The biggest AI companies, oh, no, sorry, that went away. There seems to be a growing sense of conflict between the tech justice. Well, they flipped. They flipped. And so I was, we'll get to the one you're voting. Change things on me in the middle, there seems to be a growing sense of conflict between the tech justice and the AI safety communities. Is it possible that both of these concerns are valid, and when you think about tech justice and AI safety, are those two different communities? And how would you think about Speaker 2 53:34 that one? I'm glad you only have six minutes left, because I could go on for such a long time on this topic, which really makes me so angry. Not your fault whoever asked this question. But so first of all, the AI safety, safety is a very definitive term, right? Like, is it safe for someone who doesn't know how to swim to go into the pool? No, it's not wishy washy. Like, maybe, if you use this ethical framework, no, like, you can't go in the pool, if you can't swim, right? Then ethics is like, well, if you use this, you know, this frame weight, and if you use that frame. So for some reason, these people decided to come and call themselves AI safety, and call people like me AI ethics. I don't have an issue with ethics, but I'm an electrical engineer. I'm a computer scientist. I've never taken an ethics class like I don't, but I care about the safety of these systems. I care about face recognition system, some miscategorize black people, and then they get arrested and they get somehow. That's ethics, but super intelligent machine God that doesn't exist, rendering humanity extinct, that's safety. Okay? So, you know, so both are both concerns valid? Absolutely not. Absolutely not. I'm talking about something real, empirically shown. You know, whatever they're talking about is science fiction with numbers made up, numbers that don't make sense. Lots of money behind them, and that's all I have to say. John Palfrey 55:20 Four more minutes on that too. Do you want to respond to Karen? Okay, that sounds good. The top upvoted one at the moment is, how well informed Do you believe most policymakers are, and particularly about the societal risks of AI and what more can we help them to do to govern effectively in the US and elsewhere, Speaker 3 55:44 I find that local policymakers and state policymakers are really well informed. As I've been, as I've been going on tour and going to a lot of different states, I will look into what bills are passing through, or like being proposed, and things like that. And these bills are really sophisticated. They are getting at some of the core issues around harm to children, around data centers, hiking up people's utilities and harming communities around all the many issues that people really care about. Why are they not then getting passed tech lobbying? So just a couple days ago, I was in Washington State, where there was this bill that was introduced by a representative to make companies that are building data centers pay for their own infrastructure expansion costs. Because this is not what they usually do. They will build a data center that requires enormous infrastructure expansion costs, and they roll those costs on to rate payers. And so across the US, there's been a huge increase in people's utility bills, sometimes even double or triple five years ago, and this bill would have forced the companies to take on their costs, and it would have also required the companies to have more transparency around their resource use to make it available. This should be, this should be publicly, publicly publicly available information, because the resources that they are using are public utilities, water, energy and other things. And like the day before I arrived for my book event, Microsoft and Amazon successfully through their tech lobby and killed it after it passed the House, it did not pass the Senate. And it's the same story with you know Assembly Member Rebecca Bauer Kahan in California, she proposed a bill after all of these horrible cases of children dying by suicide, where she said companies cannot market or sell their technologies to children if they cannot guarantee that their products will not be safe for kids, and she was defining in the bill safety as like psychological harms, like, if these chat bots are not able to like, be designed in a way that they're not going to persuade kids to self harm or engage in other types of horrific behavior, including killing themselves. They should not be available in the market. It passed both the House and the Senate in California, and then it was killed because Governor Gavin Newsom vetoed it after, once again, a huge blitz of tech lobbying. And so I think, like I was actually asking Rebecca, and I was like, what is the solution to this problem? Because these tech lobbyists just have an enormous amount of money. Is there just nothing that we can do as people to hold our policymakers accountable to what we want rather than what the companies want? And she was like, No. Like, there absolutely is so much to do, like building people powered movements where you hold your elected officials accountable and you make them feel that they will not have job security if you do not successfully get this bill through is the best antidote to tech lobbying. And she said, like in my own, my own experience, it's always the people powered movement that holds me accountable to my constituents, and so we just need more of John Palfrey 59:06 that, Karen, I feel like you have preempted the next question. We're going to ask it anyway, because it's been uploaded so many times by the brilliant society here in the room. But that question is, the biggest AI companies today look almost too big to fail. So where's the David strategy that could realistically challenge these clients? You've given us one, which is people power at the state legislative level. To me, how would you answer the David strategy? Is it dare? Are You the people Speaker 2 59:36 I'll say that don't get discouraged by feeling like everything has to be big and a big scale to make a difference, right? If you think about different movements, they start in a city and then in a country, and then they move, they travel, and so that's, I think that's how it works. You know, some of us couldn't show that their claims are false, but we can't just, you know, be stopped doing that, because then we don't show our alternative vision. There can be one organization that shows like lesson, one of the companies that I partner with has state of the art speech recognition systems for Ethiopian languages, two Ethiopian languages at literally, they spend 10s of 1000s of dollars gathering data that they paid everybody, and they have better stuff that works much better than any of these companies, that can inspire another company in another country, in another community, another you know? And I think that's how that's there's many Davids. I don't think there's one David that, John Palfrey 1:00:37 you know, yeah. The good news is small and medium sized models and companies and efforts can be totally beautiful and can can be persuasive and effective here. And as the South by Southwest organizer said, this was one of the themes that kept getting up voted as they're playing the conference. There's no surprise why you two are putting humanity at the center of AI. Hope you would join me in thanking to meet. You.