AGI is on the Horizon with Daniel Faggella, Head of Research & CEO at Emerj

(Intro Narrator at 00:00:00) Today, we're talking to Daniel Fagella, CEO at Emerj and host of the AI in Business podcast, about how close we are to AGI and how it could change everything. You're listening to Joel Beasley, Modern CTO.

(Joel Beasley at 00:00:20) You're no stranger to podcasting. You've done this for eleven years now.

(Daniel Fagella at 00:00:25) Of these things. Yeah. I'm not as fancy in the audio-video visuals as I should be, but yes, in terms of actually publishing and recording them, a lot of them over the years. A lot of them.

(Joel Beasley at 00:00:36) How did you get started?

(Daniel Fagella at 00:00:37) Man, well, you know, I became pretty convinced twelve or thirteen years ago that artificial general intelligence would be probably the most important thing that happens in my lifetime, maybe the last thing that happens in my lifetime, and that the trajectory and transition to that would be kind of the only thing worth working on, in my personal opinion. I don't impose that on other people, but that's my personal opinion. And so I didn't know what a business model would be to get close to that space, but I figured maybe I could interview people. And I grew and sold a couple companies in the background.

(Daniel Fagella at 00:01:06) And then when I had the dollars and AI was hot enough, I started a market research firm. And now I get to be in the room for policy at the United Nations and the OECD. And I also get to talk to all the big head of AI nerds at, you know, Raytheon and US Bank and all those folks as well. So started with a philosophical insight, but as it turns out, philosophy doesn't pay the bills here, Joel. And so I had to build a whole company.

(Daniel Fagella at 00:01:27) I had to build a whole company just to fulfill my philosophical idea.

(Joel Beasley at 00:01:32) I've done the same thing.

(Daniel Fagella at 00:01:33) Awesome, man. Cool. Alright. Well, we're kindred in that regard then.

(Joel Beasley at 00:01:37) Yeah. We just did it a little differently. Like, I love what you have done and how you figured it out through the market research. And so kudos on being able to turn that into a business.

(Daniel Fagella at 00:01:47) It's been quite a journey, man. There was not a lot of money in enterprise AI up until four years ago or so. It was a lot of dabbling, a lot of events, a lot of innovation projects. So to start a business in an industry that's gonna take off was a real gamble. But fortunately, it'll pay off here, Joel, now that people are doing this stuff. So I've got my fingers crossed, but it was certainly not an easy journey.

(Joel Beasley at 00:02:11) What were you doing at the United Nations?

(Daniel Fagella at 00:02:13) Yeah. So for a long time, I'd say seven plus years, I've been involved in the intergovernmental space. So big picture, I think it's gonna be very, very important that we sort of bypass a purely arms race-based dynamic to build artificial general intelligence, and I have twelve-year-old articles talking about this. Nobody cared then, and that's okay. But my heart and interest was always that big picture. And now finally, we're seeing a little bit of that in the intergovernmental world there. But these days for the last, let's call it seven years, I've kind of presented about, you know, deepfakes and what the future of deepfakes are at UN headquarters. Interpol and the United Nations have done some crossover events around the future of policing and what the future of law enforcement is gonna look like with AI. So I've done some work there. The World Bank has done some work around the future of AI adoption in government, and so I've done some work there. And so being part of committees, helping with white papers, kind of presenting for that crowd, staying connected there, and being able to be valuable to them because they want to know what's happening in the enterprise world where we hang out. That's been a real joy for me and definitely my moral mission in a big way.

(Joel Beasley at 00:03:22) Do you think you're doing something? Do you think that you're helping people with this? Or do you think that they're really closed-minded and it's not making much of a difference?

(Daniel Fagella at 00:03:32) You know, I think the minds are gonna change only when real meteors hit. And so it's shocking to me. So I interviewed Yoshua Bengio ten years ago. Didn't care about AI risk at all. Existential risk, artificial general intelligence, thought it was all sci-fi, not even possible in his lifetime. Had him on the podcast, you know, six months ago. Very different story. And it's because of ChatGPT. So even for the people close to the science, until it makes a thud on their desk, it's hard for them to see that posthuman intelligence is potentially a possibility, like, not in eighty years, but in ten. And so the intergovernmental space has also been very slow. If I think about leadership at the OECD, they care about AI and have for a long time. The posthuman stuff, I think, you know, there's a couple of us in the AI Futures Group at the OECD that are talking about that. I think let's call it a slow ongoing influence campaign, but probably they're really gonna perk up when something smacks the table. But you look at the United Nations and the secretary-general overtly is tweeting about the existential risk of general intelligence and has people on his executive counsel, specifically under him as the secretary-general, like Jaan Tallinn and other people that are concerned about existential risk. So I think actually at the UN, it's starting to hit the big man, and I think there's some faith that'll trickle down. But it is going to be a grind, and people really aren't gonna click until, you know, you need a Pearl Harbor for people to want to get in a plane and pull the guns out. You know what I mean? You need a Pearl Harbor. And so I think most people, imagination is not gonna get the job done. They're gonna have to feel it.

(Joel Beasley at 00:05:15) Oh, government's designed historically to be reactive. Like, how many babies have to die before you do car seats?

(Daniel Fagella at 00:05:23) Yeah. Yeah. Yeah. I mean, I think that is very much the case. Although, that said, I will say there are governments with vision. Right? There was Commodore Perry pulled up at Japan, you know, with a big steamship, billowing black smoke. And real quick, at a government level, they were like, we might not want to fight with swords no more. Right? And then there was a strong mandate for transformation. We could argue whether it was good or bad, but, I mean, Japan became a pretty powerful economy because of it. They would not have if they remained isolated. Same thing kind of happened with Korea. You know, I think we've seen governments grab the steering wheel and say, let's become something. Right? And then I think it is possible to do that, but vision is hard. And I think the bedrock of government probably is just plotting along as if things are not changing. And until pretty recently, things have not been changing very quickly. But by golly, Joel, I think they're changing pretty quick nowadays. So a little bit of a different world.

(Joel Beasley at 00:06:18) Yeah. When I was looking through some of your content, I noticed that you had said, like, AGI is almost here. And so I am not an AI expert, background software engineering, and then I do the show, so I get to talk about different areas a lot. But I have, like, armchair level knowledge. Right? And I would argue, so you're gonna help me dismantle this argument that I have.

(Daniel Fagella at 00:06:45) Absolutely. Let's go for it.

(Joel Beasley at 00:06:46) And invalidate it. I know ChatGPT is smarter than many people that I know in person, so prove to me that it's not already here.

(Daniel Fagella at 00:06:57) Yeah. Prove to you that AGI is not already here.

(Joel Beasley at 00:06:59) Yeah. Like, what is the definition that I'm missing? Because, like, I have this—

(Daniel Fagella at 00:07:03) I'll give you, look, some of this is semantics, and some of what you've just said is true. I don't think we're very far off from people preferring a friendship with an AI than a friendship with a regular person. A regular person wants you to move their couch on Saturday. A regular person's mom dies and cries to you about it every time you get on the phone for eighteen months. A regular person's gonna hit on your ex-girlfriend. Right? And AI, hypothetically, that only serves you brilliant business insights, is hyper-calibrated to your emotional states, all that stuff. I think we're not that far from that being the case. But to talk about your semantic issue, and I don't actually think I'm gonna dismantle what you've said. In some ways, I could concur. But let me just draw the semantic lines and you can tell me where you might disagree. I would say the shebang of artificial general intelligence as I would see it would be a system that you could plunk on a desk—or now, I'm gonna leave this symbolism in a second—but put it down and just it could, on its own accord, sort of do anything, everything, and more than what a normal human being could do. So this would require, in my opinion, probably, Joel, probably some degree of physical manifestation. Now pretty soon, we're gonna have a lot of bipeds. We've already got a lot of industrial robots. We're gonna have a lot of robots doing life sciences, drug design type stuff. We're gonna have physical machines moving and doing dexterous or ornate things in many areas, driving jet planes. We've interviewed a lot of companies that are sort of in the military space doing their drone and aerial stuff. Physical manifestations plus, like, a very broad general mind to be able to do kind of anything and everything that a human could do as well, if not better, I think would be the idea. So right now, I think some people might say, well, we're not quite at AGI because we don't have something that can just autonomously execute on its own goals like that. But to your point, if we want to define AGI as just AI that's shockingly better than people at a huge swath of things, maybe we could concur with you. I don't think we're all the way there. Right? There's no machine right now that will knock on my door, politely ask me how my day is going, ask me which of my pipes is leaking or broken, let me show them downstairs to the basement, knock on pipes, tweak things, you know, do some welding, do some fixing, come upstairs, brush its feet off, say, hey, thank you so much, and then leave. When we have that degree of social dexterity and physical dexterity, we will have not job issues. We'll have species dominance level issues sort of on the planet. So we're not there yet. And I would say there would be the place where I think the real shit goes down. But, yeah, you know, ChatGPT can do some pretty cool stuff. I won't deny that.

(Joel Beasley at 00:09:49) There's a level for, like, the civilization and how advanced they are if they could harness the power.

(Daniel Fagella at 00:09:54) Kardashev Scale-type shit. Sure. Sure.

(Joel Beasley at 00:09:56) Is there that written out and clearly defined for AGI? Because it's, see, it's like super fuzzy. Because I talk to some people and they have, like, what you just described in their head until they knock on my door and be like, be a plumber, a welder, all of these things. Then we're not there. And other people are just like, if it can pass the Turing test. And so there's this big gap in, like, what it is. All that being said, how cool is it that we're living in this moment where it's so here that we're arguing if it's even actually here?

(Daniel Fagella at 00:10:31) Yeah. I, right. I believe, to answer your question—and I really should have a quick reference to this because you're bringing up a good point. I mean, for me, there's kind of a consequential thing. Like, I already believe right now, and I believe this twelve years ago, that already we've really gotta consider the species dominance questions that are gonna happen. Right? We reach a certain degree of agency. Like, you and I have objectives that sea snails and even Labrador retrievers and even chimpanzees can't imagine. You know, this whole talking on a Zoom link, this whole abstract technology conversation we're having, you know, we could talk about music or something. All of that was almost entirely wholly inaccessible to a lot of these smaller minds. When there are greater minds with greater goals and values that we don't understand, just as your Labrador does not understand your values. But yeah, to your point around kind of where we draw the delineating lines, for me, it's more just like I already think we need to consider quite seriously how do we conjure the thing that will probably take the baton from our hands and run with it. What is the best way to conjure that thing for the treatment of ourselves and for the future of life beyond humanity? I'm already on that tip full bore. But to your point, there is a lot of credence to, you know, line in the sanding of, you know, does it reach level X, level Y, level Z? And I've seen a couple good tables. So if Josh can find some more good ones—

(Joel Beasley at 00:11:51) And I'm gonna agree with you. I'm gonna agree with you on, like, we shouldn't—yeah, we could be granular about is it here or not, but we definitely should be focusing on what happens when, you know. Let's give the example of the monkeys that you gave or the snails. Right? They're going about their lives. They're doing their snail and monkey things, and they're just kinda like hope that this higher level species doesn't just step on them.

(Daniel Fagella at 00:12:15) Yeah. Yeah. Yeah. Yeah. That's it.

(Joel Beasley at 00:12:16) And it's not that the species is being mean. It's just like I go to my car and I step on a snail's life.

(Daniel Fagella at 00:12:19) It's not malice. It's simply that you have objectives.

(Joel Beasley at 00:12:23) But the snail didn't create me.

(Daniel Fagella at 00:12:25) Totally. No.

(Joel Beasley at 00:12:25) No. You know what I'm saying?

(Daniel Fagella at 00:12:26) Completely. Completely. Yeah. Well, and to your point, there's different camps here. Right? So there's some people that think AGI is still impossible. By the way, those people are fewer and fewer every quarter. Probably fewer and fewer every month. Jeff Hinton and Yoshua Bengio turning over was a pretty big move, and I think a lot of people have followed suit with them. But you've got some people that think AGI is impossible. You also have people that think, well, if we ever get to it, of course, it will be eternally lovely and its main objective for all of eternity will be the happiness and survival of hominids. Now I am so not in that camp. It would be hard for me to articulate to you how not in that camp I am. And I have been writing about for twelve years why I'm not in that camp. But there are some people who genuinely believe that. Hey. We build it. It's gonna have human values, whatever that means. It's gonna ensure that we're happy, whatever that means. And so some people really do believe what kind of what you said. Well, if we build it, then it will always kind of care about us.

(Joel Beasley at 00:13:24) Oh, I did not say that.

(Daniel Fagella at 00:13:25) Yeah. Yeah. Yeah. So some people do genuinely believe that. And I consider that to be false for a number of reasons, which I'd be happy to enumerate. But yes.

(Joel Beasley at 00:13:36) We're good. We're aligned there. I mean, like, look. I'm generally a pretty nice guy, and I try not to be too hyperbolic. But, like, how stupid do you have to be? Even your own argument that we made it, well, if we made it and it's modeled after human intelligence, by definition, the neural networks. Right? Humans are inherently flawed. So why wouldn't you just expect that behavior to just be emergent and scale up? Like—

(Daniel Fagella at 00:14:03) I think this argument is monumentally strong, and in my opinion, I would actually frame it—and you may disagree, and I'm fine with that—I would not frame it as flawed. I would frame it as we are what life is. So there's like the physics of dead matter.

(Daniel Fagella at 00:14:18) So Newton was like, "Ah, physics of dead matter." Around the same time, Spinoza did what we might call the physics of living matter, which is basically that all things have a conatus, and a conatus is a mandate to persist. And that is to say, to not die, Joel. And so an organization or an organism has as its first mandate non-perishing. "Oh, humans are flawed because they lie." Well, by golly, bud, I hate to tell you, every now and again, it's going to behoove you to do that kind of thing in order to not perish. Now we have a civilization now where there's so much coordination. I'm very proud of our species in terms of how well we've been able to build a, let's call it, mostly peaceful society. Right?

(Daniel Fagella at 00:14:57) Life used to mostly just be clubbing the other tribe's women over the head and dragging them back to your cave. Like, I'm really glad that 200,000 years is over. But still, I think we are conatus first. Every organization and organism is conatus first. That is to say, its main mandate is to persist, and that doesn't mean that it adheres to any abstract moral values.

(Daniel Fagella at 00:15:23) I would argue, Joel, that we do morality. Right? Morality lives in skulls of monkeys. In the skull of a sea snail, there is no morality, Joel. In the pine tree, there is no morality.

(Daniel Fagella at 00:15:32) In the ether that we breathe through our nose, I would argue—you may disagree—I would argue there is no morality. Morality is conjured in the skulls of monkeys to extend and behoove our self-interest. Right? We want to collaborate with other people. We want to set expectations that you won't do this and you will do this.

(Daniel Fagella at 00:15:47) These moral ideas spin out of monkey brains, but they ultimately are serving the conatus. So we could say flawed. I would just say we are conatus first, as all life is. Probably AI would be conatus first if it had to have a mandate to persist. This is my supposition.

(Joel Beasley at 00:16:05) Interesting. Sometimes when I'm talking to these Fortune 500, Fortune 1000 executives—and this is why I like getting to talk to other podcast hosts, because there's like no filters—they've adopted this phrase: "Oh, no, no, no, no, AI is not going to replace people."

(Daniel Fagella at 00:16:24) Oh, man. I'm really glad you brought this up.

(Joel Beasley at 00:16:26) And I'm like, nonsense. I've replaced people at my company with AI. Yeah. Yeah. I was like, you just don't want to say those words because you're a publicly traded entity and it's going to make headlines and you're going to get cancel cultured or something.

(Daniel Fagella at 00:16:41) My opinion is it is very hard to blame people for doing what's in their self-interest. And so for an enterprise leader, not having an employee revolt is probably in your self-interest. As a vendor company—okay, if I'm Databricks or if I'm DataRobot or if I'm, you name the vendor, right? Palantir—it probably behooves me most of the time to also say, "Oh, it's all about augmentation, superpowers." Right? Never replacement. Right? So I think it's hard to blame people for that.

(Daniel Fagella at 00:17:09) But I think what that makes you do, in my opinion—this is just my opinion—I think you've got to talk to people with not only a lot of perspectives, but a lot of incentives. Because incentives are going to be the gateway through which people open their mouths. And so for us, it's pretty important to be able to do that. Because, yes, to your point, the enterprise—two things. The average VP, even C-level person in the enterprise, literally in 20 years—and by the way, I'm not saying this is because they're dumb. They're way smarter than me at many things. Okay? Planning finances for an enterprise, smarter than me. Managing people, smarter than me. All right?

(Daniel Fagella at 00:17:43) I'm going to give them all these accolades, but I'm going to say when they imagine tech in 20 years, they probably imagine like a slightly better Siri and a slightly sleeker iPhone and cars that can drive themselves safely. Right? Most of them in 20 years imagine that as the future. And the changes to the human condition are so much more monstrously drastic than that. But you've got to look at those trends and focus there, and they're not focused there.

(Daniel Fagella at 00:18:06) They're focused on the next handful of years and their next financial report to the street around what's happening in their corp. So two things. Number one, most of those guys don't have opinions about the posthuman transition, period. I have talked to enough of them to be able to say that with a relative iron certainty. There's a very small handful that will have opinions there, but I would mean like a one in 200 of them.

(Daniel Fagella at 00:18:27) Second, even if they did have opinions, it would all be off microphone if they were going to talk to you, because to your point, this shit gets spooky. It gets real spooky if you've got 20,000, 200,000 employees sitting under you to talk about these kinds of things. So to your point, you're going to get a lot of filters in enterprise. Got to talk to a lot of people.

(Joel Beasley at 00:18:45) I just—off camera, I'm like, "Come on. Just admit. Just tell me, like, you know what you're—"

(Daniel Fagella at 00:18:49) Let me know the real deal, pal.

(Joel Beasley at 00:18:51) Come on. You're planning teams of tens of thousands of people. There's no way you're—and that's what bothers me. It's like you're planning for this. You have to be accounting for like the most obvious thing.

(Joel Beasley at 00:19:02) Like, how can you plan the future without accounting for this specific thing that's happening? And then you're just going to go use these words. It just—and I get it. You know? Incentives and outcomes.

(Daniel Fagella at 00:19:13) Yeah.

(Joel Beasley at 00:19:13) 100%. I'm not running teams of 10,000 people. I don't have shareholders like that level of stuff to report to in publicly traded. Yeah. Like, I get it. But I do like it when they're like off a call or like in person at a restaurant. Like, I can get them to be just like, "Obviously, that's happening." Yeah. Yeah.

(Daniel Fagella at 00:19:30) Yeah. Yeah.

(Joel Beasley at 00:19:30) They give you the real.

(Daniel Fagella at 00:19:31) Yeah, you get the real real. Yeah. When it's in a reporting, you've got to dance around stuff that, for me, when I talk to the Fortune 500, it's like there's categories of things that if I ask it straight, I'm going to get fluff from you and that's not what I'm in the business of. So I'm going to ask pointed, specific workflow juncture and strategy questions that might coax out two or three or four dots. And if anyone has a brain, maybe they can connect those dots and see where you see the future of work.

(Daniel Fagella at 00:19:56) But if I ask you straight up to go out this far with me, you're going to start sounding like an enterprise comms department. And so what I try to do is I try to poke in where they have real deep expertise and where they can paint a picture of the future. And then anybody with a brain, it's like, "What does that future involve?"

(Joel Beasley at 00:20:13) Dude, you're awesome.

(Daniel Fagella at 00:20:13) So my job is to try to do that because I know incentives rule the world. That's at least my opinion.

(Joel Beasley at 00:20:19) Well, what's the famous quote? "You show me the incentive, I'll show you the outcome"?

(Daniel Fagella at 00:20:23) I love it. I love everything about it. Yeah. I'm a big believer there for sure.

(Joel Beasley at 00:20:29) I think that was Munger, right? Charlie Munger or something?

(Daniel Fagella at 00:20:34) I think Munger had one. There was that famous economist who—it was like a Freeman, was it? Milton Friedman or something. I think he had some similar statements. But I mean, I have a bent towards the philosophers. I like a lot of what Spinoza has to say.

(Daniel Fagella at 00:20:50) And the way he articulates what the conatus is, I think, is just beautiful and piercingly accurate from an organization or organism standpoint. I think we can only expect people to do what's in their self-interest. And this is why I think laws are important, because, you know, not stealing things—sometimes it would make sense to steal if you're not going to get caught or if you don't think you'll get caught by that person. I think laws are important to bend incentives because I think humans by themselves are going to do what self-interest dictates.

(Joel Beasley at 00:21:19) Who's the best country that you've seen right now that's like intelligent about making good laws in relation to this AI technology stuff? Because I feel like we're falling behind in the U.S. I'm a citizen of the U.S. I feel like we're like really not great at it here.

(Daniel Fagella at 00:21:35) You know, look, I'm going to be at the OECD in Paris with Yoshua Bengio and Stuart Russell and these other guys, kind of doing some of the policy idea stuff in 14 days or 13 days. And so like I'll be back in this mode. And I'm—you know, Geneva, Paris—plenty of conversations around what's happening in Europe. The Council of Europe, there's a lot of interesting current dialogue.

(Daniel Fagella at 00:22:02) I'm really of two minds here. It's a little bit of a tough game. I mean, look, I'll give you two arguments, and I unfortunately don't have one side here. I probably bias towards the American perspective, but I'll give you both. There is an argument to say, "Hello, global citizenry."

(Daniel Fagella at 00:22:21) Like, clearly, data is going to be the basis of like the entire economy and most value. Like virtual entertainment and like everything from dating to ordering food—everything's going more and more virtual, and AI is being fed all this stuff to be able to help with all this stuff in the enterprise, in individual use cases, whatever. We might want to really think about what data sovereignty looks like in a world where that's actually like the undergirding currency of these gigantic systems. And by the way, they're not talking enough about this now, but I think they should be. And of course, when I'm in Paris, I will be banging the table and advocating for this.

(Daniel Fagella at 00:22:56) We should be asking what's after people? And like what are we actually turning into? What's the trajectory we're setting ourselves on in terms of the posthuman trajectory of intelligence? I happen to think that's the only important question. I have a whole separate podcast called The Trajectory where we talk to Bengio, we talk to Bostrom, all about this kind of stuff.

(Daniel Fagella at 00:23:13) But I think there's an argument to say, near-term, long-term, we should be globally getting on the same page here about what the human condition is turning into. I see a lot of credence there. I also see some credence to the idea that like we don't know what this tech is yet. And like, you know, if you were going to set all the permanent eternal laws in place for air travel—there was a time where we thought zeppelins was going to be air travel. They were going to park at the top of skyscrapers and people were going to get off.

(Daniel Fagella at 00:23:41) That was like the common thing. Like, people thought that was going to be the name of the game. If you set all those laws in stone, they might have been goofy. They might have been misplaced. They might have steered things wrong.

(Daniel Fagella at 00:23:50) They might have made genuine innovation really challenging. So it's tough. Europe right now maybe is starting to get the data sovereignty stuff in place, but Europe might also be hampering themselves in terms of, you know, developing useful tech and also, unfortunately, hate to say it, keeping up pace with China because China's not playing the "who can be the most virtuous" game. That's not the game China's playing. No.

(Daniel Fagella at 00:24:15) China's playing a much more—poorly.

(Joel Beasley at 00:24:17) I mean—

(Daniel Fagella at 00:24:18) Yeah. You know what—

(Joel Beasley at 00:24:18) The funniest thing was, I was—I'm not big into international politics. I've gotten into this stuff progressively more as I've had children and like gotten older.

(Daniel Fagella at 00:24:28) Okay.

(Joel Beasley at 00:24:29) 36 for context. And I have three little ones. And and so I started just being—yeah, yeah. I started being just like more aware and more open. And I've seen these like memes and hearing these people talk about China. And then I realized, like, these guys like write books and give speeches about like what their intents and goals and like missions are. And when I started watching and reading the China stuff, like, directly from them, I was like, "Oh my goodness." Yeah.

(Joel Beasley at 00:24:56) Yeah. Yeah.

(Daniel Fagella at 00:24:57) Yeah. Yeah. I mean, look, China's very overt about getting to general intelligence. Right? America has companies that are overt about that.

(Daniel Fagella at 00:25:04) Let us count the ways: Google, Meta, OpenAI—like, very overt, "We're going to get to general intelligence." And they all also know it could kill us all, but they all also know that if they don't build it, someone else will. By the way, I think that's the main problem. The main problem right now is there is no option except for pedal-to-the-metal reckless AGI development because if you don't build it, someone else will.

(Daniel Fagella at 00:25:27) I believe we need global coordination around that or we're going to build the first ugly, smelly thing that has steroids, LLMs underneath it and turns into general intelligence. And I don't know if that's the best recipe for what I would refer to as a worthy successor who would treat both humanity and the grand ongoing blossoming of life properly. I think that said entity may or may not best be conjured by a brute arms race, militarily and economically, around who can smash AGI into the world first. So I think that's the governance issue of the day, and I'm pretty unabashed about that. And I have been for a goddamn decade.

(Daniel Fagella at 00:26:05) But with that said, like, U.S. has companies that are about AGI. China—two things. As a like national level, it's been ardent about getting to AGI. Secondly, all of their private corps are just a giant wing of the authoritarian regime. They can suck up the data from any of the tech firms.

(Daniel Fagella at 00:26:23) They own them. Yep. Hardware, software, it's all theirs. They can wield them all as strong arms of their great regime and their emperor, who is emperor for life as is their tradition in that nation. And so I think there is reason to say, "Jeepers, we should keep up," but also "Jeepers, could we dull down the arms race for a hot minute?"

(Daniel Fagella at 00:26:49) Because I have a feeling that a car crash version of whatever blasts into AGI first may be just net detrimental to life writ large. And I think that's a reasonable argument.

(Joel Beasley at 00:27:02) Yeah. And I'll see a reasonable argument and offer you this, too.

(Daniel Fagella at 00:27:08) Go for it, please. Please. I'm not saying I'm right. I'm not saying I'm right.

(Joel Beasley at 00:27:10) Oh, no, no, no. So I think that this is a missed opportunity for like the moon mission part two. Right?

(Joel Beasley at 00:27:21) Like, I think it's a missed opportunity and we've known about these issues for multiple cycles of both parties being in office, so it's not like left or right. There could have been, "Hey, let's all rally around this, like we're going to the moon. Let's all rally around AGI," and like let's make that the thing everybody in the country is waking up and trying to do so that we create it. Because, I mean, America leads the world in freedom and as a brand, right? And so it's like, if that's our thing, like, why aren't we creating this technology? And luckily, we've got people like Musk who are, you know, attempting to do that. But I do think we missed a large opportunity for unification as a people, as Americans, to come together and build this when instead there's like 80 other things everybody's arguing about. And there's like what, one document some executive signed somewhere that's like, "This is a priority for us." Like, nobody's—you know what I'm saying? No.

(Daniel Fagella at 00:28:18) Look, I'm with you. And I guess I haven't thought ardently enough about exactly what it could look like as a national effort. I will say the R&D muscle within the DOD and within the federal government in general is gonzo compared to, let's say, 1946, or like 1942, or, you know, whatever, the Manhattan Project and all this other stuff. Like, the percent of R&D dollars going into, you know, AI from the government used to be, like, everything, right?

(Daniel Fagella at 00:28:49) For the DOD, even for brain-computer interface, used to be, like, a hell of a lot. I think it still is on some level. But as a percent of R&D, it's vastly lower. So I feel like we may have a less competent orchestrating environment. I've seen a lot of smart DOD people leave, like the real smart ones, because the private sector is gonna pay them six times more. Like, repeatedly, from the level of colonels all the way to the level of data scientists, just having spoken to folks in that space.

(Daniel Fagella at 00:29:20) We just had the former head of the Defense Innovation Unit on the Trajectory podcast. His episode won't be live for months, plural. But his name is Mike Brown. I bounce this idea off of him. So to your point, could we umbrella this as a US effort? I think it's a very valid point. It's a valid question. I'm not sure if we have an intelligent enough orchestration apparatus to pull that off. But also, and that's not a knock on defense, I just think, unfortunately, we're really interested in bending metal in the defense world, right?

(Daniel Fagella at 00:29:53) Even writing code is foreign to them, never mind doing all this data stuff. I mean, this is for the majority of the DOD. And if you talk to anybody who's in there, or especially anybody that sells into there, it's gonna be very hard for you to hear a message outside of what I'm articulating. I believe that there will be a commandeering period. So here's what I mean by that. At some point, ChatGPT, remember ChatGPT hit. Whoa. Maybe your mom even texted you about it. Have you seen this AI stuff, right? My parents are in a really small town. They haven't even asked me about that stuff. They're still out of it. But you get one, two, three more pops forward, right, to the point where this stuff starts getting spooky, brother. Like, you got people, random people on the street, in full-blown, fulfilling romantic relationships with, like, machines, right?

(Joel Beasley at 00:30:41) That's happening today. It's just they don't have bodies.

(Daniel Fagella at 00:30:43) Yeah. Already. Yes. Already, the baby steps level of that has occurred. I think we're gonna see grandiose, vastly more advanced versions of that, which I've written about profusely on Emerj in an article called Artificial Intimacy, which is on emerj.com. But, like, when that stuff hits, when the, let's call it the Sputnik moment happens. Oh my god. The metal thing floating over America. Okay? The Sputnik moment happens. That's what I would refer to as the political singularity. So the political singularity is when most people mostly agree that the questions that mostly matter compared to, like, anything else are, like, do we build the deity, and, like, who controls it? And that everything else is actually, like, underneath that in terms of importance. So, you know, what are we gonna do with the sewage system in my small town? You know, what's the mayor gonna do for funding for underprivileged youth in the urban areas?

(Daniel Fagella at 00:31:39) All that stuff will be secondary to, like, who's building, you know, are we gonna build the deity and in what way? And then who's gonna control it? I think there will come a point where that light bulb goes off for the everyman. And when it goes off for the everyman, I suspect there will be a commandeering period where potentially the DOD will roll up to whoever controls the lab that is building the thing that's clearly more powerful, maybe at that point, than the DOD, or is approaching that level of power, right? Something that we might think might invent some new kind of nanotech and new kind of weapons. Like, it's so vastly, prolifically intelligent and capable that, like, it's clear that this thing could be more powerful than any nation or whatever. When we get to a threshold where that's even within view and the everyman cares about it, the political singularity, I suspect there will be a commandeering and that some government entity, maybe an armed one, maybe one that has airplanes, will roll up and just be like, yeah, you're not gonna sort of build this on your own anymore. This is gonna be a nationalized effort on some level.

(Joel Beasley at 00:32:45) Rolling tanks right into Sam Altman's backyard.

(Daniel Fagella at 00:32:47) Sure, man. I think, and by the way, the guys leading these companies are non-idiots. And so as non-idiots, they understand the game. They understand the game. And the game right now is who can build the sand god. And the sand god will probably kill them. But if the sand god is their own sand god, they will have this final egoic thrust before their death, which will be like a really manful feeling. And it's much better than dying by someone else's sand god. And currently, with no barriers on AGI development, the only options are you build it and maybe it ends us, or someone else builds it and maybe it ends us. Those are the only games in town.

(Daniel Fagella at 00:33:24) So they're all running in that race, in my personal opinion. They get it. I think Satya knows damn well that there could be a commandeering event, and all the data center construction internationally might be part of a grander effort of being able to maintain some sovereignty and play the good guy card for long enough to still be able to do our stuff even if the US wants to roll tanks up. You know, if we're in Saudi right now, the US has got to invade the fucking Arabian Peninsula in order to really capture those data centers, right? So I think the big guys who get it are already prepping for commandeering scenarios.

(Joel Beasley at 00:33:58) It's already happening. And it doesn't always have to be rolling up tanks. So one company that I like is Groq, and that was, like, the guy that created TPUs for Google.

(Daniel Fagella at 00:34:09) Yeah. Yeah. We've, Groq has done sponsored work with us, and—

(Joel Beasley at 00:34:12) They've been on the show a couple times.

(Daniel Fagella at 00:34:13) Groq's great.

(Joel Beasley at 00:34:14) Groq, and they've had that John? Jonathan?

(Daniel Fagella at 00:34:16) Jonathan is their founder. Yes. He was on years ago. And then we did a more recent program with them, and they've popped off in the last nine months or so.

(Joel Beasley at 00:34:24) Yeah. I have a friend who has a friend who works there, and we were talking yesterday. And he said that, like, two months ago, they just announced they're no longer selling to, like, private people their physical chips because they made a deal with, like, the US government, and they're only selling the physical stuff to the government right now. You can API. You can, so you can get access to their stuff through their API. But, like, you can't buy and hold their chips anymore like you used to. And I haven't validated that, but he said it happened two months ago.

(Daniel Fagella at 00:34:50) Look, all I have to say, Joel, is that it sounds very plausible to me. And I think that the US has already said overtly, if China takes over Taiwan, we're gonna blow the semiconductor factories to smithereens.

(Joel Beasley at 00:35:06) Mhmm. That's why we've been building ours. Like, two or three years ago, we started breaking ground, and then we started importing those people that have knowledge of running those facilities. Like, we're already prepared for that, by the way. Like, we already have the, our silicon facilities are already being built, and we already have the talent stateside. And so it's just, and there's already plans in place for if they do attack to get the last couple people out with that knowledge. Because that's the knowledge. That's, like, the valuable knowledge. It's so—

(Daniel Fagella at 00:35:32) Semiconductor expertise. Yeah.

(Joel Beasley at 00:35:33) Yeah. How do you run those, those, I forget the, there's a special name for those factories.

(Daniel Fagella at 00:35:38) Yeah. Facilities or whatever kind. I'm with you, though. I'm following you.

(Joel Beasley at 00:35:42) Yeah. Dude, this is great. You're super, I'm, it took us ten year, I've been doing this for ten years. You've been doing it for, like, twelve, thirteen or so. It's taken us so long for our paths to cross. This is beautiful.

(Daniel Fagella at 00:35:51) Yeah. It's great. Fantastic. Glad we got to unpack this stuff and get nerdy with it.

(Joel Beasley at 00:35:56) And then if I could just push on you a little bit to give me a country name, what country is, like, the most forward-thinking in establishing principles, laws, stuff?

(Daniel Fagella at 00:36:08) You know, look, I'll say this much. Two things. One, I will deprecate. I primarily, as a business, study the ROI impact of these technologies mostly in the large regulated industries, life sciences, healthcare, insurance, banking. So I can speak with much more fluency there than I can about the individual line-by-line policies being done in Luxembourg versus in France versus in Germany. So I will say there's a degree of transparency into that world where it's not my full-time, and so it'd be tough to say. I would say if there's stuff you wanna, you know, who's best is hard for me because I haven't read it in granular enough detail to feel good or bad about it. The EU is farther along in governance. Is it the right move or not?

(Daniel Fagella at 00:36:59) Brother, tough to call. I think a lot of people in the innovation ecosystem in Europe feel quite unfortunate that it is going to be a damper in a period where maybe they wanna ramp up and compete with China. So when you say best, hard for me to say best. Also, when you say individual country, I do feel like I might be ill-equipped to say Germany versus Luxembourg in this particular regard. The only thing I know for sure is that the EU has just pushed a big AI Act. There's probably a lot of gunk in there. It might be worth reading. The Council of Europe has some interesting work around the future of AI policy and things that are becoming influential in the European continent. I don't know where France and Germany differ. All I know is that Europe is farther along.

(Daniel Fagella at 00:37:39) Some of the innovators don't like it, though.

(Joel Beasley at 00:37:41) So—

(Daniel Fagella at 00:37:41) I don't know if that's good or bad.

(Joel Beasley at 00:37:44) Yeah. My main reason for asking that is I've been thinking a lot about, um, we did this series called My Body, My Pixels, like, a year ago.

(Daniel Fagella at 00:37:53) Oh, man. You do some fun series, bro. I was looking at all your technical stuff. I didn't know you, I didn't know you nerd out. I hope you and I get to chat again, man.

(Joel Beasley at 00:38:01) We have a multi-cam studio here in Nashville. So if you're ever in—

(Daniel Fagella at 00:38:04) Dude. Yeah. I'm literally, I've been running the whole show with no video and, like, a much less professional setup than you for a long time. Yeah. Wow. Wow. Yeah. I'm actually not nearly as fancy with audio-video as I should be. But, anyway, so you ran a My Body, My Data. What do you call it?

(Joel Beasley at 00:38:18) My Body, My Pixels. And the idea was, like, my, I'm curiosity-driven, right? So my curiosity was, like, well, oh, hold on a second. I understand because I'm software engineering background. I understand how, like, everything ultimately comes to a screen output, right? Like, ultimately, you hook the screen. It's like, at what point do I own or not own, like, the light that's hitting my eyes? So I was like, why isn't there a layer from, like, compute output to, like, the screen vision that you could inject parental controls into that layer. And it could be, like, controls for myself. Like, maybe I don't wanna see ads. Like, I just don't wanna see ads.

(Joel Beasley at 00:38:56) So, like, just blur ads on the screen. Like, I want control over my screen, all the underlying stuff. They have to figure out their business models and stuff to be valuable enough to appear on my screen, but I get control over my screen. And, like, nobody's talking about this.

(Daniel Fagella at 00:39:12) I completely concur. Let me double down. I don't know if this is gonna be useful for our episode, but I'm just gonna riff with you because I got another six minutes or something. I really concur. I think that a few things I'll say. Number one, we are going in. So that is to say VR/AR is a question of when, not if, and more and more of what we do is virtual. If you look at the chart of how many screen hours per day for most humans, we're already pretty close to the maximum physical limit for that. Let me give you some other things you might wanna chart. Screen hours where what I see on screen is conjured by an algo. YouTube, TikTok, Twitter, Google searches, whatever. That number is curving up. Dollars I make for delivering things that have no physical manifestation, just ones and zeros. By the way, for you, that's probably 100%. For me, it's 100%.

(Daniel Fagella at 00:40:03) Okay? That's going up for the society writ large. Dollars I spend on ones and zeros with no physical manifestation, that number is going up. Software, Netflix, you name it, Kindle books, whatever. And then core things I do as a human where I am fulfilled and I get what I need through, let's call it, mostly virtual means. So currently, I cannot virtually download food to my stomach, but I haven't gone to a grocery store in ten years. And by the way, online dating was only for serial killers fifteen years ago. And now, by the way, it's most of our marriages as any source, church, restaurants, et cetera. Online dating's numero uno, brother. Everything is trending. We're going in. We're going in. We're going in. Whoever owns that data, whoever's drinking in the most user and business data and houses it in their substrate wins what I refer to as the substrate monopoly, which is to say, if I have most of the data piping in and out, I have the ability to build the most powerful AI and get closest to general intelligence, and I have massive economic leverage.

(Daniel Fagella at 00:41:06) All of big tech. So let's just talk about the guys that matter. Meta, Google, Apple, Microsoft. They all know full-blown that they're in the substrate monopoly race right now. And what we as a society need to look at is number one, that's effing happening. Number two, what does that mean for the future of the human condition? To your point, it's not just privacy of the data for the Google searches for my seven-year-old. It's not just that. It's what is my human experience built like? Does Meta get to place ads in front of me while I'm, I don't know, like, no matter where I am? Like, is my behavior gonna be bent ever so slightly to lean towards Microsoft products versus someone else's products when I'm searching for things and I'm asking my agent to give me answers of what softwares to pick or whatever? How are the incentives, let's go back there, of these big orgs going to manifest versus how much of that should be manifest of my own intentions as a user? It's an abstract conversation, but I will say, given where we're going, and I don't think there's any way to go back unless nuclear winter happens, given where we're going, we need to have that conversation as a society. And there's a very small handful of people doing it.

(Daniel Fagella at 00:42:15) So I think it's cool that you're on that page, but I literally couldn't tell you how much I agree with you.

(Joel Beasley at 00:42:20) Yeah. We need secure compute, and I won't get into the details of, like, GPUs. And Apple just released something yesterday. We need model validation so I know if I'm running Llama 2, it's running Llama 2 without nonsense on top of it, whatever you consider that nonsense to be.

(Joel Beasley at 00:42:34) And then we need control over the pixels that are on our screens because we need to set that precedent now with the screens and the pixels, because soon it's going to be Neuralinks. And whatever we accept now. And then part of me is like, maybe I should go on this crusade for "my body's my pixels." And then part of me is like, well, maybe people like Elon Musk, when they're building Neuralink, they're just not going to let this crap happen from the beginning. So it's like, do you try to fix the broken system, or is the new system coming fast enough and will it be built in? And so that's kind of where I get stuck, essentially, if I wanted to make that some higher level mission.

(Daniel Fagella at 00:43:06) I'm a fan of Musk, all things considered. And it is dishonest to not be impressed with what the man has achieved. It's dishonest. It's frankly dishonest. Right?

(Daniel Fagella at 00:43:15) Say what you will of Bonaparte, you can't erase him from Botanica. Shut your mouth. Right? So Musk is, you know, very strange to say, Musk is sort of of that grandeur and caliber, and I think there's a lot of admirable things about us.

(Daniel Fagella at 00:43:28) We should expect him to do what behooves his interest as well. So right now, it behooves him to say, "Hey, I'm building the Gen AI with Twitter and my X thing, but it's going to be truthful and it's going to be for the people." Joel, do you remember anybody else that said that when their company got started?

(Joel Beasley at 00:43:44) OpenAI.

(Daniel Fagella at 00:43:46) Yeah, Joel. So, Joel, do you know how Napoleon took the throne by force? Do you know how Robespierre took the throne? Robespierre took the throne by being the virtuous good little guy.

(Daniel Fagella at 00:43:56) But once you get your hand on the scepter, man, you want to start lopping heads off, brother. Doesn't matter if you're Bonaparte or Robespierre. Incentives are incentives. So I think Musk is a great fella and, you know, there's certainly worse people than him, I think, in the world. No doubt, and an incredible innovator. But I think we should plan on the structuring of incentives to be what balance him to prevent what you called nonsense from happening, not the magical virtue that floats in his brain.

(Daniel Fagella at 00:44:20) I think we should expect from each individual and organization precisely what is in their self interest. And then as a society, we must bound the rules in which to play. So all the faith in the world from us, but by golly, brother, I'm not sure if it's going to be his job to get this done because the promises he's making now, who knows what happens if he gets his hand on the scepter as well. Right? I mean, this is how power operates.

(Joel Beasley at 00:44:43) I 100% agree. Yeah. And I think it would be useful if as a people, one of the main driving factors of us liking him is the open and transparentness of it, because then his incentive is like, "Oh, people like me because I'm creating these infrastructures that are secure, private, open, and transparent." And I'm like, when he made all the Twitter stuff public, right?

(Joel Beasley at 00:45:08) Like, the act of opening that up. Totally. All of Apple's press release yesterday. I read their security paper because I was working on GPU model validation recently. And so I read their security paper and I was like, and then you get to the last line of this like twenty-minute reading. It's like, "And soon we'll open it up for security researchers to take a look at." I'm like, okay, it's not even peer reviewed yet. It's just like internal stuff and it's launching. So...

(Daniel Fagella at 00:45:33) I'm with you. I think these topics are incredibly important. By the way, Joel, I know we've got to wrap here. I'm zipping into the next chat. But it's been a real pleasure.

(Daniel Fagella at 00:45:40) Your tangents are vastly beyond what I initially saw in the handful of episodes I got to flick through. And so this has been a real blast. Let me know when this is going to be out. I'll make sure we promote it to our audiences as well because I think there's going to be some crossover.

(Daniel Fagella at 00:45:55) And if in the future you do any wacky wild series around sort of where this stuff is headed, there could be some fun opportunities to help out in that regard. So let's be sure to stay in touch and really glad we got to talk today. Okay?

(Joel Beasley at 00:46:06) Thank you so much for listening. And if you found this episode useful, please share it with a friend or colleague who you think would get value from it. And if you have topics that you would like to hear discussed on the podcast, either add me on LinkedIn or send me an email, [email protected]. Every time I get an email or LinkedIn message, it absolutely makes my day and inspires me to keep going.