The Red Queen Problem with Anthony Scriffignano, PhD, Data Scientist 

(Anthony Scriffignano at 00:00:00) Sooner or later, we discovered that the weakest link in technology was what they call the chair-keyboard interface. You are the chair-keyboard interface. The Red Queen says, "Well, that's the kind of place this is. You have to run as fast as you can just to stay where you are." If you have a disruption and you try to hide the disruption from the outside world, the hiding will be observable.

(Anthony Scriffignano at 00:00:22) A lot of times when something absolutely doesn't make sense—

(Joel Beasley at 00:00:26) Mm-hmm.

(Anthony Scriffignano at 00:00:26) I will lead with "Help me understand." I will not lead with "This doesn't make any sense."

(Joel Beasley at 00:00:38) Last time we talked to you at Dun & Bradstreet.

(Anthony Scriffignano at 00:00:40) Yep.

(Joel Beasley at 00:00:40) And very smart. When your name came across—yeah, I've done a thousand of these interviews. When your name came across, I was like, I don't remember what we talked about, but I remember that guy is brilliant.

(Anthony Scriffignano at 00:00:52) And then you called me, right?

(Joel Beasley at 00:00:54) Yeah. And then we went down the next name on the list, and we're like, let's call that guy. So what happened? Where'd you get—you finished up. You were doing some work there. You were there for a while now. Where are you? Catch me up on what's going on with Anthony.

(Anthony Scriffignano at 00:01:05) I was there over 20 years. I don't know exactly how many years, but many, many, many years. Lots and lots of contribution, many, many patents, inventions, all kinds of working with regulators all over the world. Nobody deserves to get to do all the things I got to do and work with the people I got to work with there. At some point, you know, you've got to pick your time or your time will pick you. Right? So I decided to retire from that. And sort of coincident with that, some folks that I know got me involved in a space-related venture that I'm not really at liberty to talk too much about, but it has to do with hyperspectral imaging and something called phenomenology, which is sort of sense-making out of what you can observe from space and combining that with terrestrial corpora. So it turns out that a lot of the things that I invented there translate well to that environment of making sense out of things when you have no knowledge to base it on. You can't LLM your way into what's going on because you don't have any training data, right? You see what you see, or, you know, radiation. Think about what's going on in Colombia right now, right? So all kinds of examples like that.

(Anthony Scriffignano at 00:02:21) Disruption, disrupted, disruption. That's the world I've been living in anyway. And then I'm also doing some work with the Stimson Center, which is a Washington, D.C. nonpartisan think tank. So I failed at retirement.

(Joel Beasley at 00:02:37) You failed at—so you retired, but now you've got a bunch of projects, and you're essentially working.

(Anthony Scriffignano at 00:02:43) Yeah. And then I'm sitting on a couple boards because, you know, the phone rings. And when you don't have the kinds of stipulations that you have when you're in a full-time executive role—you know, a lot of things you can't do because of NDA and all that—you can start to do some of that.

(Joel Beasley at 00:03:04) That's exciting. So you're happy, though. You're happy with what's going on.

(Anthony Scriffignano at 00:03:07) Absolutely. Yeah.

(Joel Beasley at 00:03:09) Is the family good?

(Anthony Scriffignano at 00:03:10) Yeah. Oh, everything's good. Thank God. Yeah.

(Joel Beasley at 00:03:13) Where are you primarily hanging out these days?

(Anthony Scriffignano at 00:03:15) So I live in New Jersey, and I work a lot in Washington, D.C. So I'm back and forth, up and down what they call the Northeast Corridor to Trenton.

(Joel Beasley at 00:03:24) Yeah. I've got a client in D.C. I'm there every two months. But, yeah, I don't envy New Jersey, though. I went there maybe three or four months ago, and I rented a car, which was a mistake, because I had zero experience driving in New Jersey. And by the time I got out of that airport, I was like—it took me a day. It took me a day, and then I was driving like a person from New Jersey. But you realize red lights are kinda suggestions.

(Anthony Scriffignano at 00:03:53) You know, so the part of New Jersey that you were in is just insanely dense and chaotic. If you go down south to the Pine Barrens, if you go out east to the shore, if you go up in the northwest to what they call Sussex County, the farmlands and mountains and the Delaware River Valley is just spectacular. Tiny little state. There is so much variation in New Jersey. It's amazing. But, yeah, I mean, the part that I live in is the part you were driving around, except in the suburbs of it. I don't know if you've watched The Sopranos, but Tony Soprano's house is walking distance from where I live. So—

(Joel Beasley at 00:04:36) That's pretty cool. That's pretty cool. So for the main topic of today's episode, I have here listed in the notes that we want to talk about what CTOs are getting wrong about building in the age of agentic AI. Is that still a topic you want to primarily discuss?

(Anthony Scriffignano at 00:04:52) That's fine. Okay. We were talking a lot about first principles, like what do you have to believe, what are some of the things that are changing in the environment. A lot of what's going on right now is like the lobster in the pot—hard to notice the change when you're part of it. And so the CTOs that I talk to and CIOs that I talk to, a lot of them are struggling with there's some kind of a mandate—get more AI, you know, transform the workforce, something like that. And that's great, but how would you know when you were done and what are you working on? Your goal isn't more AI. Your goal is more AI to do something. So that's been the conversation that we were having, but we can go anywhere you want, really.

(Joel Beasley at 00:05:41) And these mandates are typically coming from the board or the CEO or—

(Anthony Scriffignano at 00:05:45) Well, you know, if you look at any organization, you have the typical constituencies are your board, your employees, your shareholders, your customers, and it's some combination of all of those, right? So your newer employees, especially if you're in a CTO role—you know, the folks that you're hiring want to do the cool stuff, right? If you don't give them enough cool stuff to do, the best ones are going to leave before you can say "transformer," right? But your job isn't to just give them cool things to play with, right? You've got shadow AI. You've got the guy in the sales force that's writing his own website. You've got a lot of challenges that you didn't have a few years ago. People are vibe-coding their way into disaster, you know, and proud of it, right? And then you've got these big pushes to reduce tech debt and deal with data regulations and all the things that you had to do yesterday, you still have to do today. So these are what we call Red Queen problems. You can't get out of the problem by just doing what you were doing faster and harder. You've got to do something orthogonal, but you can't stop doing what you were doing yesterday at the same time. It's a big, big challenge. It's a tough job right now.

(Joel Beasley at 00:07:07) So you can't solve it by doing—I haven't heard of this Red Queen problem, can you—

(Anthony Scriffignano at 00:07:11) So it comes from Alice in Wonderland. So when Alice is, you know, down the rabbit hole and eventually she winds up at a tea party with the Mad Hatter and the Red Queen, she says something like, "You know, this is the strangest place. I've been running and running as fast as I can, and I don't seem to be getting anywhere." And the Red Queen says, "Well, that's the kind of place this is. You have to run as fast as you can just to stay where you are." So a Red Queen problem is when—and it's used a lot, it's a computer science or a data science term—when you're in a Red Queen problem, you can't stop doing what you're doing, and yet there's no credible path to success without doing something different. So you have to find a way to do both at the same time. If you think about—I mean, you know, I don't know, I'll speak for myself. You know, when I was growing up, there was always movies with people getting trapped in quicksand. Apparently, it's not the thing that it is in the movies, but imagine that it is, right? So you're in quicksand and you're slowly sinking. You know if you keep struggling, you're just going to sink into the quicksand, at least in the movie version of it. You can't stop because you'll probably sink faster, but you can't swim in quicksand. Sooner or later, somebody comes along and throws you a rope or gives you another way to get out or you figure out how to lay on top of it or you do something different and you can get out, but you can't stop doing what you're doing while you're coming up with that other solution. Those are Red Queen problems. And that's exactly what AI is presenting to the technology part of an organization right now. You can't just stop doing the basic blocking and tackling of, you know, cybersecurity and database management and access controls and all of those other things that were really important yesterday are probably more important today, and they'll certainly be more important tomorrow. If you just do those, you'll be irrelevant, right? You've got to figure out what are you going to do about shadow AI? What are you going to do about agentic ecosystems where you didn't write all the code? What are you going to do about intellectual property rights, patents? Do they even mean what they used to mean? What are you going to do about the regulatory environment that seems to change every time you breathe in and breathe out? There's some new regulation. If you're a global enterprise, good luck right now. You've got to deal with all that too. So it's a classic problem. You've got to figure out a way to—you know, I hate to say "work smarter" because nobody ever wants to hear that, but you've got to do a lot of triage these days.

(Joel Beasley at 00:09:54) I like being told to work smarter.

(Anthony Scriffignano at 00:09:56) Yeah. I don't like being told to work smarter because it implies that I'm working stupid, but I get it.

(Joel Beasley at 00:10:01) I'm okay with being stupid.

(Anthony Scriffignano at 00:10:03) Well, you know, there's that too. You're never the smartest person in the room, right? So one of the biggest pieces of advice I ever give anybody is you've got to be humble. If you think you're going to solve problems like this without getting an outside-in opinion on something, you're just living in an echo chamber. There's no way anybody is smart enough to deal with all of these challenges on their own. You've got to get outside-in opinions, thought, maybe best practice. Although, I'll put "best practice" in quotes because that implies that there is an emergent best practice. Sometimes it's just—you know, there's people that run around saying, "We've got to fail faster." And I would say, that's true, but you should fail faster differently, right? If you're just making the same mistakes over and over again faster, that's ridiculous, right? So you have to—there has to be institutional learning. There has to be time for this sort of—I don't like the term "postmortem" because it implies that something died, but an after-action study of something that you did, whether it was successful or not, to say, "What did we learn from this? What's transferable?" Different ways to do that. There's the people that want to say what went well and what didn't go well. That's great. You'll spend all your time talking about what didn't go well, right? We all know. It's going to start with somebody saying something that went well. You know? "Oh, well, I think we did a great job on the communicating with the customers when the system went down," something like that, right? But, you know, there's also that do more, do less, keep doing. If you kind of intersect those two, one is focused on inputs and the other one's focused on outputs, something like that. We've got to get better at documenting what we learned, or we're just going to fail faster the same way.

(Joel Beasley at 00:11:55) You don't want—that's the Red Queen problem. You're just running in place.

(Anthony Scriffignano at 00:11:58) Well, yeah. I guess they're cousins of each other, right? If you just watched somebody drown in quicksand and you said, "Let's fail fast. Let's jump in." Maybe—what could we learn from—oh, that guy just died.

(Joel Beasley at 00:12:13) Yes. See? They're cousins. It's close.

(Anthony Scriffignano at 00:12:16) Or, you know, maybe better yet, when you're in the quicksand, somebody comes along and says, "I survived quicksand. Let me tell you what to do." Right? That kind of thing.

(Joel Beasley at 00:12:23) Yeah. I'm not a super smart guy. I just look at myself as a dumb guy who gets a little bit smarter every day. Right?

(Anthony Scriffignano at 00:12:31) You know, I work with a lot of ridiculously smart people, and some of them accidentally wear two shoes that are different colors. Some of them, you know, forget to log out for a week. You know, some of them define smart, right? There's people that are very knowledgeable. You know, I have absolutely no problem explaining, you know, integral calculus, but maybe I don't pick up on the nuance at the networking event that somebody standing next to me did. So everybody has their different skills. It's a salad bar, right? You've got to have enough stuff on the salad bar and enough people that are experts in all of those different ingredients to make your salad. You don't try to eat everything on the salad bar. You don't try to do everything. It's not going to work.

(Joel Beasley at 00:13:23) That's right. And if the cucumber tries to be a crouton, it's like, you need to just focus on being the best cucumber you can be.

(Anthony Scriffignano at 00:13:29) Yeah. Or, you know, go find some brine and become a pickle.

(Joel Beasley at 00:13:36) That's funny. Oh, man. That's good. I'm having a good day. You having a good day?

(Anthony Scriffignano at 00:13:44) Yeah. I started my day with a bunch of surprises. This is the high point, I promise.

(Joel Beasley at 00:13:51) All right. All right. Well, okay. I want to get your thoughts on agentic AI because everyone's talking about it. You and I—I don't want to necessarily talk about it at work first. At first, I want to talk about it in your personal life. Are you using LLMs? Like, if you want to have a conversation with your wife or something like that, are you running it by Gemini? Are you getting input? Or how are you using it in your personal life?

(Anthony Scriffignano at 00:14:18) So I want to do a little bit of defining of terms here first so that we don't get ourselves in trouble. You started asking me about agentic, and now you're asking me about LLMs, which are related.

(Joel Beasley at 00:14:33) But, I know.

(Anthony Scriffignano at 00:14:34) You know? So I'm going to answer the second question first and the first question second, right?

(Joel Beasley at 00:14:38) That's good.

(Anthony Scriffignano at 00:14:38) So I use LLMs all the time. I don't use them to do my homework. I don't use them to write for me. I don't use them to fake my way through something. But if I'm going to go into a regulatory discussion, how could I possibly know what regulation might have passed yesterday somewhere on Earth, right? So do I use the—I have a sort of a suite of LLMs that I use for different things. This one's very good for RAG, you know, where I have a whole bunch of documents and I want to kind of put them there and do research that starts in that place. This one's very good for finding random, you know, cognates. Like, I think I know this thing, but I don't know where I know it from. Can you find me some citations or some credible sources? There's another one that's very good for creating visuals. Yeah, of course I have my suite of favorites, but I don't use them to substitute for doing my work. I use them to augment the work I'm doing. I'll use that regulatory example.

(Anthony Scriffignano at 00:15:51) I was getting ready for a discussion with someone recently, and I asked for, in the context of a regulatory expert—so I made the LLM an expert on global regulation on AI—I asked it what it thought the recent evolutions were, and it completely missed some things that were going on in Asia because they're just not written about in the press. And so I said, "Well, what about this?" "Oh, you're right. You're so smart. I forgot about that." And, you know, now I'm the smartest guy in the room to that LLM, right? I realized I just trained it a little bit.

(Anthony Scriffignano at 00:16:28) But I also, by pointing in that direction, found out about a small country that did something that I didn't know about. So it's a dance. The agentic stuff—I'm sort of, as a human being, could I set up a bunch of agents that look through my email and find invitations to things and find things that appear to be people that I've interacted with that I haven't responded to and sort of keep me more honest? Because I'm horrible with email, right? I tell people all the time, if you want to keep a secret from me, send it to me in an email, right? I just get way too much email.

(Anthony Scriffignano at 00:17:08) Could I use some sort of agentic workflow to get better at that? Probably. Do I want to give up my credentials and all of the knowledge that can be gleaned from my email to go into some who knows where, what? No, I don't want to do that. So I don't do that. I'm a little bit leery of—you can't unring that bell once you start giving these agents access to your calendar and your email and your other stuff, the data that's on your desktop. And I'm not going to do that. I know people that say, "I don't care. I'm not that interesting. Have at it. Just help me, you know, manage. I got, you know, there's three kids and there's a family calendar, and it's overwhelming. Do it for me." Well, okay, fine. Have a nice day. I don't judge them, but I don't do that.

(Joel Beasley at 00:18:02) Do you use it—I asked you about using it in your personal life, and then you gave me a bunch of work examples. Do you—

(Anthony Scriffignano at 00:18:09) That is my—because, you know, email and calendars and all of that is me getting through my personal life to some extent. But if you mean, like, literally, if I'm having an argument with somebody, do I go run to an LLM? No. I said no, knowing that it's not entirely true. Do I occasionally, on my phone, look something up because I kind of think I'm right, and I want to make sure before I double down? Occasionally. But it's not my go-to. It's really—and I think when I do it, I'm pretty transparent about it. Like, I think the person that I'm talking to knows I'm doing it.

(Joel Beasley at 00:18:53) He's doing it right now.

(Anthony Scriffignano at 00:18:55) No, I'm not. I'm literally sitting on my hands. I don't know why I feel comfortable sitting like this, but—

(Joel Beasley at 00:19:02) Feel free. Do what you want to do.

(Anthony Scriffignano at 00:19:03) It's the whole Anchorman thing. If you pull your jacket down and you sit on your hands, it looks—

(Joel Beasley at 00:19:08) You feel—

(Anthony Scriffignano at 00:19:10) Yeah. You feel better. Yeah. Anyway, the—what was it? Oh, so, you know, we're all walking around with these magic boxes in our pockets that give us access to all knowledge, we think—all knowledge known to mankind. The reality is all curated knowledge, which is pretty limited. Search engines have access to less than 10% of the information that's on the Internet because a lot of it is behind a firewall or requires a password, or it's only constructed when you go there, so it's not—you know, there's a latency to it. Our perception that LLMs are looking at everything is way off. And then you think about the amount of data that's being created that never makes it into a URL—the data that's being created by IoT devices and the data that's being created by protected work product or creative things that people are doing. None of that, or a lot of that, isn't getting consumed. So now if I ask a question about—imagine what's going on right now. I don't want to get political, but pick your political debacle of choice right now and imagine what's going on in that scenario. And then, do you honestly believe that all the truth about that is in the news? Of course not. No. Nobody believes that.

(Anthony Scriffignano at 00:20:38) Well, I talk a lot about veracity, like truthiness. I've actually developed some intellectual property on veracity adjudication, and I based it on the oath. You know, you swear to tell the truth, the whole truth, and nothing but the truth. That's three different frames of lying, right? I can tell you an untruth. I can tell you a bunch of truth that allows you to reach the wrong conclusion—mislead you—or I can sort of sneak my lie in with a bunch of otherwise truthful things. So do LLMs adjudicate truth? No. They kind of consume data as if it's all true. And the argument is that if you consume enough data, the signal will overwhelm the lies that are in there. I don't know if I believe that.

(Joel Beasley at 00:21:31) Mm-hmm.

(Anthony Scriffignano at 00:21:31) Yeah. And, you know, when it happens, we call it hallucination, right? We have a name for it. You know, we have to be very careful that something happens often enough that there's actually a name for it. You might want to think about that depending on what you're doing with your LLM. I'm not anti-AI. I'm not anti-anything. I'm kind of pro-critical thinking.

(Joel Beasley at 00:21:55) So you must like Grok as an LLM. I have found that one to be—none of them are perfect. They're all different strengths, as you mentioned, the salad bar. But when I have something that I want to think through deeply, philosophical, or that I want the most amount of truth and the least amount of political correctness, I will go to Grok. What about you?

(Anthony Scriffignano at 00:22:18) Well, you know, this is a little bit like, "What's your favorite band?" right? When you say it, you know, everybody piles on. Like, they judge you. I—everyone is beautiful in its own way. I try to be—seriously, I try to be an equal opportunity offender with all of them, except for one. The one I probably go to first most—and it's usually the one you learned first—is Gemini. And the reason I like Gemini, one of the reasons I like it, is the notebooks. And I know you can do that in other platforms as well, but that's where I do it, right? One of the reasons I like it as well is because I've used it more. I kind of know when it's faking. I know when it's doing the best it can and it doesn't really know what it's talking about.

(Anthony Scriffignano at 00:23:21) I also tend to, if I'm doing anything that's really critical, I tend to triangulate. I tend to ask more than one platform. It's all about the prompt engineering. If you're literally—let's assume you're not starting from a bunch of documents. It's all about how you position the prompt and how you constrain it, as well as how you contextualize it. So I'm very cautious about contextualizing who I'm talking to, what I'm asking for, and what swim lanes I don't want to cross in getting my answer. I also very often use AI like this, and I give it my thoughts, and I give it the reasons why, and I ask, in the context of some sort of an expert, what I'm missing or what the biggest arguments are against the position that I'm taking. And if I've anticipated those arguments or if I've considered those missing points, then I'm good. If I haven't, then I got my first beating before I talk to the person that I'm actually going to talk to.

(Joel Beasley at 00:24:33) That's how I'm using it a lot as well. The blind spot detection, beating up ideas. "Why am I an idiot? I think this might be possible. Am I completely out there, or is there other people who are researching this that think I'm—" yeah.

(Anthony Scriffignano at 00:24:47) I don't say "Why am I an idiot?" I say, "What am I missing?" I say, I—

(Joel Beasley at 00:24:51) I don't say that. I know.

(Anthony Scriffignano at 00:24:53) Yeah. You know, it's like, "The dress doesn't make you look fat. Your hips make you look fat," like that. That's a commercial. I didn't say that to anyone. But the reality is that a lot of times when you ask that disruptive question—"What am I missing?" or "What are the critical arguments against this position that I'm taking?"—if the critical arguments against it are lame, that's a really good sign that you're making a strong argument. There's a thing in epistemology about proving something's wrong versus trying to prove that it's right. Very often it's impossible to prove that something non-dispositive is right. And if you can fail to prove that it's wrong, you can get a strong signal that you're doing the right thing. You know, it's that contrapositive kind of argument. LLMs are great for that.

(Anthony Scriffignano at 00:25:49) And also it depends on what you're talking about. If you're talking about something that happened yesterday and never happened before, don't go to an LLM. Like, what are you doing? You know, if aliens came down and landed in Area 54 tomorrow and we have video on the 6:00 news, I'm not going to go ask any of these platforms what they think about it. I need to go talk to somebody, right? Because there's not enough out there that's been sucked in, other than all of the hyperbole that's people writing about it.

(Joel Beasley at 00:26:22) Yeah. I love that: "The dress doesn't make you look fat. Your hips." I've never heard that before.

(Anthony Scriffignano at 00:26:27) I think it was MCI "Speak Freely on Weekends," and people were misunderstanding that they could say whatever they wanted on weekends.

(Joel Beasley at 00:26:39) What are the big mistakes that you're seeing the CTOs make today?

(Anthony Scriffignano at 00:26:42) Well, so I have to give props to the CTOs today because it's a really tough job right now, because it's nothing like what it was a while ago, and you still have to do what you had to do a while ago. The democratization of AI—the fact that everyone can use it—is being handled better or worse. And in some places, there's a—I've seen everything from a laissez-faire attitude: "Well, if they build it, then they're going to have to support it." And, you know, it's still in your enterprise, and sooner or later, somebody's going to ask you about it. I've seen, on the other end of that same spectrum, "Let me build a sandbox, let me put everything in it, and let them at least come to my sandbox and play so that I can watch what they're doing and keep them from doing something colossally stupid." I kind of like that approach a little bit better, but you have to manage the sandbox. I have seen organizations try to just lock it down and say, "You can only use this platform and you can only use this thing." And, you know, we all know that people are just going to take out their phones and do it anyway. So who are you kidding, right? You're just forcing people off of your enterprise platform onto something we have even less control over. So I think the democratized AI—the world hasn't come up with a good answer to that one yet. Kind of like when PowerPoint first came out, you know, people were doing ridiculously stupid things in PowerPoint because nobody understood how to formulate a message or how to pick fonts or anything. And you had the old joke, "Death by PowerPoint." I won't tell you the joke, but I'm sure you know it. It's horrible things that people were doing. Well, multiply that by 10 million, and you get what's going on with vibe coding right now. So democratized AI—absolutely, I don't think there's a best practice emerging yet.

(Anthony Scriffignano at 00:28:38) Another one is the regulatory environment. There's a lot of wishing and hoping going on and not necessarily a lot of attention paying to how the regulatory environment is changing, especially in U.S. companies, because a lot of these regulations are not U.S. regulations, but a lot of them have extraterritorial components that affect U.S. companies doing business either in those other countries or when their citizens are in this country. So we absolutely have to pay attention to them. And I don't, in my humble opinion, think there's enough attention being paid. And some of the argument is, "Well, you know, other people are going to get in trouble first because they're probably going to be more egregious in—" not necessarily. And then the third area, because there should always be three, is that the unintended use of AI to do nefarious things is growing quite significantly right now. The bad guys don't worry about regulation. I couldn't care less about shadow AI. They don't have to worry about their shareholders, right? And they can create botnet swarms and polymorphic code and all kinds of things without even lifting a finger now, because there's stuff that will do that for them. And so, you know, the environment that you think is protecting you is still protecting you, but it's not protecting you from the hurt that is coming tomorrow. We have to almost operate from the presumption that all of our environments have been infiltrated in some way, in some latent way, either through malware or through some artificially manipulated data. And it's a question of mitigating the harm, kind of like what your immune system does, right?

(Joel Beasley at 00:30:25) But doesn't government regulation usually follow a long history? Like, they're not very proactive, right? Like, a lot of the kids, they have to die before they make the car seats a thing or before they force the seat belt, before that becomes a regulated thing. So—

(Anthony Scriffignano at 00:30:44) Yes and no. So that would normally be true. But now there's a lot of places in the world where they're being called on the carpet because they don't have any AI regulation, or they don't have any data protections. And so somebody in the regulatory environment gets the charge to issue regulation. And they're like, "What? About what? Well, AI." You know, and it's so broad. And so they go and they look at other countries and they look at their AI regulation, and they template something based on that. And they usually tweak it and make it a little bit more strict in some way that's relative to that part of the world. And then they start with that, and they take this attitude of, you know, "We'll lead with this so that we can't get hit with the 'You don't have any AI regulation' hammer. And then if it turns out that this is overly restrictive or has some unintended consequence, we'll tweak it later." The problem with that is when you have six or seven different parts of the world—and it's more like 20 or 30 right now, probably closer to 50—how do you operate as a global enterprise? Do you adopt the strictest regulation anywhere and do it everywhere? Do you try to create some sort of matrixed environment where you flex what you're doing based on where you're doing it? Do you create some sort of oversight role? Will you hire an outside agency to kind of come in and audit you and make recommendations so you can blame them when you're wrong? Those are some of the strategies that are applied, and none of those are perfect.

(Joel Beasley at 00:32:25) So you just have to pick which one you want to roll with.

(Anthony Scriffignano at 00:32:28) I would say be anywhere in that continuum, but be there on purpose.

(Joel Beasley at 00:32:33) What sort of AI regulations exist today? Because I'm under the assumption I can do almost anything I want.

(Anthony Scriffignano at 00:32:38) You're wrong.

(Joel Beasley at 00:32:40) I'm sure. I'm sure.

(Anthony Scriffignano at 00:32:42) So, you know, without getting into a whole—there's lots of legal implications to me answering that question to you and you publishing it. But in some parts of the world, they value national security more than they value personal privacy. So in those parts of the world, companies that are using AI are required to share the data with the government. In other parts of the world, you have either a right to be forgotten or a right to control your own information. So in those parts of the world, there are strict requirements placed on stewards of data, collectors of data, and those who use data in their products and services.

(Anthony Scriffignano at 00:33:29) Many parts of the world now have a sort of a tiered system that, depending on the application of the AI, if it's being used to directly affect humans, if it's being used in an automation setting, if it's being used in a credit decision, there's tiers. And depending on what is more harmful, there are more controls at the higher levels of harm, potential harm, and fewer controls at the lower levels of potential harm. So if you're selling shoes and socks, you're probably good. If you're selling medical shoes and socks, you probably have more controls on you. And then in other parts of the world, and the United States is one of those parts of the world, you know, it's the United States, right?

(Anthony Scriffignano at 00:34:16) So there's federal law and then there's state law. And sometimes you have a tapestry where at the federal level you just have guidelines and recommendations and first principles and whatever they want to call them. I'm not trivializing them. There's a lot of hard work being done there. But it's not a law per se.

(Anthony Scriffignano at 00:34:40) And then you have the California Privacy Act, and you have this other act, and you have the, you know, and so depending on what state you're in, you might have an issue. It's very important to get a professional opinion on the permissibility of what you're doing with your AI and what you're doing with your data. And those are two different things, and you really need to pay attention to it these days. You can't just hope it's okay.

(Joel Beasley at 00:35:07) Do we in the United States have AI regulation?

(Anthony Scriffignano at 00:35:12) AI policy right now, and we have data regulation in some states.

(Joel Beasley at 00:35:16) We have data regulation in some states?

(Anthony Scriffignano at 00:35:18) Lawyer. That's what I think the answer is. Yeah.

(Joel Beasley at 00:35:21) We're all hanging out. This is not—yeah. Yeah. That's interesting. Because, you know, from what I've observed as just a casual passerby, it seems like Europe is pretty forward on this. When I see regulation stuff happen, I usually see it start over there.

(Anthony Scriffignano at 00:35:41) There's a lot in Europe. A lot. A lot. And then there's quite a bit in China. But then there's other parts of the world. It's not just those two places.

(Joel Beasley at 00:35:52) Yeah. There's a lot of countries.

(Anthony Scriffignano at 00:35:54) Yeah. I mean, I had a list that I prepared for somebody recently of all the places in the world where there's an actual AI regulation, and it was quite a number of them. The people who are good at this, the people who, I don't want to start naming names right now for your audience, but I can give you offline some people to follow that will really educate you, as they say in Jersey City, on how this is changing. And you really need to pay attention to somebody who does this every single day because literally something's changing almost every day.

(Joel Beasley at 00:36:33) But then it kind of becomes—so that's why it's so important for the companies to get a professional to go look at what's going on in their narrow space at this point in time.

(Anthony Scriffignano at 00:36:44) Yeah. And don't just go ask Annie in the legal department because if Annie is not an expert in AI regulation and data regulation, Annie is going to do what I just did, except she's got a law degree. Right?

(Joel Beasley at 00:36:56) No, I'm just going to tell Gemini to act like a—yeah. Don't do that. Don't do that.

(Anthony Scriffignano at 00:37:00) Don't do that. You know, yeah, well, that's another tricky problem, right? So if you think about—I'm not going to spend a lot of time talking about regulation, but if you think about the way a lot of these regulations, there's two ways that the regulations are written. One is they tell you what you can't do, and the other is they can tell you you can do anything except these things. Right? So one assumes all the rights and then takes them away, and the other one gives you the rights or tells you the wrongs. The problem is when you're operating in an environment where those two collide, it creates this dilemma where you almost can't do anything without bumping into one of those walls. And it doesn't mean, therefore, don't do anything. It means, again, have a documented policy and an understanding. So it's easier to talk about it with data than it is with AI, right? I talk about the Ps: provenance, permissible use, permutation. How is the data changing? Where did you get it? By what right do you know that you're allowed to use it? You got to write that stuff down. And then if somebody comes out and challenges you later and says, well, you use this data, you're not supposed to use it. Well, here's why we think we are. Here's the contract we have. Here's the language that we have. Here's the permissions that we have. Here's why we think we're allowed to do it. And that's part of my curation process for any data is, where'd you get it? How do you know you're allowed to use it? How do you know it hasn't changed?

(Joel Beasley at 00:38:34) Yeah. You were doing—you're like Mr. Data over at Dun and Bradstreet when you were over there. Right?

(Anthony Scriffignano at 00:38:41) Yeah. I mean, you know, if you think about the challenge that they have—and I am not speaking for them—trying to collect data from almost all over the world, practically everywhere in the world. Different regulations, different languages, different writing systems, updated at different rates, you know, different times for different reasons. And people want a single global view of an enterprise. Well, technically, you know, I used to joke with people. I would say, you know, let's say you're the CEO of a big global company. You're a pretty important guy. Can you pick up that phone right now and tell me exactly how many employees you have? And the smart ones would kind of look at the phone and think about it for a second and say, no, because, you know, it's nighttime somewhere, and you don't know who just left and who just got hired and who had a baby and dah, dah, dah, dah. You don't know exactly because that number is—there's a latency in collecting that answer that exceeds the rate of change in that answer. So, you know, approximately, we have about 3,500 employees. Right? You're never going to get 3,547.

(Joel Beasley at 00:39:48) Because there's no—there's not a lot of value into having the exact—like, if there was monetary value, you would still—

(Anthony Scriffignano at 00:39:53) You know, somebody's going to say, well, I'll just go into SAP and do that. But yeah, fine. But, you know, somebody just quit, and they didn't put it in SAP yet. Right? You're never going to win. You're never going to win.

(Joel Beasley at 00:40:03) You're right. You're right. Yeah. That's fun. Well, we're talking a lot about, like, different things changing from regulations, businesses, and this new technology, which means, you know, a lot of the leaders, they're going to have to manage this change. What tips do you have for—you've been through many technological changes while you were in leadership positions. What have you learned from that?

(Anthony Scriffignano at 00:40:27) Well, I would kind of summarize, you know, when you're dealing with technology change, any leader who's worth their salt has at some level heard about people, process, technology. Those are generally the levers that you're pulling, right? Do I have the right people? Am I using the right process? What's the technology? There's a fourth one that I always try to add there, which is mindset, and it's often ignored. So let's say you're implementing ERP, or let's say you are putting in a new agentic ecosystem to handle your customer service, blah, blah, blah, blah, whatever, right? You're doing something that's a very big amount of change for your enterprise. You probably understand the people, you've probably built some kind of change management process, some kind of giant project plan with dependencies and all that. And you probably understand the technological underpinnings of what you're trying to do because you're a CIO or whatever you are, right? But the mindset usually starts with questions like, why are we doing this? How would we know when we're done? What are the indicators that we would get while we're doing it that the conditions have changed sufficiently that we need to reconsider what we're doing? Right? How do we understand the opportunity cost of doing this versus some of the other things that we might be doing? Those kinds of questions get at the mindset of, what do we have to believe in order to keep doing what we're doing tomorrow? And it's usually that person who asked that question gets shut down. Nobody wants to hear it.

(Joel Beasley at 00:42:05) I know.

(Anthony Scriffignano at 00:42:06) Too busy cutting down trees. Right? But you're in the wrong forest.

(Joel Beasley at 00:42:12) But we're cutting the trees faster.

(Anthony Scriffignano at 00:42:14) Busy, damn it. Right? Yeah. And, you know, the closer you get to the end date, the busier you get cutting down trees. And it's really important for somebody—I used to be a management consultant and we did big ERP transformations. And I used to, to people I knew well, I would give them copies of The Emperor's New Clothes. Right? It's a great story about, like, everybody's busy doing stuff and, you know, you're not thinking about the obvious thing in front of your face because you're too busy doing that thing. So that mindset will get you every time. If you don't stop and examine your first principles—why are we doing this? How would we know we were done? And how would we know if things have changed so that we shouldn't be doing this anymore and we should be doing something else? If you can't answer those questions, sooner or later, you're going to regret it. Unless you get lucky.

(Joel Beasley at 00:43:12) You just summed up, like, this is a good conversation for me because, I mean, for twenty years, those questions and pretty much only those questions were the reasons why people thought I was smart. But for me, it was just the natural, like, coming to, like, why are we doing this? And and explain to me, like, how did we get here and what are we doing and when will it be done? And and they're just like, shut up.

(Anthony Scriffignano at 00:43:37) A lot of times when something absolutely doesn't make sense, I will lead with, help me understand. I will not lead with, this doesn't make any sense. So, you know, what are you doing? Well, we're implementing AI. Okay. Great. What are you implementing AI for? Well, we're going to—we're going to completely reengineer our, uh, let me make something up, our distribution resource planning. What does that mean? Well, you know, the way we ship stuff. You know, we got—we're going to—we're going to take cost out of it. We're going to monetize the data and a lot of words. Right? I'm like, okay. That sounds great. Why? What do you mean why? Why are you doing this? Well, we're doing AI. Didn't I tell you that? Okay. So your goal is to do AI. And the reality is, when you can—if you can sort of humbly ask that question, that person you're talking to is getting a lot of pressure. Why haven't you put more AI into your system? Right? And maybe what they haven't thought about in the mindset is that they're automating a process that was never really designed to be automated. And it's managed by spreadsheets, and it's managed by the knowledge of those people in the shipping department knowing, don't do this, do that. And and you're not ready to automate this thing yet. And if you try to automate what you're doing today, you're just going to accelerate the speed that you hit the wall with, right? So, how do we know we're ready to do this, is the right question. Well, I don't know because somebody's yelling and screaming that they want to do it. Okay. That's not a reason. Right? Getting what you want all the time is for babies. We have to figure out what the preconditions are. One of the preconditions for—remember RPA? Everybody was doing RPA for a while. Right? And some companies, you know, automated stupid processes and went out of business because they could make mistakes faster. Right? So, you know, you've got to be able to ask that question. Help me understand why this makes sense. And if somebody can say, well, you know, we looked at our shipping department and we got 49 people doing exactly the same thing all day long. And those are really smart people. They're not doing smart things. They're doing stupid things because they need to get done. That's probably a good start. Right? Maybe you're ready. Maybe this is exactly the right thing to do. And now, why are you trying to AI your way into that? That's a different story. But, you know, leading with the tool is almost never the right answer.

(Joel Beasley at 00:46:06) Oh, yeah. You learn that you learn that in the first five years of your career. That's an early learning.

(Anthony Scriffignano at 00:46:11) People doing it every day all over the place.

(Joel Beasley at 00:46:14) Even after five years?

(Anthony Scriffignano at 00:46:16) Especially after LLMs became cool. Oh. Right? People were drunk on them. Like, we're going to win for everything.

(Joel Beasley at 00:46:23) I guess. I mean, look. My my experience—you've had a very different experience than me. I've only ever worked in, like, a startup that I started and then grew up to, like, 30 people. I just did that, like, several times. And so I don't have large corporation experience. I like to build things. I like to sell things. I like to solve new problems.

(Anthony Scriffignano at 00:46:44) I mean, the reason I'm giving you all these big company answers is because you're talking to CTOs and CIOs, and I assume that they're in bigger companies because of the title. Right? But if you look at the startup, there's a different kind of problem, which is the sort of hubris that comes with, look how much more I can do now. Right? I don't want to pick on a platform. I'm going to make up a platform. Right? There's no AI platform called Mod. Okay? I went to Mod and I created the code and, look, I have this website and, look, I created an API. I don't even know. I couldn't spell API yesterday and I created an API and now people can query our data and, you know, they can they can find out, they can look at our product catalog or whatever, right? That's great. Are you ready for when some agent tries to call that API a million times a second? Obviously, not a million times a second, but a million times a minute? What do you mean? Have you ever heard of a DDoS attack? What's that? Let me tell you how to spell something else because it's about to happen to you. Right? There's a reason why some of the drugs in the pharmacy are behind the counter and require a prescription, because you need an expert to help you use them right. And, you know, Mod is going to help you do things that maybe you shouldn't be doing. So, just because you can doesn't mean you should. And that's a lot of times in startups, they're like, you know, damn the torpedoes, full speed ahead, you know, hurry up, you know, let's get this thing up and running. So we have an MVP and then, you know, let's get the investors and and and, you know, let's hurry up and and and, you know, basically sell it before it falls apart. Right? I'm not accusing you of that, but there's a lot of that behavior, and it's pretty negligent. You got to be really—it's a lot easier to be negligent now than it was five years ago.

(Joel Beasley at 00:48:42) Well, we've increased the attack vector. So sure, like, before, you needed somebody like me with lots of experience to build a larger system or something that even looked like a larger system or that even functioned or smelled like a larger system. Now you can have the server at Chili's go home and start vibe coding and come up with something that looks 80% legitimate.

(Anthony Scriffignano at 00:49:03) Right. And and be handing out QR codes to their dining customers with the bill. Yeah. Right? On on a little card that they got, you know, at Jiffy Print. Right.

(Joel Beasley at 00:49:17) That's like the new—that's going to be like the new scammer. Oh, you know, I saw people doing that. You know, I don't want to mention the brand. There's a parking company. Yep.

(Joel Beasley at 00:49:24) Have you run across that one?

(Anthony Scriffignano at 00:49:27) No, but I know where you're going, and I can give you ten examples like this.

(Joel Beasley at 00:49:31) They put the stickers, you know, you go to the park, it's like you're in Zone 154.

(Anthony Scriffignano at 00:49:34) I do know about that one. It's my favorite example of that. And I'm not picking on this company because it was a brilliant ad, but a few years ago, there was a Super Bowl ad for a crypto company and it ended with a big QR code on the screen. It was basically the whole ad with this QR code.

(Joel Beasley at 00:49:49) That's right.

(Anthony Scriffignano at 00:49:50) Millions of people scanning a QR code, and they have no idea what they're doing. They have no idea where they're going. It turned out they weren't going any place bad, but what if they were, right?

(Joel Beasley at 00:50:03) The QR code.

(Anthony Scriffignano at 00:50:04) Yeah, yeah. So the weakest link in technology has, since at least the, I'm gonna say like the late seventies—so prior to that, the people that were using computers were just super uber nerds, right? But then after that, you could go in, like in the late seventies, you could go buy a computer in the store: TRS-80, Apple, whatever, somewhere in that time frame, right? Sooner or later, we discovered that the weakest link in technology was what they call the chair-keyboard interface.

(Joel Beasley at 00:50:43) Yeah.

(Anthony Scriffignano at 00:50:43) You are the chair-keyboard interface.

(Joel Beasley at 00:50:45) Oh, yeah.

(Anthony Scriffignano at 00:50:46) Stupid things that you will do—not you, but the stupid things we will all do. The spreadsheet runs much faster if I don't let it recalculate. It also doesn't work, right? It's amazing how we kid ourselves into convincing ourselves how smart we are. And AI is, to some extent, helping us do that now and actually saying in English how smart we are. And so these first principles of how would I know it's working, what do I have to believe in order to do this, how do I know the environment hasn't changed—those critical questions are what's gonna save us, not the here's the new transformer model, here's the new foundation approach, here's quantum AI. Like, all those are great. Please keep doing that. Maybe slow it down a little bit. But, you know, it's the chair-keyboard interface that I worry about the most right now.

(Joel Beasley at 00:51:45) Do you think, like right now, information spread pretty wide—books, different mediums, things like that. I think in a generation or two, the majority of it will be coming from the LLM type of technologies. But do you think that, or the intelligence is gonna get locked down?

(Anthony Scriffignano at 00:52:08) So I think I already told you earlier that to some extent it already is. When we are searching or using an LLM, we're only really looking at a very small amount of the data that's in the data sphere. Most of the data that's in the data sphere is sitting in your Fitbit. It's sitting behind your firewall. It's sitting in your encrypted database. It's somewhere that it's not being consumed. More and more now, Gen AI—we call it Gen AI because it can generate content as well as synthesize it—Gen AI is consuming data that was produced by other Gen AI. So we're reaching a moment in time where there is this echo chamber effect. Do we wanna call that locking down the knowledge? I would say no, because I'm still allowed to think a new thought right now. And if I don't put it on the interweb, it's my thought, right? There's a lot of people that would argue with me that none of us are thinking that creatively, and artificial general intelligence will eventually come along and make us—we'll be like the guys in WALL-E, you know, at the end with the little floating chairs, walking around.

(Joel Beasley at 00:53:32) No, I'm gonna stay fit the whole time.

(Anthony Scriffignano at 00:53:34) It's not gonna happen. I don't think it's gonna happen. And for a lot of reasons, I'm not in that boat, right? But there are people who think that. The two analogies I use: Star Trek, you know, they fly around the universe and they assume a lot of physics that doesn't exist yet and all that. Let that go, right? But they always have their computer, and they can always ask the computer a question, and it seems to know anything they need to know, and that's great. There's another version of the future, which is in The Jetsons, where he spends his entire day dealing with the not-quite-perfectly-working technology. He's got the bed that throws him out of bed and the maid that doesn't do what he wants and the car that doesn't go where he wants it to go. And he spends his entire day trying to deal with his technology. I think that we're getting to a point where we spend a lot of time dealing with our technology and serving it, if we tell the truth. You gotta upgrade the operating system on your phone. You've gotta reboot your router. You've gotta update your MFA, right? We spend a lot of time—the Bluetooth thing didn't work right, and I've gotta disconnect it and forget it and reattach it. And this is like Tuesday, right? This is everybody's day now, right? So are we gonna take this, or are we gonna assume it's just gonna get better? Do we honestly believe that it's getting better? I would say that I'm still hoping for the creativity of man, that there are things that are uniquely human that we will uniquely do. And I'm hopeful that the technology that we build will give us more time to do it by doing some of the stupid stuff for us, and I'm okay with that. I'm okay with both of those things can be true at the same time. Technology can get smarter, and we can do better things if we choose to. We choose to float around in floating chairs and drink mai tais, then we get what we deserve.

(Joel Beasley at 00:55:33) Yes. Like, if you look back, you know, ten thousand years or whatnot, the amount of—we had obviously the survival tasks that we had to do. But outside of that, the growing the food and the surviving, we had a large part of our day to ourselves. And but then it's like you don't really wanna go back there, because we ran that experiment in the Beasley family and we moved out to a farm in the middle of nowhere, and we did all of that. And what we realized really quickly is that, you know, without medicine and advanced stuff, like life was not good. Life was really not good. So I'm excited. I think we are on this journey where, like, we had a lot of time, we could be free, we could think, we had complete control over our schedules. We just had to do these few survival tasks and grow food and stuff. And but we didn't have all of this other stuff—modern medicine and the entertainment the way it is today, on demand, and the ability to communicate and travel and all of this. But we started to gain all those abilities, but we started to lose all of our free time for like the average person. I'm hoping it swings back to where the technology then becomes so good that we get both worlds. We get the world where we have ample free time to explore and do as the humans do, and we also have the advancements of a modern society.

(Anthony Scriffignano at 00:56:53) I would add something to what you're saying, which is that all of this is not even close to well-distributed on the planet Earth. So I'm not suggesting that we all serve one AI master or anything like that, but there are places in the world where you're born into a life of struggling to get food or water or personal safety from the turmoil that's around you. And I think that as we free up our time, if there are some people—if you gave them, if you let them win the lottery, they'll spend all their money on yachts and stuff till they have no more money. There's other people that will try to give it all away and do something truly noble and then have nothing again. There's somewhere in between there where if we can start to think about—AIDS, COVID was a great example. Whatever that was and whatever happened and whether it was man-made or what, I don't wanna get into it, right? But it started to happen in one part of the world. It didn't happen at the same time all over the world. It started to happen in one part of the world. That part of the world had to deal with it. The rest of the world was watching, and then it kinda moved and spread. And it hadn't finished in the first part when it hit the second part and the third part and the fourth part. So each part of the world that encountered COVID unartfully dealt with it, and the resources were—they were sort of unequally consumed by the parts of the world that already had them. So the respirators and the masks were in one part of the world, and then the next part of the world needed them, and that was crazy. The information was not shared well at all. So different parts of the world knew more than other parts of the world. They didn't always tell the whole story. I'm not sure we ever got the whole story. And then the response to it—in some parts of the world, you got oxygen. In other parts of the world, you got the latest vaccine and anything we could do to save your life. Did we learn anything from that? Did we learn how to share better? I don't think we did. So maybe AI being democratized helps to some extent, in that we can all have access—not necessarily in the next COVID, but maybe as quantum AI starts to happen or as the next big disruptive thing starts to happen, maybe we can use a little bit of that to solve some of these real serious problems that we all still face and not just use it to sell more shoes and more socks and beat the other guy to monetize the blah blah blah and, you know, grow our EBITDA and like, yeah, okay, fine. Do all that. But like, do something that might matter for the next generation. Because the next generation right now, I don't even know what they do. They don't do what you and I did. They do something different that we don't have a name for yet.

(Joel Beasley at 01:00:04) Well, we will eventually, but right now, it's a—

(Anthony Scriffignano at 01:00:08) Right now, they're struggling to do some version of what you and I did, because that other thing doesn't really exist yet. I think they call that other thing the gig economy, but you know—

(Joel Beasley at 01:00:18) Yeah. Well, I don't know what you're—so like, I've got a nine, a seven, and a four-year-old. So like, I know those age ranges. I don't know what's going on with the eighteens, the fifteens, and the twelves right now. But I know my kids, you know, we made intentional choices for our lives to give them—like, my dad told me when I was having my first child, I asked him about like how he thought about parenting. And he said he just tried to be slightly better than his dad. He's like, if I can be better than my dad, I'll move us forward one rung on the generational line. He goes, and if you do the same thing and then you teach your son to do the same thing, he goes, eventually it'll be this really good system. And I said, okay, I believe that. So my thing that I did for my kids that was better than what I had is time with them. So I intentionally make less money so I can be with them. We homeschool them so that they're with us, because I did not like school. And those are the things that we did intentionally to make it better.

(Anthony Scriffignano at 01:01:27) I applaud that. While you were telling me that story, I was thinking you have to kinda define better, though, right? So if I'm not—you know, bravo, right? I'm not suggesting I have a better answer. I have a lot of little ones at different ages in my life. And some of them, if you listen to them talk to each other, you have no idea what they're talking about. The language is—I don't know what language they speak, right? Our parents probably would have said the same thing about us, right? You listen to what they care about, and it's very different than what we cared about. I practically majored in video games when I was in college. Video games were these things that you went into an arcade and played. You look at a modern video game now, and that generation can just put the headset on and play the game. You know? And I'm like, well, what do you do? Like—so there's definitely skill sets that they have that we don't have because they're native to it. You hand them a phone. You don't have to explain how it works. You know? You hand them a device. They don't need it explained. That doesn't mean that they understand how to have a conversation like you're talking about spending more time with them. It doesn't mean they understand how to have an argument without having a fight. It doesn't mean they understand how to learn something that takes more than five seconds to learn, right? We could sit down with a book and study something for an hour. Try to get a kid these days to study something for ten minutes, right? They're distracted by everything because life is very distracting right now. I don't necessarily know that it means they need to learn the way we did. It means that we need to hopefully help them make new mistakes, but their world is just very different than ours.

(Joel Beasley at 01:03:32) Oh, yeah. Yeah. And I think one thing that I've learned—I don't think, I know—one thing I've learned as a parent: it doesn't matter how much I teach them. They're just gonna copy some version of my behavior. So my wife and I, as we've worked on ourselves and then our marriage so that we can—you see it so clearly when they start to grow up, you know, over the past nine years, it's just like they mimic us. And so it's like, how do—if we wanna make them better, we gotta make ourselves better and then spend time with them. And so that's been our strategy. And so far, it's been—I'll reach back out to you in fifteen years and let you know, but it's been rewarding for me. I love that I get to have a relationship with my kids.

(Anthony Scriffignano at 01:04:15) One percent of one percent of all parents can say that right now, which is, you know, maybe that's where the true hope in this is. If you—you know, I'm trying to, to some extent, at least a little bit, keep the technology in this discussion. If we think about all the potential devices that could be in their own head, the way they go to school, go to a library in a college these days, not a lot of books. There's not a lot of books in the library anymore. There are some books, like the old collections of whatever, but most of it's a very different experience, the way that they do research. When I did my PhD dissertation, I ordered books from Amazon. I went to a research librarian, and I had materials delivered to me. And I did a lot of online things as well. It was sort of, you know, in that cusp, right? Nowadays, the way they write a dissertation is almost completely digital. So the way we learn is changing. What does the research librarian do now? Are they a prompt engineer, or do they help you do something new that we didn't do? I don't know. I don't know the answer to that. I had that conversation with a librarian recently, and they thought it was amusing and terrifying at the same time.

(Joel Beasley at 01:05:35) They're hanging out with the people that make the horse and buggies, right?

(Anthony Scriffignano at 01:05:38) Well, no. I mean, the best ones—the best radio TV technicians, you know, made the switch from tubes to solid state, right? So I suspect that the best research librarians are all about asking a better question and understanding critical thinking, and they don't care how you do the physical research. They care that you're doing empirically rigorous research. And my hope is that that's what matters, not how did you get the words. I don't care how you got the words. I care that the words you write ultimately are your words.

(Joel Beasley at 01:06:15) So I know this is kind of just a context switch from what we were talking about. But personally, I marked this question as one that I was very interested in. You focus a lot on spotting novel behavior that an AI can't predict from past data. What's an example that stuck with you?

(Anthony Scriffignano at 01:06:32) So I can give you an example of something that I just published with Doctor David Bray. So this is a real thing and it's a really good example of that. The research question that we were looking at was something we call walled gardens, places that you can't see inside of. This could be an actual place that has a wall around it, like a military base or whatever, or it could be just a place that doesn't have a lot of signal. A lot of large corporate campuses, universities, and so forth.

(Anthony Scriffignano at 01:07:07) They have their own infrastructure inside that enterprise. So their own mail, their own police force, their own pizza places. And so it's hard to know what's going on in there unless you go in there and look. Right?

(Anthony Scriffignano at 01:07:24) So the question that we asked was, could you look at, from a—now this is all using commercially available data—could you look at the immediately codependent ecosystem around the walled garden? So think of a—we used, we looked at military bases. It doesn't have to be. Could be shipping ports. Could be anything. If you think about that military base, there's people coming and going into that base and out of that base. Those people buy gas. They order pizza. They get their windshields fixed. They hire practitioners. You know, they get physical therapy. They do things on their way in and on their way out. Right? And if you can identify that immediately codependent ecosystem that's close to it and where it breaks down and starts to be influenced by other things, you get this sort of weird shape that is—think of it as like a doughnut, but it's not a circle.

(Anthony Scriffignano at 01:08:20) It's like an inkblot kind of shape that's around the walled garden. If you establish all of the connections to all of the entities that are in that codependent ecosystem and look at them over time and through time and find periods where they are disrupted in an unusual way—we call this irregular irregularity. Everything's irregular, but there's an irregular irregularity that popped up—that very often indicates that the walled garden has some kind of disruption within it.

(Anthony Scriffignano at 01:08:56) So the premise was, when there's a disruption in the ecosystem that's around it, there's a disruption inside the walled garden. So we built the math and the data to do this. It's higher order dimensionality, really high order dimensionality. Really hard to do. And what we measured was this disruption effect. And then it turned out that when we back tested it—so we peer reviewed this with the APL, Applied Physics Lab of Johns Hopkins.

(Anthony Scriffignano at 01:09:26) So the math was checked by somebody else. The method was checked by somebody else. People looked at it. And then we actually had people call them and say, look, we think on this day, at this time, something happened. Sometimes we could find it in the news, like there was a fire or something. Sometimes there was nothing in the news, and we called them. And we would say, we think something happened on this date. And they would say, who told you? You know, that wasn't made public. There was a suicide or we had a change of leader or whatever. And it turned out that we were quite good at finding these disruptions inside the walled gardens. Turns out, if you have a disruption and you try to hide the disruption from the outside world, the hiding will be observable. So we can see that you're disrupted and we can see if you're trying to hide that you're disrupted. So give up.

(Anthony Scriffignano at 01:10:17) We'll know when you're disrupted. And the ability to do this, to find these areas of disruption inside walled gardens is a harbinger for a different way of thinking about disruption, that you can't just hide it when something's going wrong. And now what does that mean? This isn't just a military thing, but in the world writ large, do you spend your energy trying to cover your tracks, or do you spend your energy being transparent and dealing with whatever it is that's going on? And the data suggests that B is a better course of action. Pretty powerful research.

(Joel Beasley at 01:10:54) That's brilliant. That reminds me of the Pentagon pizza tracker.

(Anthony Scriffignano at 01:10:58) Yes. Think of that with like a thousand more dimensions of signal. The problem is, if you track pizza purchase and mentions in social media and cars in and out and, I don't know, traffic accidents—right? Those all have different units.

(Anthony Scriffignano at 01:11:21) And so it's very hard to compare those different systems because what is it? Pizza purchases per traffic accident. Like, the math starts to make no sense. So what you do is you spread it all out into an observable period of time, and you develop a distribution. I'm trying not to say probability, but it is a probability distribution of what you saw.

(Anthony Scriffignano at 01:11:49) And then you convert everything to a z score, so measures of central tendency. And so you take the units away. And once you take the units away, now you can compare one level of disruption with another level of disruption that have different units using something called Jensen Shannon Divergence, which is a statistical method that lets you compare distributions with each other. So it's a bunch of math, basically, but it's a bunch of math that makes a lot of sense. And what's really cool about it is the normal stones that you throw at a method like this is, well, you don't have all the data. Doesn't matter. As long as the data is equally deficient over the period of time that you're studying it, the missingness is uniform. As long as you don't suddenly disrupt the data by adding a new source or doing something to it that creates its own disruption in the data, all of that stuff washes out.

(Joel Beasley at 01:12:49) But even then, it's fairly dynamic. So if you did open a new part of the city with a bunch of new stuff, there would be a slight change in some of the data, but it would restabilize.

(Anthony Scriffignano at 01:12:57) It would, because the measurement of that—we call it the Haversine Annulus. It's not, it's neither, but it doesn't matter. That inkblot, the measurement of that inkblot, the construction of that inkblot is a dynamic thing. And all of this data is changing quite dynamically. None of that matters either.

(Anthony Scriffignano at 01:13:17) So it turns out you can use that same kind of a method looking at all kinds of data. It doesn't have to be commercial data. It can be satellite data. It can be data that comes from the networks, you know, looking at your network traffic. It can be data that—the only thing that's important is that you can create a centroid. So this method doesn't work well with any kind of data. There has to be a physical walled garden that you're interested in to use this particular approach.

(Joel Beasley at 01:13:52) Thank you so much for listening. And if you found this episode useful, please share it with a friend or colleague who you think would get value from it. And if you have topics that you'd like to hear discussed on the podcast, either add me on LinkedIn or send me an email, [email protected]. Every time I get an email or LinkedIn message, it absolutely makes my day and inspires me to keep going.