Implementing Trustworthy AI's with Anthony Scriffignano, SVP and Chief Data Scientist at Dun & Bradstreet

(Intro Narrator at 00:00:03) Hello, my friends. Today we're talking to Anthony, senior vice president and chief data scientist at Dun & Bradstreet. And we discuss the ways in which careers are formed from uncertainty, how to think about and implement trustworthy AI, and how leaders can encourage teams to be their authentic selves. All of this right here, right now on the Modern CTO podcast.

(Joel Beasley at 00:00:33) Here we go. This is the Modern CTO podcast. So I never set out to be a podcaster necessarily. It just sort of kinda happened.

(Anthony at 00:00:50) Well, I think a lot of times people ask me, "Well, how did you wind up where you are?" I think many of us, if we tell the truth, the answer is by accident. You know, you think you have plans, and you should have plans, but things change. And if things don't change, they die. So I didn't start out thinking I was gonna do data science. That phrase didn't even exist when I started out more than forty years ago in technology. I've always done things that were related to technology, but when I first started in university, I was actually thinking more along the lines of nuclear physics. So I took almost all the courses you would need to get a degree in physics, and then this computer science thing became possible. So, you know, let's move all of these things over here and count them for a different major and take a few other courses, and all of a sudden computer science starts to become a thing.

(Anthony at 00:01:43) I was working full time through all of my degrees. So I have four degrees. It doesn't matter. Nobody cares. But during all of them, I've worked full time. And so even as an undergraduate, I was working a full work week, I was going to school a full load of credits, I was a calculus tutor, I was on the swim team. So, you know, I get tired of thinking about that right now. But I think when we start out in our career, it's normal to have too many things going on. And then those things get winnowed, and they get refined, and they get transformed into something probably better than what we intended because we didn't have the perspective that we did when we started. So it sounds like you did a similar thing. I have programmed, the last time I counted, over 30 languages, programming languages. You wouldn't want me to write code right now. I can, but you wouldn't want me to because I think that our efforts at some point become better placed maybe thinking about what the algorithmic approach is or what the epistemology is, or what are the sources of bias, or being the red team and figuring out why that code isn't doing what it's supposed to do, or the person who brings up, you know, pesky things like regression and all of that.

(Anthony at 00:02:55) So I've taken on more of that role of looking across development efforts and being the customer of the folks who actually are the practitioners of being the chair-keyboard interface. I'm more of the adjunct provocateur of that kind of exercise.

(Joel Beasley at 00:03:14) I haven't heard that word before, epistemology?

(Anthony at 00:03:17) Like a belief system. What do you have to believe in order for this to be true? I can give you a really good example. Given where we are right now, everything is disrupted, right? So everyone's asking questions like, "When do we get back to normal?" Define normal. If you think you're gonna get back to what was normal before all of this disruption, you might wanna think about that a little bit differently. If you think that you're gonna push the AI button and just consume all the data from the past and predict what's gonna happen in the future, there's an epistemology under that. There's a belief system that that data from the past is sufficiently representative and stable to represent the sufficiently unperturbed future close in. That's not true. That belief system is not true. It's invalid. So you can push that button, and the math will work, and the AI will do something, and it'll call it accurate based on the prior data. But the belief system is what will kill you there because the future doesn't look enough like that past for that to be the only thing you do.

(Anthony at 00:04:20) Now if you're doing it to predict the weather or you're doing it to predict something that was unperturbed, you might get away with it. Or if you're doing something a little bit more, we use terms like Bayesian, you know, consume a little bit of the future and then go back and reevaluate, that kind of thing, then you can get away with that sort of thinking. But you have to stop and think, what do I believe when I use a method or a process? That you can code is code, right? It's hard to do, but it's doable. The really hard part is challenging what you have to believe in order to use that tool, that approach, that data.

(Joel Beasley at 00:04:52) Are there other beliefs in AI that you see common technologists, people that aren't necessarily experts in AI, but they're in technology, misinterpret or beliefs that they hold that are just wrong?

(Anthony at 00:05:05) Many. The biggest one I see lately is this whole AI ML thing, like, "Let me just consume a whole bunch of data and hit predict," right? Another big one is there's so much data now that if you look into any sufficiently large corpus of data, you'll find something that supports what you believe. That doesn't make it true. So that's called confirmation bias, right? Where you go in with an in-going assumption of what the answer is to the question, and then you find, conveniently, only the data that supports what you think is true because you think you're so smart. And maybe you are, but maybe you ought to let the data speak for itself because it doesn't care what you think. And unfortunately, there's a lot of that. And more unfortunately, the higher up you get in organizations, the more you are likely to introduce confirmation bias because you've been successful.

(Anthony at 00:05:57) And so you convey a sense of certainty when you speak. You convey a sense of knowing things or at least feeling strongly convicted in what you believe. And then people want to please you, and they want to bring the answer to you that will make you happy. Those are all different sneaky ways that confirmation bias sneaks into data science. Another one—I mean, there's so many—but another one that I see a lot is just really bad sampling. We forget all the stuff we learned in statistics. There's a reason why they make you take statistics when you study computer science, because you have to understand whether the data that you're looking at is representative of the thing you're trying to study. And a lot of times we use what's called a convenience sample, the data we happen to have, or the data we most recently got, or the data that is permissible for us to use. You have to be able to answer the question of how the data you're not using looks like the data you are using enough to draw sense from it. And a lot of times we don't do that these days. We just use whatever data we have because we're in a hurry and there's a lot of data out there. "Let's go use this data." It's a very dangerous thing to do. You inherit whatever's wrong with that data, you inherit and tend to multiply in the conclusions that you're reaching from it. Those are some of the big ones.

(Joel Beasley at 00:07:20) And how are you using AI properly at Dun & Bradstreet? And what are you using it for? I mean, I have a very basic understanding of what Dun & Bradstreet is because I needed a DUNS number once for something I was doing, and I had to go register. And I was like, "Alright, I got this DUNS number now." And it looks like—you could correct me, please—but I was like, "Oh, this is some business credit score type deal, information database thing."

(Anthony at 00:07:44) Yep.

(Joel Beasley at 00:07:44) And that's the very eloquent conclusion I came to.

(Anthony at 00:07:48) It's a fair conclusion. So it's like the story with the blind guys that all encounter an elephant for the first time. And one is touching the leg, and he says the elephant is like a tree, you know, it's virtually immovable and it's thick. And then another one is touching the tail, and he says it's like a snake, and it moves. And another one's touching the trunk, and he says it is like a snake, but a very thick snake. And they're none of them really understand the elephant because they're all touching sort of a part of the elephant, and they don't have the sense of what an elephant is. That's a lot like what our company is like. I think you tend to touch it in either looking at total risk or total opportunity or what you're calling credit on self, you know, trying to get people to give you money or do business with you or whatever. Those are all different major use cases that we have. So in total risk, it's helping companies understand who they're doing business with and whether or not they want to assess the right amount of risk in doing that business.

(Anthony at 00:08:48) That's how we got started way before this was a thing, way before there was an internet, way before there were databases, way before the Civil War. There was Dun & Bradstreet. And what was happening at that time—it wasn't called Dun & Bradstreet—but the seeds of what we are now. At that time, what was happening was westward expansion, and all of a sudden you couldn't just get on a horse and go see the guy that you were gonna go do business with. First of all, you might die doing that. Second of all, it might take months. Third of all, they might not be there when you get there. So this business of visiting businesses and forming an opinion about the character and quality of the business and their creditworthiness and various other aspects of the business was born. And then from that was born all kinds of other things. Sales and marketing: How do I find new customers? How do I find businesses that look like my best customers or businesses that look different if I wanna branch out? How do I find businesses in a certain part of the world? How do I make myself known to other parts of the world? That's the sales and marketing side. And then there are use cases that are combinations of those, like when companies are doing mergers and acquisitions and divestitures, they need to check each other out. They need to check their customers, their supply chains. So all of those start with collecting information about companies all over the world, hundreds of millions of entities in this database, understanding how they're changing, so millions of updates a day, and understanding how they connect to each other. So the linkages. Most companies in the world are not public. They don't have to tell you anything, and we have to figure it out. So there's a lot of modeling.

(Anthony at 00:10:30) There's a lot of discovery, recursive discovery. The kinds of things that we call AI now were being done here way before they were called AI. And then, you know, the dirty little secret: a lot of times this data is coming in in different writing systems, in different languages. There's different laws around the world about what you can collect, what you can't collect. "Keep this data over here. Don't keep this data over there." We have to comply with all of that as well and do all of that millions of times a day to get it accurate, complete, timely, connected, and relevant end product. Other than that, it's pretty straightforward.

(Joel Beasley at 00:11:03) Do you, when you're displaying data, data that you've inferred that wasn't necessarily given, does the consumer of that or the person experiencing it have some sort of scale or rating as to the accuracy?

(Anthony at 00:11:17) So accuracy is a tricky thing because there's many ways to measure accuracy. If there is some authoritative truth that you can compare to, then you can measure accuracy. Let's say we have a standard that says this is exactly how long a meter is, and then you measure something and you can compare your measurement to that scientifically accepted standard for what a meter is. When the question is, "How many employees does a company have?" that has businesses all over the world and who's getting hired and who's getting fired, and who's asleep when you're asking the question, and how do you count employees and do you count contractors or not? And there's a nuance to that type of question that makes it such that if the company is willing to tell you what they think the answer is, you can accept that in some cases, in many cases, as an authoritative source. It's a public company. There's laws against lying, you know, shareholder transparency, all that kind of stuff. But if it's a private company, getting at that answer sometimes involves consuming the answer in any way that it's available and then testing the veracity of that or the reasonableness of that. Sometimes it involves modeling. Sometimes it involves taking samples of other companies that are similar, looking at a bunch of parametric views of the entity to try to zero in on what you think that might be. When you're done doing that, you can back-test the accuracy of that model to the extent that authoritative truth becomes available. But it's difficult. What everybody wants is, "Tell me exactly what the number is today and how many degrees of freedom you have around that number for that particular entity in that particular context of that particular question." That requires an authoritative truth that you can go look up. So it's kind of a vicious circle.

(Anthony at 00:13:14) But the short answer to your question is, sure, internally we measure the performance of these models, the performance of our estimations. We have to. You're talking about a broader question here, which is around synthetic data, filling in data when the boxes are empty, right? You can't model a phone number. You know it's a phone number, right? But there are ways to estimate the size of an aircraft manufacturer or a law firm or a consulting company, something like that.

(Joel Beasley at 00:13:42) Like I said, I'd set up a profile. Do you have interfaces that I could give you more data?

(Anthony at 00:13:47) There's lots of API ways of connecting your data to our ingestion of data. And that is definitely a best practice for a small business because you want to be known to the world in a way that accurately represents your business so that they can do business with you and lend you money and decide whether or not they want to do whatever it is you want them to do, so that your future customers can find you. It's really important to get this right. A lot of times people look at it as, "Oh, well, why should I tell them?" Because you're out in the world and you want—do you have a website? Do you have a business card? Do you have a—you want people to know about your business. It's not a secret. I mean, if your business is something very confidential and you don't want the world to know about it, that's a different problem. But 99-plus percent of businesses wanna do business.

(Joel Beasley at 00:14:37) Yeah. If you were to, in one or two sentences, describe exactly what Dun & Bradstreet does, how would you do that?

(Anthony at 00:14:44) So I call it the one-breath resume, right? Like, before I get bored and walk away, tell me what your business, what your company does. And I usually say, well, it's essentially total risk, total opportunity. Helping customers, helping companies understand the risk of doing business with another company or the potential opportunity of doing business with another company. If you let me—if you say, "What do you mean?" and you let me get one more breath in there, I'll say, well, you know, there's a lot of reasons why you might wanna do that. Credit scores is an easy one to understand. You know, how likely are they to pay me? How likely are they to pay me on time? But then there's probably some other things you should think about, like fraud. Are they who they claim to be? Are they authorized to do what they're purporting to do with you? You know, you wanna push that AI button, right? The best bad guys, if they think they're being watched, they change their behavior. So if you model, you're modeling how the best ones are no longer behaving. Uh-oh. How do you deal with that? Well, let me tell you. Right? So there's all sorts of interesting corners of this that are fascinating to those who care. But total risk, total opportunity, compliance, sales and marketing, risk—those are the rocks that stick up out of the water.

(Joel Beasley at 00:15:58) And then what does your day-to-day look like? And what's your official title, by the way?

(Anthony at 00:16:02) I'm the chief data scientist at Dun & Bradstreet. My day-to-day is a combination of creating new capabilities that don't exist yet to make all of those things happen. So we work on things like linguistic disambiguation and semantic inference and understanding dyadic relationships like buys from, sells to, network effects, finding anomalies, finding fraud, finding new ways of understanding identity resolution, geospatial inference. So all kinds of sophistication behind the scenes to make all of those things that I just talked about happen. And then also dealing with, generally, our bigger customers or our customers that have the more complex problems to sort of bigify what they do from just—you probably think of us as either a credit report or a source of data, both of which are true, but there's so much more.

(Anthony at 00:16:52) So can I please help you graduate from that view to something that can enable you more? If you're a smaller business and I'm getting involved, I'm probably getting involved because you're representative of some larger class of problem, like many businesses have this opportunity, whatever it is. When lockdowns were happening all over the world, you had companies that all of a sudden the gig economy was huge focus. You know, people worked for more than one company if they worked for smaller businesses or owned small businesses. You had businesses reinventing themselves.

(Anthony at 00:17:25) They used to make scuba gear, and now they're making respirators, right? How do you figure that out? How do you—that semantic inference that I talked about, that's one of the ways you might do that, from how they talk about themselves or how others talk about them. So it's a combination of creating new capability and then enabling that new capability into use cases that are bigger than the one that they had before you got in the room. And then I also work around the world with places where the laws are changing and the view of what you can and can't do or shouldn't do with data is changing.

(Anthony at 00:17:58) So I sit on a bunch of external advisory groups with universities, with think tanks and so forth. And part of that is to share what we think, our epistemology, if you like, but also to—you know, these regulators, just to take that as an example, regulation, you could say, "Well, it slows me down." Well, I'm a scuba diver. That regulator is pretty important. Without the regulator, you're gonna die.

(Anthony at 00:18:24) Right? But if it overly regulates to the point where there's no air, then you're gonna die also. Right? So we have to help make sure that concepts like trustworthy AI and permissible use and data rights and all of these really squishy problems that are coming up, that we bring our best thinking to the table and that we speak where possible and join with others to make sure that we, as a human race, make the right decisions. We are at a cusp very much with AI and with data right now, where we have enough compute power and probably enough data to do most of the things we wanna do.

(Anthony at 00:19:02) Now the question is very often between, can we do it and should we do it? Or what happens if we don't do it? You know, those are much more nuanced questions. So I do a lot of work in that space as well.

(Joel Beasley at 00:19:13) Given your expertise and the amount of data that, you know, the company has and you said you sit on some boards and do those types of things, I was curious to know, like, how often is the government, specifically with the situation that we have now. Right? All the experts are arguing, just flip open YouTube. Right?

(Joel Beasley at 00:19:32) Somebody says this, somebody says that. But you guys have this massive amount of data of what's happening with businesses. Like, that's your entire existence is evaluating and understanding these businesses and what's happening to them and how they're changing and whatnot. Is the Fed coming to you and saying, "Hey. What's going on at least from your perspective?" Or is there not communication there?

(Anthony at 00:19:52) Certainly, serving the public sector is part of what we do as a business. So, but, again, without getting into specific relationships with specific customers, it is absolutely a part of our business to serve parts of the public sector that need to ask very important questions, like, where do we lend the money to have the most impact, or did the money have the impact we intended it to do? How do we help respond to this disaster? And it's not just in the United States. It's all over the world.

(Anthony at 00:20:18) One of the most personally inspiring things I've ever gotten to do was to work on the remediation of the data in Japan after the tsunami and the earthquake and the tsunami and the reactor issues at the Daiichi nuclear power plant and all of those horrible things that happened. There was a lot of people assuming nobody was in business anymore in Japan. Well, that was the farthest thing from the truth. So you don't want to add starvation on top of all of those other problems that they have, because everybody stops doing business with them. You're going to need to invent some new ways of figuring out who's still in business a lot faster than the way you did it yesterday, if you're going to respond to an emergency of that magnitude. So we tend to lean into big problems like that.

(Anthony at 00:21:00) We are not afraid of running toward the problem and helping whoever's got that problem see it differently and hopefully see it more clearly and in a way that's immediately actionable. This is not just theoretical data science. This is for real problems in most cases.

(Joel Beasley at 00:21:17) In the business world, at large, has the volatility of the market made data scientists or data officers more valuable?

(Anthony at 00:21:29) It certainly has made them more visible. I think time will tell whether it has made them more valuable or whether they're—the term is sort of fading. There's a lot of titles and positions and companies that stay because they are unambiguously necessary forever. And then there are others that can be more ephemeral because they get federated. Whatever that person in that function did becomes part of more of the rest of the organization.

(Anthony at 00:21:59) I think this particular field is a little bit of both. I think that it's evolving. I think that data officers, that role of being responsible for the discovery and the curation of data, making sure that we're bringing the right data in, that we're compliant, that we care for that data, worrying about what that data is intended to do. And then you cross the line into synthesis, like making sense out of the data. There are other roles, like keeping the data safe from cyber attack and ransomware and all that. That's another whole branch. Right? There's a lot of debate about what job title has all of these responsibilities. I don't tend to get involved in that debate. I think it's more important that there's a proper RACI in an organization, that you understand what needs to be done and that it's being done and that there's somebody held accountable for that and that they're getting the right advice from others who also have a stake in that game.

(Anthony at 00:22:53) It's not an easy thing to do. For smaller companies, it tends to be one person or two people. For larger companies, it can be so many that they forget to talk to each other sometimes. And you get these silos where—you know, I like to say that lessons learned are only lessons learned if you learn from them. Right?

(Anthony at 00:23:08) The learning isn't institutionalized across the organization because it happens in a silo, and then we make the same mistake over and over again in different parts of the organization. So it's an important role that I think will continue to evolve. I think it'll become more like a practitioner, clinical practitioner. As these systems and environments become more complex, we have to form a sort of a differential diagnosis of what we think is going on or what we think can be done, and then test it, and capture those learnings, and then intervene again. It used to be possible to just know all the data that was in the environment and know what you could do with it.

(Anthony at 00:23:46) When you and I got started doing this, I'm sure you could take the computer apart, put it back together, write your own drivers. You know, wasn't that fun? Well, you can't do that anymore. Right? So it's gonna have to evolve.

(Anthony at 00:23:58) And it better evolve because there's no promise that things always get better. We have to make them get better.

(Joel Beasley at 00:24:04) I always, when I'm talking with newer engineers, we have an advantage because we're higher in the stack now, but we're also at a disadvantage because in the nineteen sixties or seventies, you could walk into a computer. Right? And you could see all the parts that made it work. Now it's just this impossible chip that you can't—it's not just a square. Yeah.

(Anthony at 00:24:23) And some would argue, who cares? Right? There's open source code. There's libraries. There's edge computing.

(Anthony at 00:24:30) There's all these ways of putting increasingly complex things into increasingly smaller, more obscure components. Right? And then you start thinking about supply chain resiliency. You start thinking about what happens if there's microcode in there that does something you didn't intend it to do. What happens if you are overly dependent on a particular, within your development stack, a particular capability that evolves in some way because of a problem or an unknown vulnerability.

(Anthony at 00:25:07) All of a sudden, now your product doesn't work. Everything's connected to everything now. And so it's no longer possible to do anything other than reasonably trivial things. In a silo, you're incorporating a lot of other people's work, and that means you're inheriting a lot of people's assumptions, and it means you're multiplying the error in some cases. It means that you are inheriting vulnerabilities that you may not know about.

(Anthony at 00:25:35) So the job of being a developer, being a leader of people who are developing anything has become much more complex in the sense of you have to be able to vet those people that you're hiring. Are they just integrators of other people's stuff? Or do they think about the bigger picture of what they're doing and the implications of it to those around them and those critical dependencies. And, you know, in some organizations, you don't want that. Just do your job.

(Anthony at 00:26:04) Here's the spec. Do this thing. And, you know, we're agile and hurry up and don't ask too many questions because we'll deal with that in the next sprint. Right? I'm not anti-agile, but I'm kind of anti-negligence.

(Anthony at 00:26:16) So you have to be careful that you don't take that too far.

(Joel Beasley at 00:26:20) Got more questions about you. So you speak clearly. Your vocabulary is higher than average. How do you do this? Do you think your thoughts through and write? Do you have a writing habit? How do you organize your thoughts so that they come out clear?

(Anthony at 00:26:37) Well, thank you for that. I hope it's true. You know, Einstein said that if you can't explain something, maybe you don't understand it. Right? So I think that part of the secret to explaining really complex things is making sure you understand them before you open your mouth and start talking about them.

(Anthony at 00:26:57) I think it's also important to listen to what you're saying while you're speaking, because sometimes you get into this loop where there's a lot of words coming out, but you're not really advancing the thought. I think it's also fair to say that there's an element of practice in it. When you respond to a lot of people asking you questions, what do you think about AI? What do you think about bias in data? How do you know if something's true?

(Anthony at 00:27:22) You get to a point where you have an opportunity to refine what you said the last time and make it even better the next time instead of just pushing play and saying the same thing because it seemed to be, you know, somehow pleasing to the crowd. You have, I think, a duty when you're speaking publicly, even like we're doing right now, to pay attention to what you're saying. You never know who's listening. You never know who you might be influencing. You never know who might be taking notes.

(Anthony at 00:27:49) And you have an opportunity, but you also have an obligation to those unseen others to pay attention to what you're saying and not just hit play and act like a star. I mean, you're never as important as you think you are. You should really pay attention because there's somebody really important out there that you don't know about.

(Joel Beasley at 00:28:06) Yes. I mean, I happen to love this. That's—I've made a career out of it, just finding smart people and asking them questions. Yeah.

(Anthony at 00:28:16) And there's an element of that too. If you love what you're doing, which I do, and it's obvious that you do, it comes out in how you talk about it, the passion with which you can talk about it. There are times where you can take that too far. You know, ask me about quantum algorithms, and pretty soon you're gonna be saying, "Please stop talking about quantum algorithms." Right?

(Anthony at 00:28:38) But they're fascinating, right, to some people. We joke in my group, alright, we're arguing about calculus again. You know, like, we actually use calculus, but most people don't care. Right? So you have to be careful that you don't take it too far, because anything can be taken too far.

(Anthony at 00:28:56) And now you're down in the weeds talking about how you connect the flux capacitor to the phlomnosticator, and nobody cares. And they'll nod, and they'll do the "Fascinating." And they're Googling phlomnosticator. And you know, that's not a thing. But you're serving no one when you do that. So I think it's important to not cross that line.

(Anthony at 00:29:15) I also have a kind of a weird thing that when I get more tired, my words get bigger. I don't know why. It's just a weird thing. So I have a tendency if I'm really busy or if it's the end of the day or if there's a lot of things going on, I will have a tendency to say things in a more complex way because it's more dense. It's more efficient.

(Anthony at 00:29:35) And then, you know, it gets to the point where it's mockable. I said to somebody one time, "Do you think this is abrogative of our preexisting strategy?" And he looked at me with a straight face, and he said, "It might be if I knew what that meant." And so, you know, you can take it too far. Definitely.

(Joel Beasley at 00:29:51) I do something different when I'm tired. I just get funny.

(Anthony at 00:29:54) Well, I do too. Yeah. I get silly.

(Joel Beasley at 00:29:57) Silly with big words? Yeah. That's an awesome combination.

(Anthony at 00:30:00) There is a tendency—actually, I'm glad you brought that up because I think people misinterpret humor, especially in the workplace. You don't wanna be a clown. You don't wanna be a buffoon. You don't wanna be always joking around. But if people are afraid to show their authentic self, to laugh a little bit, to have a little bit of joy in what they're doing, you're probably not going to get their best creativity.

(Anthony at 00:30:24) You're not going to make them feel like they're doing something they wanna do, they're doing something they have to do. There are leadership styles that are very effective that are predicated on doing that, instilling fear. When you study—I used to teach courses in power and influence. One of the most effective types of power is the ability to punish and then the ability to reward in that order. Right?

(Anthony at 00:30:48) So when we castigate somebody, when we criticize them, that's a type of punishing. Right? When you give them the endorphins that come with humor, that's actually less effective, studies show. I wanna say I don't care. That's not necessarily who I wanna be.

(Anthony at 00:31:07) But I do respect the fact that there are situations in organizations, in life, where there's no time for that, and you've gotta get people to stop, focus, pay attention. This is what we're doing. This is very serious. You gotta know when to do which, and it's not always obvious. So—

(Joel Beasley at 00:31:26) Well said because, you know, through all of these hundreds of interviews, one of the things that I have found is when I started, I thought there would be an answer. Like, okay. Here's your style. Take this test. Here's the type of leader you need to be.

(Joel Beasley at 00:31:38) Yeah. And then I realized it's more like cooking a meal where you have to adjust as you go.

(Anthony at 00:31:45) Yes.

(Joel Beasley at 00:31:45) And you have to kind of figure—there's some basic concepts that you need to understand that work, you know, relatively well as a base. And then you sort of just have to figure it out as you go. And knowing when to inject humor versus when to be serious and understanding the energy of a team and how to raise it or lower it based off of what you need right now, all of that mixed together is what I found according to my definition of what I consider a great leader.

(Anthony at 00:32:12) So what you're talking about in the study of leadership is called reflective leadership. Not just leading the way you lead because it feels right, but thinking about why you do what you do and actually reflecting within yourself on the choices that you make as a leader. It is a very compelling point that reflective leaders are way more effective. Sometimes intuitive leaders who just lead and they know what to do, and they say we're going that way can be very effective, and they can be this sort of unconscious competent. They don't know why they do what they do and they're not doing it on purpose.

(Anthony at 00:32:52) Those are a very rare breed. The folks that challenge what they did today, how effective was it, where could I have been more effective, how could I have been more effective, who seek feedback from others. And I'm not talking about sending out a survey or a thing. I'm talking about going in authentically, having a talk with somebody and saying, you know, let's take the stripes off. I don't feel like that went well.

(Anthony at 00:33:15) Tell me how you felt. You know, tell me where it came from from your perspective. Sometimes explaining why you're about to do what you're about to do. We were talking earlier that I have many years in as an EMT, so emergency response. When you take incident command, one of the things they teach you is how to evacuate an area.

(Anthony at 00:33:33) Like, if there's a gas leak or some not obvious danger to a large number of people, you don't have time to do what they do in the movies. You know, run around yelling the thing's gonna blow up. That's not how it happens in real life. In real life, if the incident commander makes a decision, we're gonna go to the high school. You don't always know that that's the best place to go.

(Anthony at 00:33:54) You gotta pick a direction. Because if you tell people to evacuate the area and leave immediately, half of them are gonna move in the wrong direction, and they're gonna move to someplace more dangerous. So you pick a place and you say, we're evacuating the area. Please make your way to the high school. Further instructions will follow.

(Anthony at 00:34:10) Right? And you have to say that with conviction and clarity and certainty that you may not actually have in that moment. That's not a lack of authenticity. That's called saving people's lives. Rarely in corporate America is somebody gonna die if we don't make the decision in this instant.

(Anthony at 00:34:26) So that ability to stop and say, why am I doing this? Don't hit send yet. Right? Why am I doing this? What am I trying to achieve?

(Anthony at 00:34:37) How could this be misinterpreted? Who can help me? It doesn't mean you have to overthink everything, but it means you could probably think a little bit more about the really important decisions and probably be a little bit more effective if you just take your ego out of it a little bit. So that's sometimes called authentic leadership. That's sometimes referred to as reflective leadership.

(Anthony at 00:34:55) They're kind of cousins. They're not the same thing. And then there's another cousin of that, servant leadership, which is leading for the benefit of those you lead rather than for your own personal health, you know, just trying to get more of whatever it is you want. Like, there's a lot of people that that is really their objective function is to climb over the bodies of everybody else and get more of whatever it is they want. It can be very effective for some people.

(Anthony at 00:35:18) I don't want to be that person. So you have to decide who you want to be and why you want to be. These are questions that have been around for thousands of years.

(Joel Beasley at 00:35:26) But it's amazing. That's like a choose your own adventure. You can decide what type of leader you want to be. You can go hard in that direction and then realize what the balance is for it and then adjust accordingly.

(Anthony at 00:35:38) And you may not get what you want. You may get more of what you need. Yeah. You may not actually get to where you thought you were going. You have to be open to that.

(Anthony at 00:35:47) You know, in martial arts, there's a principle they call being like a reed in the wind. You see it all the time in movies and stuff. Instead of resisting whatever's happening to you, you sort of bend with it and then use that energy to go somewhere else. And then depending on the style and the epistemology, some were better for both of you or some were better for you. But a reed is not weak.

(Anthony at 00:36:11) A reed bends in the wind, and that's what makes it strong. You can kill somebody with a reed. Right? But if the reed were to try to oppose every breeze that blew on it and resist everything like an oak tree, it wouldn't last very long. So thinking about it like that, reacting, this is gonna sound really weird, but reacting in a proactive way, like reacting with purpose, reacting to a place that you choose to go is a very empowering thing to do as a leader.

(Joel Beasley at 00:36:41) When I was in my early twenties, I was having a rough time. And somebody that I was close to at the time, they said, it's clear that you're drowning, and you need to, like, instead of fighting the waves and drowning out there and splashing around, they're like, you need to learn how to flow with it.

(Anthony at 00:36:59) Yeah.

(Joel Beasley at 00:37:00) And instead of drowning, you need to ride the waves.

(Anthony at 00:37:02) Yeah.

(Joel Beasley at 00:37:03) And figure out where that's going to be and position yourself there, and then you'll have a much smoother experience than right now. Things are just coming at you out of nowhere.

(Anthony at 00:37:11) If you watch Olympic swimmers, they don't make a lot of splash, except for butterfly. But in general, even in butterfly. If you watch Olympic swimmers, there's not a lot of wasted water. Right? You don't see their arms flailing around.

(Anthony at 00:37:26) It's very efficient. And a lot of that wasted energy of pontificating and pampering about how important you are and making people fear your next words. And they, alright, fine. You can win. You want to win? You can win.

(Anthony at 00:37:41) But we have a bigger problem here than making you win. We're trying to do something. And obviously, you can't say those words because you can be right and dead.

(Anthony at 00:37:53) Right? So you have to find the way to get that advice in there in a way that can be received by the other when you're in those situations. Not always easy. And then, you know, since this is a tech audience, very often these things happen when the thing broke or the thing didn't do what the thing was supposed to do. Or you're trying to replace the thing with another thing and the other thing doesn't do the thing that the first thing did.

(Anthony at 00:38:17) And you got a lot of experts running around trying to out think each other and out expert each other. And very often, they are, I use the analogy of trying to push the rock up the hill. Right? If you all get behind the rock and push, the rock goes up the hill as efficiently as a rock can go up a hill. But if you start, even if you're all pushing up, but you're pushing at vectored angles, you're wasting a lot of that energy, and that tends to happen.

(Anthony at 00:38:43) And then somebody will come along and yell at the rock, or somebody will come along and say, lead harder. You know, like these ridiculous, you know, clearly, you're not effective in pushing that rock, so I'm going to ask you to push the rock and a bigger rock. You know, that happens in tech all the time because it's a constantly changing environment. And you thought you were responding to this thing, and then that other thing happened, and you never finished this thing. So you get into this state of hyper disruption is what I call it.

(Anthony at 00:39:14) Disrupted disruption. Peter Vaill, amazing man, wrote a book called Learning as a Way of Being, and he talks about organizational white water. When organizations are in this state of constant frenzy, you do different things when you're in white water than you do when you have time to patch the holes in the boat and so forth. So as a leader in tech, if you're always in white water, you have to reflect on how much of that white water is being caused by your attempts to deal with the white water, to use your example of trying to swim so hard that you're creating a lot of splash and waves. And sometimes the easier way to move forward is to not push as hard and to push more efficiently.

(Anthony at 00:39:52) It's not, it's easy to say, not so easy to do when there's 17 priorities and four people trying to address them.

(Joel Beasley at 00:39:59) It's hard because it's also a little bit, it's like, abstract. It's a little ambiguous, right, to try to figure it out. And it's not a quick thing either. Like, it takes, for me, it took like, years.

(Anthony at 00:40:11) Yeah.

(Joel Beasley at 00:40:11) One of my first moments growing as a leader was, you know, I was individual contributor, software engineer, and I'd sit there and write test driven development. I'd command the computer to do things all day. A hundred times a day, thousand times a day, I'd send commands, and if anything failed, it would give me a debug log output. Right?

(Anthony at 00:40:26) So I

(Joel Beasley at 00:40:26) know exactly what I did.

(Anthony at 00:40:27) What you told it to do, whether that was what you intended or not.

(Joel Beasley at 00:40:30) Yeah. And then I'd go interact with people in my life, whether it's a relationship I was in or, you know, work thing, and they don't do any of that.

(Anthony at 00:40:38) No.

(Joel Beasley at 00:40:38) And so somebody had said something to me because I was getting frustrated. And they said, you know, you can win the conversation, you can win the argument, but lose the relationship.

(Joel Beasley at 00:40:54) Yeah. And I was like, oh, wow. That, you can be right and lose the relationship. So having that self awareness of like, how hard do you push and like, where do you push, those things are actually really important.

(Anthony at 00:41:08) It's the concept of overt intentionality, doing what you do on purpose. We can't always do everything we do on purpose because we get tired. We're human beings. We have, you know, egos. We have history.

(Anthony at 00:41:22) We attach that history to people that don't even know us and say, oh, they're like that person. They don't know they're like that person. Right? I can't work with him. I worked with him two years ago and he was a jerk.

(Anthony at 00:41:32) Well, are you the same person you were two years ago? Of course not. Well, why don't you let him not be who he was two years ago? Like, why don't you, you know, at least start from the premise that life has put you back together for some reason? These are very easy things to say, very hard things to do.

(Anthony at 00:41:49) People are not generally as nondeterministic as programming environments. And sometimes the way we see the situation is clouded by all of that context that we imagine to it that may or may not be there. Some scientists argue that that's how the human race has actually survived is the ability to intuit context where we don't have enough data. You know, you hear a growl, you see rustling, you imagine an animal, you don't have to actually see the animal. That probably helped us not die in caves and forests, but it might get in the way in the corporate jungle.

(Anthony at 00:42:26) Can't believe I just said that, but yes.

(Joel Beasley at 00:42:31) As you've gone through your career and as you've grown in your leadership and, you know, your responsibilities and all of that, there's obviously stress and there's things that go along with that. And I'm curious to know, as you traversed and had that journey, how has, like, God played a role in all of that?

(Anthony at 00:42:47) Well, if I can substitute spirituality in your question just to be more inclusive of people who might be listening, huge. Absolutely huge in my life. I, you know, I don't have any problem. I consider myself a scientist, a for real scientist. I don't have any problem also being a spiritual person.

(Anthony at 00:43:06) They're not in any way in conflict for me. I understand that there are people who can construct arguments that attack that position that I just took. But you asked me how it has affected me, and I'm telling you how it has affected me. So my spirituality is a big part of who I am. It's part of how I try to treat other people regardless of whether they respond in kind.

(Anthony at 00:43:25) It's a way that I try to think about serving others first, about being as authentic as I can, as truthful as I can, you know, not to want their stuff, to help them keep their stuff. Those are all very spiritual things. I don't want to crush the other person in order to win. Now in business, you know, somebody's gotta win. One person gets the contract.

(Anthony at 00:43:53) I'm not talking about that. I'm talking about these broader things. I've led very large organizations with hundreds of people, and I've led very small organizations with, you know, you count them on one hand. The smaller ones are much harder because the larger ones, you don't lead hundreds of people. You lead seven or eight people, and then they go and lead other people.

(Anthony at 00:44:11) And, you know, you've convinced yourself that you're controlling this very large archetype. But then the reality is you're dealing with this counsel of others. Right? In order to have a counsel of others, you have to take counsel from others, and that's a very spiritual thing as well. So I don't, they're indivisible to me.

(Joel Beasley at 00:44:27) Nice. Yeah. What is, you mentioned it in your biography, I think, or in your about section. What is trustworthy AI?

(Anthony at 00:44:35) So I would direct you to the OECD work that's being done in this area, one AI.org, because there's volumes and volumes of information on this. But in a nutshell, AI doing what we intend as humans for it to be doing in a way that we as humans can understand. Not only that it's doing what we intended to be doing, but that it's doing things that project our values and our intentions in so doing. So I'm trying not to name specific situations, but there's been some very visible things in the recent past where AI accidentally did something that the purveyors of that AI didn't intend it to do. You know?

(Anthony at 00:45:20) And let's say you take a quiz on social media. You know? If you were a tree, what kind of tree would you be? Let me ask you a few questions. Well, do you really think that's about what kind of tree you are?

(Anthony at 00:45:32) You know, you're an oak. Oh, look. I'm an oak. No. You know, you just answered all those questions.

(Anthony at 00:45:37) Right? That's not very trustworthy. That's pretty out there. That's pretty obvious. There are way more subtle examples.

(Anthony at 00:45:44) Really, AI isn't necessarily doing what we think it's doing or what we would intended to do or producing the outcome that we expect or need. It's a whole field. We used to say that we wanted AI to be transparent. The reality is that methods have, especially hybrid methods have become so complex now that if you said push the explain button and tell me everything you did to get to that answer, you wouldn't understand the explanation. And there's millions of steps that went into that.

(Anthony at 00:46:13) And it's nondeterministic, and it's not gonna do the same thing next time because the conditions have changed and all that. So what did you gain from this is an evolution in that kind of thinking to get to something more useful. We need new words to talk about AI. There's no intelligence in artificial intelligence. It's math.

(Anthony at 00:46:31) Right? And there's no learning in machine learning. It's regression. Right? So, but we use these, it's called anthropomorphism.

(Anthony at 00:46:39) We put these human terms on these machine things. Right? They're not people. You know? And they're not really thinking.

(Anthony at 00:46:45) And when they have a goal, they do have a goal, but it's an objective function in a mathematical sense. It's not a goal, like a spiritual goal. And so we call it a goal, you know. And we need new words to talk about this stuff that we now call AI.

(Anthony at 00:47:03) And I don't know what those words are yet because we haven't invented them.

(Joel Beasley at 00:47:06) Yeah. No. I love it. Dude, this is great. First, I got a number of things to wrap.

(Joel Beasley at 00:47:11) Alright. We're gonna wrap up the interview on this final question. I'll come at it with a creative angle, though, for you.

(Anthony at 00:47:17) Sure.

(Joel Beasley at 00:47:17) Let's say that you're driving to the grocery store, and you're at a red light. And up next to you, another car pulls up, rolls his window down, and it's Elon Musk in the brand new Tesla. Right? And he says, hey, Anthony. I want to come show you something. Come back to my estate.

(Joel Beasley at 00:47:35) He doesn't have a house. He has an estate. Right? You go back there, and he's got this time machine. You go into the time machine, and the only thing you can do, so the rules are important here.

(Joel Beasley at 00:47:44) You can go back to yourself the day you graduated your first degree of college, and you get to give yourself one sentence. So you're walking up to your past self. You get to say one thing, and that's it. What would that be?

(Anthony at 00:47:59) Well, I think the first thing, not to be snarky, but I'm very troubled by the grandfather paradox here. Like, if I can go back in time, this is terrifying. But I'm going to wave all that off for a second. So a piece of advice to my former self—do I get to then come back and be who I am now? Because who I am would change as a result of that advice, right?

(Joel Beasley at 00:48:23) Mhmm. Let's say that you don't know it's you, and you just get to walk up to yourself, and you don't recognize that it's you in the future, and you just get to say something that would be useful.

(Anthony at 00:48:36) Because there are some really troubling problems with temporal dynamics here, but I'm going to ignore them all because I'm a nerd.

(Joel Beasley at 00:48:41) I appreciate it.

(Anthony at 00:48:42) What's the advice you would give to your former self that you wish you could give yourself? I probably would say, don't take yourself so seriously. I would probably say things are not as black and white as you tend to feel that they are. There's a lot more gray. I would say that you will probably live a lot longer if you take a breath before you get dramatically passionate about something you believe. It probably will change if you stop and look at it a little bit more. And I would probably also say that we're not—I say this a lot—we're not human doings. We're human beings, right? What you do is what you do. Whatever you do in that moment should never define who you are. So be authentically who you are. Your job, your title, the work that you do is certainly important, but it should never be all that defines you. There must be more, and if there isn't, that's really sad.

(Joel Beasley at 00:49:41) I love it.

(Anthony at 00:49:42) Can I give advice to people who are coming up? Because I get asked that question a lot, and I think it's important as we get older and have more gray hair, of which I have a lot—fascinating, artificially influenced. You know, we should always think about how that same question would be if I was giving advice to other people, not just myself. The one thing I would say is to be humble, especially in computer science, data science. Things have gotten sufficiently complex now that you can't do it all yourself. You think you can because you came out of school and you had data sets that contained the answer, that were permissible to use, that were already clean. The world isn't like that, right? So make sure that you expand the circle enough to get to the right answer, because it isn't about you. It's about getting to that answer. And along that same vein, help other people because they're going to be seeking your input. The second piece of advice I would say is, and we talked about this a bit, you should learn to make things simple but not oversimplify them. Get to the right level of sophistication and don't push it more than it needs to be. But don't also let somebody trivialize it. And then the third thing, and I think this is really important, is when you're doing things, don't forget to pick your head up and reassess whether the environment has changed while you're doing it to the point where you should be doing something else. Be open to the fact that we're in a period of time now where the world is changing faster than the data that describes it in many cases. So be really careful about how things are changing while you're working on that thing you're working on.

(Joel Beasley at 00:51:24) Nailed it. That's a mic drop moment right there. Thank you so much for listening. And if you found this episode useful, please share it with a friend or colleague who you think would get value from it. And if you have topics that you would like to hear discussed on the podcast, either add me on LinkedIn or send me an email, [email protected]. Every time I get an email or LinkedIn message, it absolutely makes my day and inspires me to keep going.