Managing the Risks of AI Generated Code with Matt van Itallie, Founder & CEO at Sema Software

(Intro Narrator at 00:00:01) Today, we're talking to Matt van Itallie, founder and CEO at Sema Software, about the risks of AI generated code, catching up on life, and more. You're listening to Joel Beasley, Modern CTO.

(Joel Beasley at 00:00:17) But I'm a curious person, and I did get to see what you were doing at your company, and I have about a million questions. But I was hoping that you could just give me a quick overview from your perspective. Like, what is the company? What's the problem it solves? What are you doing over there?

(Matt van Itallie at 00:00:36) Sure. So I came up with the idea of Sema back in 2015 when I saw two really good friends of mine who were CTOs really struggling to connect with the rest of the C-suite. One of them needed to make the case for refactoring, and the other had to explain why we needed to invest in, broadly speaking, tech debt reduction. And I just saw the tech folks and the non-tech folks talking past each other. And I was in leadership team meetings at both of these software companies, and I would watch the chief revenue officer push a button on Salesforce—how good are sales and what do we know about salespeople? Literally all of it with one button. A C-suite tool for the chief revenue officer, and every other team had it, and of course had it thanks to code, except for the CTO who would have to use hand-waving and explain what a refactoring was. I'm going to have to combine six different tools to tell the story. And it just drove me nuts. I'm a pretty logical guy, and so spent about two years researching it. And then we've been around for six years building tools for CTOs and helping tell the story of connecting tech to non-tech folks.

(Joel Beasley at 00:01:58) I love it. So you've been doing this for a while, but now you're in a growth stage right now?

(Matt van Itallie at 00:02:04) That's right. We introduced two products recently. One, a CTO dashboard, and one, an AI code monitor, which helps understand the use of GenAI code in the code base, helps understand if it's going to be in compliance with the regulatory changes coming and legal changes, and also acts as a coach for engineers to get the most out of it.

(Joel Beasley at 00:02:32) And so what does this dashboard look like?

(Matt van Itallie at 00:02:35) Sure. There's two parts to the dashboard overall. One part is what we call functional areas, general code metrics, and it's a rollup of the code security tools, like a Snyk or a Dependabot, of the open source tools, IP risk tools, cyber tools for an outside-in look, and then code quality, like CodeSee, process tools like Flow and Jellyfish, and also a team retention view. We spend a lot of our time—we got our start in technical due diligence helping—we've looked at over a trillion dollars of companies in strategic situations. And one of the most important things, which I'm sure is obvious to your audience, but we had to teach the non-technical folks, is by far the most important thing that will drive the success of an engineering team is having enough engineers who know the code. Having the novel or the half-written novel and not having the novelist just doesn't work. And again, I'm sure your audience knows it completely, but I'm pretty proud of our work helping prove that to deal teams and investors and the like—to pay a lot of attention to developers, to their retention, to their experience.

(Joel Beasley at 00:03:57) Yeah. That's interesting too. So can you tell which code is AI generated versus which is written by people?

(Matt van Itallie at 00:04:04) Yep. So we have a pretty high accuracy rate at this point. It's going to be a never-ending journey to stay up and keep getting more and more accurate. But where there's different confidence intervals where we are very confident the code is AI generated, our accuracy rate is also very high, both false positives and false negatives. Some of our clients who asked us to build it also want slightly less likelihood to make sure that they're covering all of the risk, sort of like security warnings, SAS warnings can come with severity and likelihood. And so at the lower confidence levels, it's less accurate. But what we're hearing is folks want to make sure, just like a software bill of materials, they want to know that their generative AI bill of materials is comprehensive.

(Joel Beasley at 00:04:56) I'm thinking about how I would use this, and I think it would be really useful to detect that it's AI generated code when people aren't trying to hide that it's AI generated code. That sounds like a use case where maybe some new engineers just do a bunch of stuff and commit a bunch of stuff to the code base, and they just went on their own and used these tools versus someone who was actively seeking to keep this information from the software systems.

(Matt van Itallie at 00:05:26) So it's a really good point, Joel. And in fact, we're working on both use cases at once. Let's talk about the first one. We call it the enterprise use case. The goal of that is really simple. Get engineers comfortable and feeling supported, not judged, about their use of GenAI because we know under the right circumstances it is a massive productivity boost and also helps them stay in their flow state. That's number one. And number two, use it and interact with it. So we would say that AI generated code can be pure or blended. Pure is straight copy-paste. Blended, of course, is an engineer interacting with it. It turns out that almost every benefit and almost every risk is improved. You know, the benefits are more and the risks are lessened if you work with the code, if you interact with it. For example, pure GenAI code will probably not get copyright protection. So all the folks who've started using GenAI tools for coding and just been copy-pasting it in, there's going to be a huge cleanup effort for their teams to go find that pure code and modify it enough to meet copyright. And so really, that's one of the risks. The other big one is team risk. I mean, there's a couple more, but on the team side, CEOs, CTOs we talk to, they really want to make sure that someone understands the code. And if all it is is copy-pasted in, people will understand it less. That also argues in favor of more blended code. So all to say in the enterprise use case, the name of the game is measuring it, comparing it to compliance standards that are coming out, and then encouraging developers. We're on your side. We just want you to use it, number one. And number two, in almost every situation, we want it to be blended, not pure. So that's the enterprise case.

(Matt van Itallie at 00:07:26) Then the other, what about for folks who are trying to hide? We're incredibly excited to work with a, let's say, a leading HR provider. It's not announced yet. And they are—we're working together to find folks who are trying to hide it. You can think of some situations where, let's say, in the hiring process, where folks might want to hide the composition of that code. And so there, it's a different training set, obviously, and looking for ways that people are trying to tune or maybe hide their tracks or just not demonstrate how GenAI has been used.

(Joel Beasley at 00:08:01) Interesting. That would happen in interviews, right? You would find them out in two weeks or three weeks when you actually start working with them on a day-to-day basis.

(Matt van Itallie at 00:08:10) Exactly right. Exactly right. And this isn't—that's not a new problem, finding copy-paste from open source, from Stack Overflow, et cetera. What's really interesting is, to give huge kudos to this unnamed organization, you know, open source, you probably get dinged in an interview. You probably get dinged in an interview if you're pulling in copy-paste open source in 2023 and 2024 and on. You would likely get dinged for not using GenAI as the starting point, but then you have to go on and modify it because it is one of the most powerful engineering tools of the last 25 years between—I mean, obviously you have your list. Mine is VS Code, open source, Git protocol. They're actually going to reward—organizations may choose to reward the initial use of GenAI, but then, again, making it blended instead.

(Joel Beasley at 00:09:11) These copyright things—I know you're not attorney, AI attorney guy, but just being in the industry, as casual people talking—how hard is this going to be to prove that AI? I mean, I think OpenAI, their own rate on speech detection, not code, but their own rate on speech detection generation was sub-30%. If a tool does detect the copyright issues for enterprises and things like that, that this is generated, how do you prove that it wasn't? Like, what's the evidence there? Because, I mean, at the end of the day, you and I understand programming. We've been doing it for the better part of two decades, right? The code is in the documentation and you kind of put it together. So how do you look at code and guarantee from a not from a guessing perspective what we were talking about earlier or from a management perspective of this, but from a legal perspective to actually look at that and be like, you can't do that.

(Matt van Itallie at 00:10:15) Yeah. So it's an awesome question. As part of technical due diligence work and our time in helping companies in strategic situations, like getting ready for sale, like meeting compliance standards, we spent a lot of time working on open source. And for better and worse, that is an analogy for us. The way that open source works in practical terms from our experience is stakeholders want a really high-quality scan. It doesn't have to be the maximum quality. So as an example, in open source, it is sufficient in most situations to look for licenses, at-risk licenses, without necessarily having to do a code fragment search. So the maximum possible detection of open source is not state of the art—is not required in what's expected in business transactions. And then in high-stakes situations, a high-quality tool spits out results, and the code owner is expected to triage them. Of course, in the open source context, the question is make a note as to whether or not it's distributed code, meaning it's accessible to your customers either through SaaS or on-prem. We believe the same process will happen. So there's a thing called the software bill of materials, of course. We've built a generative AI bill of materials, or GBOM. Folks will, in a high-stakes situation, have to demonstrate it, demonstrate that it exists, and then in some situations have a human go through and validate. And just like we do in open source, we at the end of the day have to rely on some human intervention to do that final step. And so our expectation is that open source—excuse me, our expectation is that GenAI legal risk management will be happening the same way.

(Joel Beasley at 00:12:15) Yeah. I did get to talk to one of the lawyers that was dealing with Eminem and David Guetta's lawsuit about him using the voice in a live performance. And I think his name's Oliver. And as I was—I think it was either on or off the show—but at some point he was like, yeah, basically attorneys in every different country are just going to be hashing this out for the next five years, figuring out how to deal with this in a real way. Because it's really, really tough when you create a technology that was designed with neural networks, which is designed to mimic the human brain. And now it is this type of intelligence that is generating content, and now you're having to differentiate between, I don't know, biological intelligence and silicon-based intelligence, right?

(Matt van Itallie at 00:12:59) Exactly so. And, you know, the folks who have a vested interest in getting to the bottom of this, they're only going to be as good as the tooling. And so folks will—for example, it is all but certain that organizations serving the federal government will have to produce a GenAI bill of materials, just like after '21, the US federal government, just like after '21, the executive order, they have to produce a software bill of materials for open source. There's going to be a thing, and even before we get to the perfect accuracy of it, folks will still have to respond to the state of the art as it exists today. I completely agree. This is going to make a lot of lawyers a lot of money for many years ahead. But they're still, in the near term, going to have to explain themselves with the tools that are available. I should say, and I probably should say this sooner, there is some minimum length of tokens or characters that you're never going to be able to tell. Literally never, right? Hello, world could have been written by anybody. And so the goal for our amazing team and others who may be working on this is decrease false positives, decrease false negatives, and shrink the size of the code where you can accurately make predictions. But it will be—we don't know yet, but it will certainly be more than two characters, two tokens long, because it's just—it is literally the same.

(Joel Beasley at 00:14:29) Yeah. No. That is—that's an interesting point. I didn't even think about that. I guess that's what I—that's what the root of what I was getting to, I guess, in the back of my mind. But, yeah, you formed that super, super well. Okay. So GenAI, you spend a lot of time there. Where else do you spend a lot of time?

(Matt van Itallie at 00:14:47) So, as I said at the beginning, our CTO dashboard has two parts. One, this feature is an AI code monitor and detector and monitor. And the other part is what we call core metrics. Core metrics of code base health. And for us, we break it into—this is all non-functional requirements. Functional requirements will come at some point, but just with the non-functional requirements, we would say that code base health is there is product risk, the ability to deliver the code consistently, predictably, and then there is compliance risk, things that lawyers might get concerned about. And so there are measures of product risk and there are measures of compliance risk that come together in a rollup format. We actually created a one to 100 score of code base health that—if I could—I've been thinking about this literally for eight years now. The idea of a single "how are we doing" score was always something that would be nice. But literally until six months ago, we didn't have enough data collected that I could look an engineer in the eye and say, I actually think the number can be reduced. I actually think the code base health can be reduced to a single number. We needed to look at 800 companies. We needed to scan a trillion dollars worth of code doing the math to get to a one to 100 score. That's mathematically not hard, but doing it with integrity that if we want to get the score from 30 to 40, it is actually in the best interest of the code and the best interest of the coders—that took six years. And we released that this year, and I'm so honored by our engineers and scientists who put it together and so honored by the response and the feedback. So I spend a lot of time thinking about—have spent a lot of time this year thinking about the rollup of code base health metrics.

(Joel Beasley at 00:16:47) And is that something—I've seen somewhere in the notes—is that something you offer, like a code base review?

(Matt Van Itallie at 00:16:54) Yeah, exactly. So we can come in with our own tool and run a security scan, a cyber scan, IP risk, code quality, process, and team, and due diligence is a very frequent situation for that. Also, new CTO onboarding and other places where the tooling isn't set up yet. We are definitely not a replacement to engineering tools. You need to get developers the right tools in their flow or it's just either workflow, or, you know, things don't work.

(Matt Van Itallie at 00:17:21) Our other tool, the CTO dashboard, is a roll up of existing engineering metrics. So we connect via the APIs to many of the leading, not all yet, but many of the leading engineering tools. We had never been a developer tool company. We're never going to be. We're a CTO company.

(Matt Van Itallie at 00:17:37) And so we think in good faith we can help aggregate the best tools that people are using. And so you can get a SEMA codebase health score either by our showing up and running our set of analytics on your code or by hooking up your tools and having it automatically and continuously run.

(Joel Beasley at 00:17:54) I think you should say that again because that's where everybody's mind wants to go. Everybody's mind is constantly evaluating developer tools and having similar conversations about security and all of that. And so that's exactly where my mind was going. I mean, I can tell you I read it and I can tell you that I went on your website and it all said things about CTO and these specifics, but my brain just kept telling me this is a developer tool. So this is not a developer tool.

(Matt Van Itallie at 00:18:21) Definitely, definitely not a developer tool. Thank you so much. It's, uh, we just have too much respect for the craft of coding that to not give an extremely excellent engineer experience is just not the right thing to do. So instead, one of the many tools we really like is Snyk. Don't use run SEMA on a regular basis, on a continuous basis. That doesn't make sense. Use Snyk or another great tool because they've spent so much time on the developer experience. We are a roll up. We connect to Snyk's API. We connect to Codacy's API, etcetera.

(Matt Van Itallie at 00:18:55) Letting them do their best work while we help consolidate and tell the story. I actually think that's one of the reasons that, in our view, there still is a need for CTO metrics is that if your first principle is make sure the developers have an awesome experience, my god, you're spending so much time getting that right, and it's a never ending journey. And to do it in more than maybe two different, maybe three different subsectors is possible with giant engineering teams. But after that, it just becomes too hard. And so we help, you know, probably 100,000 engineers are working on engineering tools that SEMA's CTO tool connects to.

(Matt Van Itallie at 00:19:40) They get to do their excellent work. And, frankly, the hard part of being in the CI/CD pipeline, we are not the engineer. We're not the dev tool. We're the CTO tool that connects on top.

(Joel Beasley at 00:19:50) That's interesting because a lot of the CTOs, when we're thinking about a platform or different services to use to manage workloads and all of that, the dashboards of those things are so important. But because we need such a large variety of tools, one can never, even if they try, it's like, oh, yeah. Well, that software has a version of this thing. But over here, this is all PagerDuty does. Or this is all this company does, and they do it much, much better.

(Joel Beasley at 00:20:18) So we're not going to use that feature. But then now we've just, you know, bifurcated our dashboard data for lack of a better term and this idea. And then if you go over to the tools that are just pure dashboard tools, they're missing that layer of logic and that understanding of the context of the specific data. And so you have this gap in the market. And, yeah, I'm available if you want to bring me in when you guys are raising money.

(Joel Beasley at 00:20:45) Okay. But amazing, absolutely. But I see it. I see it. It's really clear.

(Matt Van Itallie at 00:20:49) Amazing. Thank you for pitching me a softball to talk about context. So the reason we didn't, I'm getting goosebumps. I love talking about context. The reason, I mean, I'm the son of a math teacher and a computer programmer. I could have built a rubric in 2017, but it wasn't going to be right. A trillion dollars of code later, and we publish the methodology that we have. It's not unlike the AI detection, which does have to be secret sauce, we're literally explicit. You can go to our website and read it. There's a quiz you could take on your own of these metrics and see how you score. If you want the research paper, we'll send it to you.

(Matt Van Itallie at 00:21:27) But a tiny example, you have a codebase that's 20 years old. Codebase is 20 years old. A thousand developers have worked on it. If you have 100 high-risk SAST or DAST warnings, so warnings within the code itself, then you've done something magic. You must have a tool like Snyk or Veracode, shout out to both of them and there's several more.

(Matt Van Itallie at 00:21:53) And you must, asterisk, you must give your engineers time to work on it. You could literally not have only 100 high-risk security warnings if you're that big and that old if you didn't have a tool and you didn't give engineers time. Here's the asterisk. Everyone who's not a CTO who's listening, the most important question if we're ever doing a diligence, and so if you ever run into us, we'll ask is how much time does sales and the CEO and finance and product give to engineering for non-functional improvement? It is not the engineering team's sole responsibility.

(Matt Van Itallie at 00:22:31) It's the organization's responsibility to work on this. I've never met a team that didn't want to invest more in making the code safer or higher quality. There's just constraints. So I really hope those of you who aren't CTOs listening, you have an honest conversation about the trade-offs of investing more in security debt reduction and tech debt reduction. Thank you for that speech.

(Matt Van Itallie at 00:22:52) So if your company is 20 years old and 100 high-risk security warnings, you have a tool and developers are allowed to work on it. If you're a one-year-old company and you have 100 high-risk security warnings, then no. There's really nuance. If you're a year old, you're probably working on product-market fit. And unless you're in a massively regulated environment, it would be understandable that you haven't done any investment in security warning remediation in the first year, because until you make it, it doesn't matter your non-functional requirements if you're not going to make it.

(Matt Van Itallie at 00:23:26) Two-year-old company, or a three-year-old company with 100 security warnings, that's probably not in the right position. By that point, you should have fewer warnings. And so our tool counts the number of high-risk warnings. And then with our, you know, trillion-dollar dataset, contextualizes the results. So if you're 100 and what we call enterprise or giant, you get a relatively high score. And if you're not the babiest of companies, but growth, and you have that many, you get a higher risk score or a lower score overall. That context. So thanks for letting me go off on this, Joel. But the point about getting the context right, that is why this is so hard, and that's why you need the dataset. It's not just about APIs. You need the dataset to be able to say, this makes sense and this doesn't make sense.

(Joel Beasley at 00:24:14) For the generative AI. Right? There's a lot of tools out there that people are using, all different models as well to generate this type. So can you tell if it's AI generated and what the model was that generated it or no?

(Matt Van Itallie at 00:24:33) Extremely good question. We're not there yet. Big picture, our detection tool has a combination of gen AI detection itself, with some explainability so we can improve it, and a behavioral component and a feature assessment. So the behavioral component is, we have time series data. Zero coding, zero coding, zero coding, zero coding, 500 tokens of coding, you know, over a five-minute period. That is likely not, you know, there's exceptions, but that is likely not human-generated code. That is likely came in from somewhere.

(Matt Van Itallie at 00:25:13) And then you map it against the gen AI model to see how likely it was that it was copied in from open source or from gen AI. And then the last part of the triad is features. So, an early, and again, there's going to be adversarial models. This is never going to end. This is we're going to be working very hard to keep up with advances and changes.

(Matt Van Itallie at 00:25:39) But right now, in 2023, generated code has really distinctive patterns of commenting. Humans don't write comments like gen AI code writes comments. And so that's one of the features, meaning, you know, we look at how the code looks differently on top of having the model take a look. Today, that's a long answer. We don't distinguish between models, but that is, it's on the roadmap.

(Matt Van Itallie at 00:26:09) I'm sure you've thought of this, of course. The other one of the big risks for gen AI is data leakage. Right? Fancy for folks who are taking code and putting it into an open model. Our hunch is in the next, you know, one to three years that there will be incredibly strict portals on where you're allowed to do it.

(Matt Van Itallie at 00:26:30) GitHub Copilot, obviously, is one, but then also teams, companies building their own models and forcing code entering the system into a portal that they can control rather than, you know, to avoid the private code getting out into other people's code.

(Joel Beasley at 00:26:47) Interesting. You gave me a number of ideas. The first one being, I'm going to create a new app. I'm going to call it Sloth. Great. And it's going to space out the codes. You just take your generative code, you dump it in there, and then it'll space out the pasting and the copying, and it'll take care of the comments as well.

(Matt Van Itallie at 00:27:02) Exactly. Exactly.

(Joel Beasley at 00:27:03) There we go.

(Matt Van Itallie at 00:27:04) And remember, though, unlike, I mean, there's this, that assessment use case folks may try to hide. I may be an optimist, but I think the vast majority of cases, they will actually want to prove that they interacted. I mean, I guess Sloth, if they really like the code and didn't want to do any work on it, you're absolutely right. But we do think most organizations will, I could be naive, but I really believe most organizations will be encouraging of transparency and encouraging folks to use it. So it's not like they have something to hide. They just have to show how they're interacting with it.

(Joel Beasley at 00:27:40) I get you. Sloth will also steal your Bitcoin too, so be careful. Exactly. Be careful with that.

(Matt Van Itallie at 00:27:46) If we had time, I would show, I can't speak that slowly. The Zootopia. You've seen Zootopia. Right?

(Joel Beasley at 00:27:53) Oh, I have kids. Yes. I've seen it 100 times.

(Matt Van Itallie at 00:27:55) Of course. Obviously, that. You've seen it a million times. The very slow sloth.

(Joel Beasley at 00:27:59) Yep.

(Matt Van Itallie at 00:28:00) That would be your mascot.

(Joel Beasley at 00:28:03) Every time I see that, I'm like, I hate this scene, but I love this scene.

(Matt Van Itallie at 00:28:08) Totally. Totally.

(Joel Beasley at 00:28:10) Do you have kids as well?

(Matt Van Itallie at 00:28:11) I do. Three.

(Joel Beasley at 00:28:13) Oh, what ages? What age range?

(Matt Van Itallie at 00:28:15) 14, 12, and 10.

(Joel Beasley at 00:28:17) Oh, wow. So you're right in the technology adopting maturity, puberty phase. That's tough.

(Matt Van Itallie at 00:28:26) Yeah. No. They are amazing kids, and they make super good choices with some guide rails from parents, but super good choices about how they ingest content.

(Joel Beasley at 00:28:37) How much time do you—

(Matt Van Itallie at 00:28:38) They don't say the phrase ingest content. How'd they—

(Joel Beasley at 00:28:40) Use it? My dad taught me. How much time do you get to spend with them? Are they homeschooled or public schooled or private school? Like, how do you deal with that?

(Matt Van Itallie at 00:28:49) They're in a mix of schools, not homeschooled. You know, I've pared down my life a lot because I am really passionate about solving the CTO problem. Some of my loves include plants, so I have a lot of flowers.

(Joel Beasley at 00:29:05) I can see.

(Matt Van Itallie at 00:29:07) But other than that, family, my amazing wife, time with kids. We do a ton of cooking together. Some not enough exercise, and I work. I think about this. I talk to great folks. It's, you know, if you're doing what you love, it is so energizing. I'm incredibly fortunate to be able to do this.

(Joel Beasley at 00:29:29) Oh, 100%. It's like life has become a series of just time allocation amongst things that I love.

(Matt Van Itallie at 00:29:36) Exactly. That is such a gift and is such a, and you've worked so hard to get there yourself and to be able to set up your life that you're doing the things that do right by you and give you more energy and do right by your family and loved ones is, like, it's the best. It's the best.

(Joel Beasley at 00:29:52) What app do you use to track taking care of your plants, or do you know that all in your head when they need to be watered and such?

(Matt Van Itallie at 00:29:58) I use it's super secret, don't tell anybody, Google Calendar. Okay. And so I have, I do what my calendar tells me to do. And on Fridays at 5:00, I get 20 emails on home chores, and one of them is plant watering. And I give it just enough, I like to think so. I'm good at almost all kinds of plants except for orchids. Conveniently enough, my wife is amazing at orchids, so we have split the responsibilities. But, yeah, I do zero inbox. And Friday night, I get 20 more messages of individual tasks that I have to take care of by Monday morning. And if I don't do that, I still have messages in my calendar, reminders in my inbox.

(Joel Beasley at 00:30:42) My wife does zero inbox, and I do it, but she doesn't agree with zero inbox where I just turn all notifications and badges off. So I never see the calendar.

(Matt Van Itallie at 00:30:51) No. You should never, if you're listening and you have any notifications of any kind, you turn them off. Everyone listening to this is very important. Everyone's trying to get you at all moments. Turn off every single notification. You be in charge of your attention. Do I do it? Yeah. Yeah. Of course. No notifications. No noise. No freaking nothing.

(Joel Beasley at 00:31:10) I think we're going to be best friends, Matt.

(Matt Van Itallie at 00:31:13) You do this as well. Everything's turned off.

(Joel Beasley at 00:31:14) I started this as a, I did an episode, I think, two or three years ago with a guy that studied distractions and Internet and tech, or distractions and technology. And he convinced me to run this little trial where I turn off all notifications on my devices. And I said, okay. I can do something for a week. I've got good discipline. I can do it for a week. I did it for a week, and I'm like, never going back again. Never. It's been two years.

(Matt Van Itallie at 00:31:42) I can hear text messages. And our team, we take, the most important thing about SEMA besides our team is our set of values and communication principles, and sort of the hierarchy is values matter more than communication and communication matters more than the work. And so we are super, because if you have shared values and shared communication, if something goes wrong, you know, and the people are competent, you just fix it. But if communication is off, you start going in different directions, and obviously if your values are off, you stop trusting each other. And so we are in, I have a two-page set of instructions on how to communicate with me, and all of that's here are welcome.

(Matt Van Itallie at 00:32:20) It includes never use high importance. That feels like shouting. We don't need to shout. And if you're stressed out, I don't need to be stressed out. So we use lowercase. We use the phrase "time sensitive," which sounds very polite, but it means everybody drop your shit and help us out, whether it's an outage or a situation.

(Matt Van Itallie at 00:32:41) And so if someone writes me "time sensitive," I drop everything I do. And if you text me "time sensitive," I know there is literally nothing more important. And so we've done the work. We're in 10 countries. We've done the work to figure out how to communicate with each other.

(Matt Van Itallie at 00:32:56) And yeah, I don't need it. Developers, you don't need it. You have work to do. Take breaks. Take your own decided breaks.

(Matt Van Itallie at 00:33:04) You're in charge of your time, not other people.

(Joel Beasley at 00:33:07) Yeah. Let's just try to convince the rest of the world for it. It's amazing. I actually went even farther where, this might sound a little bit crazy, but after doing that for about two years, last year in January, I got a personal phone that's just a smaller iPhone. And the only people that have this number are like 10 people. It's my dad and my brother.

(Matt Van Itallie at 00:33:34) Did you say the number?

(Joel Beasley at 00:33:36) There are like 10 people. I got my dad. I didn't hear your answer on that. Oh, boy.

(Joel Beasley at 00:33:44) That would—

(Matt Van Itallie at 00:33:45) That would have made it in post-production. Yes. So you know if they're calling, then of course that's super different from work stuff.

(Joel Beasley at 00:33:52) Yeah. Well, I found that I was trying to, I was going through the evolution of the business, right, and trying to figure out how to work myself out of the business within reason. Right? That takes a long time. That takes multiple years.

(Joel Beasley at 00:34:07) And so I got to a point where I wanted to see if I could have the phone off outside of work hours. And that means if emergencies happen and emergencies get resolved without me involved, that's a good test of emergency. And so I built up all this structure. I mean, I've been doing this company for like seven years, so it wasn't like an early days thing. It was just in this past year. So it's like, alright, the next step for this business is for me to be able to, nights and weekends, not have my work phone and see if things still go smoothly. And inevitably, things break, issues happen, and then I slowly fix them over the course of this year. Now we have it at a point where the past couple of months, I've got all the right people in all the right places and everything functioning to a point where I don't have to deal with work phone.

(Joel Beasley at 00:34:54) And now, here's the catch: my wife is in our team Slack. So if the emergency really does happen, they're going to get at her through Slack, and she's going to bother me because she keeps her phone going. So they know that there is a way to get me, but they also know the experiment that we're running.

(Matt Van Itallie at 00:35:12) Alright. So you need a family KPI, and that is every month, what is the longest period of time that neither of you is interrupted by work? And the goal is, easy for me to say, we've just met, but long lost brothers, by, let's say, 2024, a whole week you have turned off, both of you. Because we now know the secret backdoor, both of you are not checking messages. And so can you do 12 hours with none of you checking messages?

(Joel Beasley at 00:35:39) She'll never know. She is—

(Matt Van Itallie at 00:35:42) Come on.

(Joel Beasley at 00:35:43) She is full-on openly addicted to her phone. Okay.

(Matt Van Itallie at 00:35:47) She's addicted to plants? How about Peepshe? Do you know what Peepshe are? No. Okay.

(Matt Van Itallie at 00:35:53) Well, you're about to get your mind blown. Sounds like Peeps. Also sounds like sushi. You stock up in March, as many Peeps as you can. You buy Rice Krispie treats or make them yourself, but you don't need to be artisan.

(Matt Van Itallie at 00:36:08) And then you get Fruit Roll-Ups and Nerds. And the Peeps become the fish. The Rice Krispie treats become the rice. Fruit Roll-Ups become the seaweed, and Nerds are the roe. I highly recommend everyone immediately giving up on all of your career dreams and aspirations and saving March and April to only work on Peepshe and mastering your craft, which I have done now for 10 years.

(Matt Van Itallie at 00:36:34) It's for the kids. Yeah. No. It's for me to get better and better at making Peepshe. So maybe we can try an experiment: a little bit less phone time and a little bit more Peepshe.

(Joel Beasley at 00:36:46) I'm going to send you my phone number after we get off here, and you can text me some pictures. I want to see these things. Yeah. I want to see these things. Yeah.

(Joel Beasley at 00:36:54) And alright, in all fairness for my wife, I made her kind of sound like a crazy person. And if I said that she's a crazy person, she would smile, like, "Yeah, a little bit." She spends 80% of her phone time on a group chat with her mom and her three sisters.

(Joel Beasley at 00:37:08) So it's mostly just communication on family and what the different families are doing and the nieces and the nephews and the husbands and all of that. But yeah, she does have access. You know what? It's actually pretty funny. The amount of times that people have gotten to her through Slack is like, I would say it's zero. The coolest things that'll happen that she'll see and she'll tell me about is like when someone closes a deal, they'll post it in the office channel on the Slack, and she'll be like, "Oh, so-and-so closed the deal." I was like, "Oh sweet. I wasn't going to see that till Monday, but that's good news."

(Matt Van Itallie at 00:37:42) Good news. It's good to be interrupted with good news. Bad news, preferably someone else can deal with it. Yeah.

(Joel Beasley at 00:37:47) So what's the current stage, and what's the next stage for your business?

(Matt Van Itallie at 00:37:52) Yeah. We are very fortunate with our current customers to be self-supportive and investing in our new products. But we're definitely fundraising next year because to do this right, both the core CTO metrics and the GenAI monitor and the VS Code widget and the GitHub plugin, it's serious engineering time. So we'll be fundraising next year.

(Matt Van Itallie at 00:38:21) I'm so happy to be with my team. I could not be more excited. I could be more excited that we could spend more time together, but we're in 10 countries, and so we can't connect that often in person. But just continuing to add amazing folks and just build—sometimes we build hard things like the AI code monitor, and sometimes we build important things. API connections, huge shout-out to Mark.

(Matt Van Itallie at 00:38:49) Right? It's not technically hard, but it's so important to get it right. And to be able to work on those kinds of puzzles all day and try to make it as correct and as useful, it's just such a blast. It's such a blast.

(Joel Beasley at 00:39:01) Yeah. What you're doing is actually really cool. The distance from zero, knowing nothing about your idea, to understanding this—like, oh, I mean, for us, it's HubSpot. Like, they should have a sales HubSpot-like thing for the engineers. And it doesn't make sense just to look at your Atlassian dashboard because of A, B, and C.

(Joel Beasley at 00:39:20) And there is this market because if you tried to get a general dashboard, it doesn't have the context of understanding what this type of data actually is and some beautiful little tools on top of it. And it's a really easy picture to paint because the pitches happen so fast when raising money. The pitch is, you have to catch them and hook them right away.

(Matt Van Itallie at 00:39:39) Salesforce is a salesperson tool. Someone's typing into Closed Deals or HubSpot, shout-out. Right? Salesforce is a sales manager tool where they're looking over their team and seeing how they can be helpful and seeing if there's blockers. But it's also a CRO tool. There is, you know, to manage across and to manage up.

(Matt Van Itallie at 00:39:58) There is not—and so now go to our world—literally infinite number of engineering tools thanks to open source. At least 1,000. At least 250 great engineering manager tools, which also obviously, like Snyk, is in that as also an engineer tool. At least 250. You tell me, what is the CTO tool?

(Matt Van Itallie at 00:40:19) It is combining spreadsheets. It is having an engineer put it together. I read this good book. You might want to read it. It's called Modern CTO.

(Matt Van Itallie at 00:40:29) And in it, the author said that it took him about 10 years of being a CTO to really be great. And I think he said it was the first five years to not be an idiot, and then another five years to get to excellent. Folks who have done it for more than 10 years, they don't actually need this. I mean, is it helpful? Sure.

(Matt Van Itallie at 00:40:49) Is it good to have benchmarking across? Sure. But you learn the craft, you figure out a way to do it, you have your methods, you have the relationships. But folks early on, oh my God, is this helpful? And I think of a client who I'm super proud of. I will not say his name.

(Matt Van Itallie at 00:41:04) He just took over the CTO role, and he had a choice. They bought a company that had a lot of technical debt and security debt, and he had a choice of whether or not to tell his board and show them the data or to not tell his board. And he did the hard thing, which was to tell him, which was the right thing. Because of course, all of us who've been in this world, if you freaking hide that, the best possible time to share bad news is the moment you get there or the moment something happens, because the longer it waits, the more it accrues to you. And super easy for me to say.

(Matt Van Itallie at 00:41:36) I've made so many mistakes. I've lived a long life. Right? I've learned that the hard way. But this guy, I love you, man. He did it right. He showed up to the board meeting and said, "It's going to cost some money to fix." Right? And it doesn't hurt that explaining the results in terms of dollars and cents and saying the reason that we have to do this is because we are literally the bottom quartile of this business compared to other businesses in our space. Right?

(Matt Van Itallie at 00:42:03) But the hard part is the courage. It's the courage to have a conversation. The data just helps. And so, do I, of course, do I think everyone should use it? Of course. I think everyone should use a CRM. But in the early days, folks who were in their zero to 10 years, they're the ones that it's the most needed.

(Joel Beasley at 00:42:22) 100%. It's so much harder to be kind than nice. Right? Yeah. Somebody explained to me—

(Matt Van Itallie at 00:42:29) That's great.

(Joel Beasley at 00:42:31) I didn't make that up. Someone said it to me. But I'm that guy. Just like a couple weeks ago, I was walking through the grocery store and one of the managers at the grocery store was doing something with a box, walking down the aisle, and you know, the nice white shirt and everything with the tag on, and his fly was down. And I was like, "I don't know how to say this exactly, but your fly is down." He was like, "Thank you, thank you, thank you so much." And it's so much easier just to walk by and be like, "That guy's fly is down." But it's such a good human-to-human thing to be like, "Hey, look, this is difficult for me. It's going to be difficult for you, but your fly is down." And then they love you for it, and they're super, super grateful for it. And I am that guy.

(Joel Beasley at 00:43:09) I will tell you, if we're hanging out, eating dinner, Matt, if you got food in your teeth, I'm not going to shout it and point, but I'll be like, "Yo, Matt, you've got food in your teeth." You know? Because I think that's what I want people to do for me.

(Matt Van Itallie at 00:43:20) That's exactly right. Right? The standard, the golden rule is really right here. If you would want to know, then you can, and you've done a little bit of thinking about that person's wants and needs. If you're aligned with how you'd want to be treated and there's nothing obvious that's—I know this sounds like a rule, but I think it's, I think you're absolutely right.

(Matt Van Itallie at 00:43:38) If there's nothing obvious about why they wouldn't want that, then you can sleep at night knowing you treated someone how you wanted to be treated. How they react is up to them. It's out of your control. But if your intentions are right and you're honest about them, you can sleep at night while, you know, obviously trying to be the most kind, not the nicest, which is an expression I invented and copyrighted.

(Joel Beasley at 00:44:02) Yes. Thank you. I'd like a limited license to be able to use that.

(Matt Van Itallie at 00:44:07) You dragged it now? No.

(Joel Beasley at 00:44:08) I'll give you credit. I'll give you credit.

(Matt Van Itallie at 00:44:09) Yeah. No. Don't lie. Come on. You just made it up.

(Matt Van Itallie at 00:44:13) Every time anyone on earth says, what is it? "It's easier to be nice than kind."

(Joel Beasley at 00:44:18) "It is easier to be nice than kind." Yeah.

(Matt Van Itallie at 00:44:20) Isn't that the quote? So anytime, yeah, all of your listeners, you ever say that ever again, you want it in pennies, right? Send Joel envelopes of pennies, 99 cents to ship it in any currency.

(Joel Beasley at 00:44:30) I prefer foreign currency. Oh man. This is good. Where do you live? Where are you at?

(Matt Van Itallie at 00:44:37) Baltimore, Maryland.

(Joel Beasley at 00:44:40) Baltimore, Maryland. Is it cold there yet?

(Matt Van Itallie at 00:44:43) It is. We had a frost already.

(Joel Beasley at 00:44:44) I'm down in Nashville.

(Matt Van Itallie at 00:44:46) You are outside of Nashville?

(Joel Beasley at 00:44:47) Yep. Yeah. Yeah.

(Matt Van Itallie at 00:44:48) Yeah. You live on a farm?

(Joel Beasley at 00:44:50) I do. A little farm. And we don't have, other than the kids, we don't have any livestock currently. We've got the kids. Our, I don't talk about it a lot, but our third child has Down syndrome, so he has special needs.

(Joel Beasley at 00:45:05) And so we've been kind of taking that a day at a time. He's about a year and a month now. And when we moved here, that's, we moved here about a year or two ago, we were like, "Let's have our next kid and then get animals and get the livestock and all of that," because we have the space and the setup for it. But then we're kind of taking this thing one day at a time. Luckily, he's progressing super fast, top end of where he should be and everything. So we couldn't be more blessed there. There's no health concerns.

(Matt Van Itallie at 00:45:34) That's so wonderful. Yeah. And I imagine, I don't know actually, is having a lot of outdoor space on the farm helpful? Makes it easier because all of your children, I suppose, can roam and have some freedom.

(Joel Beasley at 00:45:46) Oh yeah. Yeah. We used to live in a cookie-cutter neighborhood, and we were always scared because the kids could make it to the street before we can get them, like because of the driveway and the garage and everything. And so we would always be a little nervous about that. But out here on the farm, I mean, they have to run like an acre before they get to a dirt road.

(Joel Beasley at 00:46:03) Right? So yeah, the space helps oddly enough. We homeschool them too, so we do a lot of outdoor stuff with them. But it helps me a lot because I built, and feel free, if you're in Nashville, let us know, we have an 1,100-square-foot studio on the property. It's like a separate building. Come on by and we'll do an episode in person. But I walk back and forth throughout the day for lunch and such, and it's like a forest and a mountain and beautiful and everything. So that little 30 seconds to one-minute walk back and forth every day, it resets you internally.

(Joel Beasley at 00:46:36) That's why you're all around those plants all the time, because that's got to help.

(Matt Van Itallie at 00:46:39) Exactly. Yeah. Exactly. So did you—is there a connection? I don't want to pry, but it sounded like you figured out in high school that the formal system wasn't giving you what you needed. Is there a connection between that choice and choosing to homeschool your own children?

(Joel Beasley at 00:46:56) I hated school. I was not.

(Matt Van Itallie at 00:47:00) I said that too long. Let me try it again.

(Joel Beasley at 00:47:01) Yeah, cool.

(Matt Van Itallie at 00:47:02) Seemed like from reading your book that you hated school. Is that true? Yes. Okay. And is that why you've chosen to homeschool?

(Joel Beasley at 00:47:09) Well, it wasn't the original reason. So originally we put them in one of those kids—what are the Montessori—Montessori schools, which I used to think was Catholic because I had one friend who went to Catholic school. It sounds it, right? But it's not. It was a lady whose last name was Montessori who had an education system for kids. So they were in that for like a year, and then we did this trip where we sold everything. We bought an RV and we traveled the U.S. for a full year, like a year-long camping trip. And in that act we started homeschooling them just because we couldn't get them in Montessori school, right? Because we were just always moving and going around visiting and things. And so we just started doing like Abeka, or there's some homeschool program. We started doing the homeschool with the kids. We bought them these worksets. And then when we did get a full-time permanent place, we had the choice of kind of cookie-cutter neighborhood and being close to a couple of those schools, or like 20 minutes from everything and not. And so we just said, well, let's just keep homeschooling the kids and see how that goes. And like most of these crazy adventures that we do, we go into it—my wife and I are best friends, we're a team—and we go into it with, like, we'll do it until we don't want to. Like when people, all of our families, like, "You're gonna go live in an RV? This is crazy. Your whole life you're gonna be?" I was like, we're gonna do it until we don't want to. And how long are we gonna homeschool the kids? We're gonna do it until we don't want to. It's like, that's just how we do things.

(Matt Van Itallie at 00:48:40) We haven't yet had time to have some family adventures like that, but when I was younger, my dad and I bicycled cross-country from Boston to Seattle. And I had a friend of mine—we hiked the bottom third of the Appalachian Trail from Georgia to Central Virginia. You'll see the segue in a second. Both were hard. Both took a lot of work, but it was actually more—I was more courageous stopping walking the AT because I had had enough. I'm a pretty goal-oriented guy, and to say I have hiked as long as I've wanted to hike and step away, rather than, oh my God, I need to say that I finished it so I can be a through-hiker, that was a huge growth moment for me. So it totally resonates. We'll run this experiment as long as the experiment works, and then when it stops working, we'll do something else, rather than—should is like such a crazy word, and it has to be unpacked at every moment. Why should you do something? Who said? What's the reason? Understand it. Be you. Understand the reasons behind it and then make your own, or you know, you and your wife's partners make those choices. I do not like the word should, unless we can have a conversation about the why underlying it.

(Joel Beasley at 00:49:53) 100%. And yeah, there is this one book. I didn't read it, but it had a cool cover. It's like "Should versus Must" or something like that. Have you read that book? Because I haven't.

(Matt Van Itallie at 00:50:04) I have not, but it sounds great.

(Joel Beasley at 00:50:08) And there are—to build on top of that, I would say that there are a handful of times that I have run experiments where the purpose was not to quit. It wasn't to have the achievement to talk about it or to show off about it or whatever. Like the purpose was, I'm gonna go into this and I'm gonna do something super, super difficult. And the entire point of me going into this from the beginning is not to quit.

(Matt Van Itallie at 00:50:36) What's an example?

(Joel Beasley at 00:50:39) The podcast.

(Matt Van Itallie at 00:50:40) Come on. You actually have to say it. The podcast, really?

(Joel Beasley at 00:50:43) Yeah. When I went into this, my wife and I were talking. I was looking at an AI company or investing in AI-type stuff, or investing in the podcast. And when I went into it, I said, okay, this next thing that I go into, I'm going to let it kill me. I said, that's what I'm gonna do. I go into it with—I had heard somebody give a speech about finding what you love and letting it kill you and, like, making that a conscious effort. Like, I love music, so I'm going to dedicate my life to music. And ultimately, in hindsight, music will have killed me in some regard. And I was like, that was kind of poetic. I was like, alright, so I'm gonna find something. And whether I like it or I don't like it, from before I even started it, I went into it saying no matter what I will—even if I don't make $1, I'll be on my deathbed still having done a podcast, just because I have this feeling that I think it'll work if I say that no matter what, I'm never going to quit. And then, you know, it was self-sustaining growth income within three years. And it's all sorts of amazing things. We're north of 700 episodes now, and also it actually happened way faster than I thought. I was like, maybe I'll hit it when I'm in my fifties.

(Matt Van Itallie at 00:51:59) So if you were giving career advice or life advice, let's just say life advice, how would you encourage someone to tell whether or not this is an experiment they should run until they decide they don't want to do it, or it's an experiment they should run where they should let it kill them? Like, how would you help someone make that decision?

(Joel Beasley at 00:52:19) I would say if you're going to make the decision of choosing between those things, to do it before you go into the experiment. Like, so just as long as—if before you go into the experiment, choose it. Like, don't say I'm going to try podcasting and see if I can get it to work, and I'm gonna science podcasting and try different things to see if I'm gonna get it to work. That's one thing you can do. You can do that. But if you're gonna choose to do an experiment that you know you're not gonna give up on, then you just have to go stand in front of the mirror and make that choice.

(Matt Van Itallie at 00:52:56) And when you guys did your year in the van, it was we are deciding we're gonna do it for as long as it's fun.

(Joel Beasley at 00:53:03) Yep. As long as we want to do it. We don't like it, we'll make another choice.

(Matt Van Itallie at 00:53:07) How many times in your life have you done—do you have a count of how many times you said I'm gonna do this till it kills me?

(Joel Beasley at 00:53:12) Oh, absolutely. Two tasks. One, learning how to walk again after I got hit by the car.

(Matt Van Itallie at 00:53:16) Yep.

(Joel Beasley at 00:53:17) And two, the podcast.

(Matt Van Itallie at 00:53:19) How about getting back into shape?

(Joel Beasley at 00:53:21) I didn't—I mean, no. I didn't have it at that point in time. I'd say consciously only—I mean, I know I did it with the leg stuff, but I didn't really have the same frame of reference. I mean, I was 13, so like I knew I had to do it. But I'd say consciously, then it would only be the podcast as, like, an adult. Because, I mean, you're not like really thinking straight as a 13-year-old.

(Matt Van Itallie at 00:53:49) You are not. No. No offense, Austin or Jane. I love you, but probably not. I put my heart and soul into something else before I did this, and it was I'm trying to help improve urban education, in low-performing school districts. And I gave it everything I could for about six years, proud to have helped some more teaching and learning happen in some places at least. But in the end, I wasn't the right person to be in the space, and that was a really hard decision because if I have to pick one topic that moves me more than anything else, it's helping people realize their full potential and, subset, pulling people out of poverty through education and other means. It's the most important thing. Those are the most important things. But I actually wasn't able, for a variety of reasons that are, you know, my own fault primarily, to have enough impact, and I was just at war with myself over it. You know, the best, the absolute best improvement in an urban school system is 1% a year continuously, and that would be astronomical to be able to do that for five years. Astronomical impact for the kids and for the long term. Actually, I'm not wired to make the kind of incremental progress. So the best I can do is work in areas where I can move at my pace, and then give back to amazing education leaders and teachers, included, obviously, who are doing that. But I almost let Ed reform kill me, and I have finally had to say no.

(Joel Beasley at 00:55:30) Yeah. That can crush your soul, the education world. Yeah. And it's important too when you're picking something like this, if you're gonna—just to talk about it, I guess, a little bit more. But through the act of doing super difficult things, you get to know yourself a lot better. And as you get to know yourself a lot better, you stop caring about what other people think as much, and you start trying to just have peace within yourself. And then you learn all sorts of stuff. So even if I—I don't know where I was going with that. But with the podcast thing, I definitely pick something I could control. I know everyone says be, like, super, super specific, and I was specific. But I would say after this conversation that we're having and after several hundred conversations, the difference between the successful people—the advantage that they have—is they all seem to have some sort of system that they stick to, but it's almost entirely different and unique to each individual. They all have something they focus on or something they stick to, but it's just that act of trying, I think, if you're gonna bring it up to like a heady-type conversation. I think the universe respects persistence and your ability to continue to try and to get back up after you get knocked down.

(Matt Van Itallie at 00:56:55) Well, a friend of mine told me about a book cover that he looked at. "A System Is a Must," not a should, which I hopefully the book is about that. If not, we've made a terrible mistake. I love that very much. It reminds me—I don't remember where I saw, but I'm sure you've seen it too—that literally many of the most successful people have a uniform. So, it is one less thing of your brain space to have to worry about. And second, it's a system, right? You begin your day or in the previous day with, this is what I'm wearing, this is just one less thing to think about and to get you back into the thing that you do best and can make the most contribution with.

(Joel Beasley at 00:57:39) It's got branding benefits too. I call it the comic book character, because the comic book character is always wearing the same thing pretty much. Like, you can always recognize that character.

(Matt Van Itallie at 00:57:48) Black t-shirt on over a long sleeve shirt.

(Joel Beasley at 00:57:51) I didn't say I do it.

(Matt Van Itallie at 00:57:54) Exactly.

(Joel Beasley at 00:57:55) Dude, this has been fantastic.

(Matt Van Itallie at 00:57:58) Such a pleasure.

(Joel Beasley at 00:57:58) In Nashville area, we should definitely do a—

(Matt Van Itallie at 00:58:02) I'll come down to see you. It would be super awesome. It would be so fun.

(Joel Beasley at 00:58:05) Thank you so much for listening. And if you found this episode useful, please share it with a friend or colleague who you think would get value from it. And if you have topics that you'd like to hear discussed on the podcast, either add me on LinkedIn or send me an email, [email protected]. Every time I get an email or LinkedIn message, it absolutely makes my day and inspires me to keep going.