Where Do We Draw the Line on Letting AI Build? With Zach Goldberg, CEO at Gruntwork

(Intro Narrator at 00:00:00) Today, we're catching up with past guest Zach Goldberg, CEO at Gruntwork, about all the intricacies of where AI use is best suited and lots more. You're listening to Joel Beasley, Modern CTO.

(Joel Beasley at 00:00:18) Well, I'd love to talk about video games, but I think we're here to talk about CTO stuff.

(Zach Goldberg at 00:00:22) Yeah. We could. Game companies are technology companies. They've got really interesting challenges. The timelines, the crunch, the working with the artists. I can imagine that is a really hard problem.

(Joel Beasley at 00:00:32) You know, we did a—this is the cool thing about the show going for ten years. Five years ago, we did a series with some different game developers, I think Activision or Blizzard or something.

(Zach Goldberg at 00:00:44) And how their world must have changed as well in the past year and a half. Can you imagine—I don't know, this maybe segues a little closer to my neck of the woods—there's such a public backlash to the use of AI in games. Right? Steam has a disclaimer: AI was or was not used in the production of this game. Even if you used it to make one asset, now your whole game is tainted and some percentage of your audience is no longer interested in your product because of a shortcut you used to make some inconsequential thing. The whole game has changed. The world we live in now is totally different.

(Joel Beasley at 00:01:22) Where did they draw the line? Because you can apply AI so broadly if you would like. Before LLMs, there was major progress in the auto-generation of worlds happening. Now I don't think they would consider that AI.

(Zach Goldberg at 00:01:42) In this particular case, I think it was Steam, Valve, that published guidelines on it, and they did draw a very clear line. It was something like, if you use generated content to plan or to conceptualize, that's fine. But anything that's—I'm probably misquoting this—but anything that's actually shown to a user had to be created by a human. Procedurally.

(Joel Beasley at 00:02:08) That's right. You can't—yeah. It's just honestly, I think it's—I don't know. I think we're about to enter a bit—when do you think that's going to become political, by the way? Let's make some future bets, because everything becomes political.

(Zach Goldberg at 00:02:21) Is it not already? Fable is not available to the world because somebody in the Trump administration said something to Anthropic and they pulled it out. Right?

(Joel Beasley at 00:02:31) Did they really?

(Zach Goldberg at 00:02:33) Are you not familiar with this? Yeah. So Anthropic's latest model called Fable, which is the next version—

(Joel Beasley at 00:02:38) Access to it for a day.

(Zach Goldberg at 00:02:39) Yeah, exactly. Yeah. And then Anthropic—it must have been public for twenty-four hours, forty-eight hours, something like that. And then somebody in the Trump administration—at least this is how Anthropic tells the story, and I have no reason to believe that's not the case.

(Joel Beasley at 00:02:53) Read it from them, not an article about it.

(Zach Goldberg at 00:02:55) Anthropic themselves is directly pointing the finger at the U.S. government, saying they asked us to turn it off. And so it is still off a week or two weeks later, presumably for some political reasons. Yep. There it is. I'm sure that one more will tell you all about it.

(Joel Beasley at 00:03:12) Yeah. Interesting. Okay. So—and, wait—

(Zach Goldberg at 00:03:16) So we're here. Politics and AI are one and the same.

(Joel Beasley at 00:03:20) Okay.

(Zach Goldberg at 00:03:20) It's in the hero headline: Statement on the U.S. government's directive to suspend access to Fable 5 and Mythos 5.

(Joel Beasley at 00:03:28) Yeah. I'm curious because it's always nuanced. Right? There's always details. And so I'm always curious to know—okay, I wonder what the case was. Did they share that? Did they say, "This specific use case is why," or did they just say ambiguously that—

(Zach Goldberg at 00:03:47) I was just speculating.

(Joel Beasley at 00:03:49) I don't know. I'd be guessing. Yeah, I don't know.

(Zach Goldberg at 00:03:51) I don't know if they did.

(Joel Beasley at 00:03:53) Yeah. There might have been something serious. You know?

(Zach Goldberg at 00:03:56) I think what I find really intellectually fascinating is—I like your point—nuance is the name of the intellectual, almost philosophical conversation we're having right now. You know, as we started a minute ago, AI could be banned because, "AI bad," because some ecosystem governance body was like, "Oh, we don't want AI," or whatever. Or it could be fully adopted, and our company uses only AI to write code. And the reality is the correct answer is, well, there's nuance and shades of gray and different things to take into account. And I find it exciting because we have not yet evolved systems and frameworks and language to describe that nuance that we all share. Right? So what is an acceptable usage, and what are the consequences of that? And how do we talk about this? It's a very actively evolving culture, sort of as we speak. I don't think anybody really has the right answer for that yet.

(Joel Beasley at 00:04:51) No. You're exactly right. The words will mature. We'll learn the discussions that are important. We'll find the dividing issues. We'll find the common issues. For example, I like to go back to the knife.

(Zach Goldberg at 00:05:06) Mm-hmm.

(Joel Beasley at 00:05:06) Right? Because you can use it to kill someone. You can use it to save someone's life. It can be very controversial. It's banned in certain places, but allowed in others. There's versions of it that are banned at the airport. You can't get through security with a bladed knife, but inside the airport—

(Zach Goldberg at 00:05:21) Right.

(Joel Beasley at 00:05:21) There's—auto-closing knife, not auto-closing. There's all of these—the concept of a knife exists in all these various different contexts with all of the stuff around them. The size of the knife matters. A pocket knife can be so many inches, but if it's longer than that, it becomes this other thing. It's just—there's so much to it. But that's old technology. So as humans, our generation, we grew up with it. That wasn't even really controversial. It's just like, "Oh, this is how it works."

(Zach Goldberg at 00:05:50) You just know you don't bring a kitchen knife onto an airplane. It's just a thing you know.

(Joel Beasley at 00:05:54) Exactly. But then again, if you're in the sixties, you're bringing a pocket knife on the airplane because you have to carry a pocket knife because you need it for various activities throughout your day, and no one even questions it. So for me, I'm interested in looking at that analogy, seeing how far we could take it, how applicable it is. But then also, I feel like we're the adults when the knife was coming about. You know? And then our kids are us. They're the generation below that's just going to grow up and be like, "Oh, this is just how it is." So it's like, how can we be good stewards of this and do the right thing? And the answer is it's incredibly unclear, and it's difficult, and we're probably going to screw it up.

(Zach Goldberg at 00:06:39) Yeah. And I think, you know, I don't work at Anthropic or Google's AI division or whatever. So you have to imagine, my concern as the person who doesn't work at these companies is the adult in the room right now is—we're sort of depending on the employees of these companies to make good decisions about safety, about governance, about when is it a good time to release Fable and Mythos, so on and so forth. And the concern as an external party is, where's the incentives? Right? Anthropic is in a race with OpenAI and Google and these other companies making these big models. So there's a capitalistic market incentive going on to be the best, to be the best price, to get the adoption, get the users, whatever. Right? But at the same time, the rest of the world is looking at them saying, "Please don't screw this up. Let's make sure these things are not getting into the hands of the wrong people and causing major security exploits or whatever other bad things happen—Skynet, in the future—with the direction AI is going." And that feels very tenuous.

(Joel Beasley at 00:07:42) Are you of the mindset that we should have 100% unmoderated models and they—Anthropic shouldn't have pulled Fable? We should just let everybody have access to as much information as they possibly can? Or are you like, there are situations where we should limit it?

(Zach Goldberg at 00:08:04) You know, it's an interesting—I don't have a clear black and white answer to that question personally. Right? Nuance. I don't know what that is.

(Joel Beasley at 00:08:13) Yeah. Because my default is freedom. But then there's—there are definitely situations—

(Zach Goldberg at 00:08:20) A crowded movie theater.

(Joel Beasley at 00:08:21) Yeah. Right?

(Zach Goldberg at 00:08:22) What is the equivalent here? What is the—our social contract says we agree to bind ourselves to certain limitations that we all agree are reasonable. Right? I don't know what the equivalent of fire—obviously, I don't want a terrorist nation having access to the most powerful models that can exploit all of America's financial systems or power grid systems. Right? We both probably agree that would not be a good thing.

(Joel Beasley at 00:08:45) Right.

(Zach Goldberg at 00:08:45) But how to do that, it's not quite as simple as saying, "Don't bring a knife on an airplane." Right? Once it's out there and it's available, how does Anthropic or OpenAI, whomever—it's not like deep state or—terrorist nations are going to raise their hand and say, "I'm a terrorist. I want your AI."

(Joel Beasley at 00:09:01) I think one unifying area that we can just discuss a little bit is, you know, our world is designed where you and I operating outside in reality, going about our day—there is all the people that exist from just got out of prison for murder, just got out of the mental institution, all the way up to prize-winning physicist. We've got this huge spectrum of people, and I'm constantly surprised when I see different signs that say things. Because I saw a sign the other day in the bathroom that said, "Don't flush diapers." How many people were like, "Yeah. This is a good idea to flush a diaper," before they went through the effort of printing up a sign? There was a metal sign too. It wasn't just a piece of paper. It was—they had a metal sign that was bolted into the wall. And so I'm like, "Yeah. Those people exist. They're out there walking around too." Now I don't want the super-intelligence or whatever helping them execute some of their ideas. You know?

(Zach Goldberg at 00:10:05) The reality is the filter at the moment is not on the user. It's on the action. Right? If you ask it, "How do I make a bomb?" it says, "I can't help you with that, Steve." Or what—that's the guy's name from Space Odyssey. Hal. Yeah. No, thank you, Hal. But if you say, "You know, what was the backstory of the French-American war?" Okay, it'll tell you that. And so—but that's tenuous. Right? Trivially, the other day—true story—I had a bank statement that was encrypted with a password. My bank. Genuinely my bank statement. And I didn't remember the password that the bank used to encrypt the PDF and I didn't feel like calling the bank and sitting in a phone tree for an hour to figure out how to unlock my own PDF. And so—I happen to know that there are command line tools to crack PDFs—and I was like, "All right, it's my data. Just give me my own data." So I asked Claude, "Make a command line to crack this PDF." And Claude said, "No thank you, I can't do that." And so I Googled, "What are the common CLI tools for cracking PDFs?" And then I asked Claude, "Run this tool in parallel." And it said, "Sure. I'll write a batch script to run the tool in parallel for you." So the circumvention is very nascent and still seems not bulletproof at this point.

(Joel Beasley at 00:11:16) Right. But the limiting factor is your intelligence. So I think that's a—I don't even know if I have a thought. It's an interesting thought that if you're smart enough, you can get access to it. If I wanted to build a bomb, I could download a model, crack it, you know, break it. There's different versions of them that have different levels of security. Get the model to do what I want locally, and then have it assist me in whatever nefarious thing I wanted to do. That is complete—but the skill level it takes to be able to do that and execute that is high.

(Zach Goldberg at 00:11:55) So you're saying we prevented not smart people from doing bad things, but we've made it even easier for smart people to do really bad things.

(Joel Beasley at 00:12:03) Psychopaths are probably going to become pretty empowered right now.

(Zach Goldberg at 00:12:08) Or, you know, nation-state actors.

(Joel Beasley at 00:12:11) Like I said, psychopaths. Yeah.

(Zach Goldberg at 00:12:13) No kidding. Fair enough.

(Joel Beasley at 00:12:15) No. But it is very true. I think we forget because we live in such a—you live in the United States. Right?

(Zach Goldberg at 00:12:20) I do. Yeah.

(Joel Beasley at 00:12:21) Yeah. We live in such an amazing country, and we live in a first-world country. And there are actively entire countries of people waking up every day trying to end us. That is a reality that we are so well-insulated from. We don't think about it on a daily basis.

(Zach Goldberg at 00:12:37) That's a really powerful frame. Right? They wake up every day trying to think, "How do I cause harm to maybe it's us or it's another country?" And then you put that in the context of these capabilities.

(Joel Beasley at 00:12:48) They could get a VPN and then use Fable. And now they have this assistant that—because they're limited by their access to intelligence. That's a huge wealth thing, by the way. Kings have always had that. If they wanted to do something, they could just summon the most intelligent person from their kingdom and talk with them and have them execute it. All now kings. We are all now kings. We have all of this access.

(Zach Goldberg at 00:13:12) It's a $20 a month OpenAI subscription, and you are a king.

(Joel Beasley at 00:13:15) And you don't even have to pay that. If you have a Mac mini, you can just—you just download local. They've made it so easy. Now have you played with LM Studio?

(Zach Goldberg at 00:13:24) I have not personally. No.

(Joel Beasley at 00:13:26) Oh, man. You just—it's one-click install on Mac. You select your model. Instantly runs it, boots it up, runs it as a server so you can run it with OpenClaw. So you're just using all local stuff for all its tasking if you want. It is absolutely fantastic.

(Zach Goldberg at 00:13:41) Yeah. Now I haven't done all that much in my personal automation, but I am very doubled down on at-scale code generation. At scale, compared to—what's your key piece? What's your stack?

(Zach Goldberg at 00:13:54) It's, for the most part, Claude. Just straight Claude. A little bit of the—you know, Claude Projects. And then it's homegrown tools to stitch together things that look like Claude or Vertex AI and other important softwares in our ecosystem, is where I have found lots of enterprise value being generated. Getting, you know, we have a CRM, for example.

(Zach Goldberg at 00:14:17) At GruntWork, we use HubSpot. HubSpot has lots of great capabilities, and there's a hundred features I wish it had that it doesn't have that I can now just build. It cost me about $3 in tokens, and features that integrate data across multiple places or take actions across multiple different systems. And from that perspective, it's been phenomenal over these past six or eight months, the things we can do.

(Zach Goldberg at 00:14:40) Just integrating better across our software ecosystems.

(Joel Beasley at 00:14:45) I have created more applications in the past six months than I have in the past five years. I need a utility for something. It has almost become faster, Zach—

(Zach Goldberg at 00:14:56) It is.

(Joel Beasley at 00:14:57) For me to build the application with Cursor and deploy it to Heroku or right onto my iPhone, whatever the app needs to be, than it is for me to spend the afternoon hunting for potential solutions to see if I need to build my own.

(Zach Goldberg at 00:15:11) Thousand percent. Thousand percent. True story. Literally, this past week—for years, I've managed my finances in Empower's Personal Capital, right? So online dashboard, you link your bank accounts, it has views. And I've always just been frustrated with the data visualization. Over the past five years, I probably made half a dozen support tickets with this company asking for small features and tweaks here and there to make it easier for me to understand my own data. And it's just like, I had the same frustration a week ago, and I'm like, you know what? This is just a single pane of glass SaaS tool that pulls in some data.

(Zach Goldberg at 00:15:42) An hour is what it took me to vibe code a thing that replicated every single feature I needed. And now I can just, with two sentences, add any capability I want. And I think this is unbelievable that we can all just do this. And I'm in the, you know, millions of these apps now.

(Joel Beasley at 00:15:56) You know what's interesting? As you just said that, I've had a lot of conversations where people—you know, over the past year or two, things have matured quite a bit, but there's a lot of punching down at the vibe code. And I do it too because, I mean, I've built enterprise level applications, and I'm like—but then I started to think as you were talking, like, what if the future—like, vibe coding does work for personally? Like, if you just need to do this very specific task and you—

(Zach Goldberg at 00:16:24) Call it the SPA, trying to be funny. Rather than the single page app, it's the single purpose app.

(Joel Beasley at 00:16:29) Okay. The single purpose app. Yeah. So what if we don't need things—we don't need not all things. We don't need most of our applications to run at scale.

(Joel Beasley at 00:16:39) We just need them to run for us. What if that's the future? What if my future is a hundred little applications that my agent is maintaining for me versus me logging in to a Facebook UI that's being maintained for 10 million people?

(Zach Goldberg at 00:16:56) I think this is a new category that partially overlaps with the world of software that used to exist. I'd never would have spent probably the hundreds of hours to hand code a replication of a personal finance app, right? But now that I can do it in an hour, that's an application that does exist now that wouldn't have existed before. And so, you know, in the universe of software that could be written, we were only writing 50% of apps or whatever percent that humans wanted.

(Zach Goldberg at 00:17:23) And now we've expanded that bubble, which is to say, there is still a very—my belief is there's still a very large set of apps that are not replaced or replaceable by vibe code app, right? Facebook still has value because there's still—or I don't know if you believe Facebook has value—or Instagram has value, because there's still a network of a billion people that want to share photos, right? And I can't vibe code that, right? Because that does have to actually exist that a billion people use.

(Joel Beasley at 00:17:49) But their interface, though—that's like, I think they're going to become more like a data store where the API is powerful. Where I'm just—everyone's plugging in through the API. I don't even know, man. I've been running an experiment for fifteen days now, so I usually don't like to talk about stuff until I've done it for a while, but you've inspired me. I got an Apple Watch.

(Zach Goldberg at 00:18:12) It's very pretty.

(Joel Beasley at 00:18:14) Thank you. That was the point. I was like, I want to look beautiful. I don't care about the technology. No. I got this cool green band, though. But the purpose was I want to—I noticed that I was having self-control issues overriding my time limits on social apps, and I have to use them for work.

(Zach Goldberg at 00:18:35) Like, I—

(Joel Beasley at 00:18:35) Have to check the stats. I have to make sure the post happened. But then I would go in there to make sure the work happened, and then I'm doing fifteen minutes of just junk. Just junk. And so I said I need to do the screen less, but the thing is I need access to my wife for the kids, and we need to be able to call each other.

(Joel Beasley at 00:18:52) So I went with the Apple Watch about fifteen days ago with the cellular and everything. My screen time is down like 65%. I can leave my phone at home. Like, I just put it away. I just put my phone away unless there's something I need my phone to do.

(Joel Beasley at 00:19:10) And so my—it can do my Tesla. It's my Tesla key. It can do my Apple Pay. It can call people. It can text people. Anything that I'm required to do—

(Zach Goldberg at 00:19:23) Were you able to integrate your important work-related social media stuff into the Apple Watch, or is that still required?

(Joel Beasley at 00:19:29) No. So I've just—I have compressed that down to scheduled screen time. So now I've got a three-hour block every day where I can use my phone and my computer, and I have to get everything I can get done in that—you know this as an entrepreneur. You have to figure that. So I have to get everything I can get done in that time, and then I use a lot of that time to make stuff more efficient.

(Joel Beasley at 00:19:52) Like, I recently found Superhuman email. Have you used that before?

(Zach Goldberg at 00:19:56) Mm-hmm.

(Joel Beasley at 00:19:57) Oh my gosh. Dude, Superhuman? What do you currently use it, or do you just play with it once?

(Zach Goldberg at 00:20:01) Currently use it, but there's folks on my team who use it every day, yeah, and they rave about it. I've dabbled with it.

(Joel Beasley at 00:20:06) Yeah. It's got this new stuff in it where you can tell the AI, like, look out for these types of emails, and it then puts them in a priority section for you. Brilliant.

(Zach Goldberg at 00:20:17) You know? So that's the next question, though, is when do you hook up your Mac mini OpenClaw whatever—

(Joel Beasley at 00:20:23) Yeah.

(Zach Goldberg at 00:20:23) To your email? And rather than relying on Superhuman's prompts, if you want to customize it into your own workflow, email is another API to integrate into your—

(Joel Beasley at 00:20:33) Well, that's how I started, actually.

(Zach Goldberg at 00:20:35) Okay. Is it better than what you got?

(Joel Beasley at 00:20:37) Yeah. Well, for the intended use case—

(Zach Goldberg at 00:20:41) Mm-hmm.

(Joel Beasley at 00:20:41) What I was doing had more breaking points than just—like, I built this infrastructure to do all of that, to API and Gmail to read them all and then to move them in—but then the Superhuman had a better interface. It had snippets for quick replies, and then it had keyboard shortcuts. And it was so polished that I was like, yeah. I'll give them $40 a month. This is saving me time, and now I don't have to maintain, which is another behavior I think is going to happen too a lot with people, Zach.

(Joel Beasley at 00:21:11) I think we're going to build stuff, but if your personal finance app, if you went to go prompt it and then it was like, hey, there's also this out there that this other person did that Joel built that achieves that. Would you prefer to use that and not maintain your own project? You could click, yeah, let's give it a shot.

(Zach Goldberg at 00:21:29) Yeah. It's interesting how—I had a very fun story in developing this personal finance toy. At one point, it needed API access to some banking thing. And the AI, Claude, told me, oh, there's a wrapper that already exists around the API. And I said, okay, great.

(Zach Goldberg at 00:21:53) Use the wrapper. It's probably been tested. It said, no. I'm going to just build it from scratch because I'm going to use TypeScript and have types, and the other one isn't maintained all that great. It didn't quite say no, but it pushed back.

(Zach Goldberg at 00:22:04) Yeah. And I was like, that's interesting, right? So given the opportunity to use something off the shelf that has some sense of validation that but maybe it had some, you know, minor trade-offs. It chose to just know, I'll just write the 500 lines of code from scratch.

(Zach Goldberg at 00:22:17) That was, you know, its instinct as cheaper. And one could say, well, perhaps it just wanted to burn those tokens. You know, my instinct was, just use the thing that I know works. Perhaps I shouldn't be surprised that its own homegrown 500-line version actually did work on the second try. So it's this idea of software reusability for the single purpose applications, I think, is we really need to rethink that.

(Zach Goldberg at 00:22:43) Do we care about something that already exists that can just be recreated with six minutes of thinking, of open thinking?

(Joel Beasley at 00:22:51) I think that's the question every SaaS company is asking themselves right now.

(Zach Goldberg at 00:22:56) Where is real value generated with software today? I don't know. Do you have a thought on that?

(Joel Beasley at 00:23:03) No. Compliance. I think that's going to be a huge one when there's regulatory and compliance reasons why—like, when the technology could one-shot prompt something that I can't because I have to go to the government to get some form done or something like that. Or I think that's going to be—if I was investing, I'd probably invest into that because I think long term, that's got a really good shot.

(Zach Goldberg at 00:23:27) Yeah. Yeah. I—yeah. So at GruntWork, we sort of have a perspective. There are things we do that are vibe codable replacements, right? Because they're table stakes as part of a broader ecosystem. But where there's still lots of value is foundational tooling that's expected to last, right? Where there is value in large amounts of people all understanding the same software.

(Zach Goldberg at 00:23:52) So, yeah, great. We maintain Terragrunt, right, an open source infrastructure as code tool, right? And there if I'm some, you know, platform architect that doesn't work at GruntWork, I work at whatever company—even if I'm using an AI, I'm still relying on foundational tools.

(Zach Goldberg at 00:24:08) Like, I'm writing code in Python or TypeScript or Terraform or OpenTofu, whatever, right, in Terragrunt. And I'm going to need to hire other people, and I'm going to use AIs. And all of these people and AIs need to have familiarity with these tools, right? It would be—it's akin to I'm starting a new company, and, yes, I have Claude. And rather than have Claude write code in Java, I'm going to have Claude invent a brand new programming language. And all of my programming will be done in this new programming language that's only used at my company. And so now I'm dependent on exclusively AI model's ability to use that tool. I can't hire people easily who master that tool, and there's no guarantee that the next version of the AI will be as competent with that tool. And so there's still a lot of value in these common layers that we reuse that needs to be well understood.

(Zach Goldberg at 00:24:56) And those things, because the blast radius is so large, need really deep thought as to how they should work, right? You don't want to just say, hey, Claude, what's the next feature of OpenTofu? You want to actually think, how is it going to work for the million existing users, and how is the next million users going to use it? And that's still a very contemplative process.

(Joel Beasley at 00:25:18) I've noticed that I have made the AI write everything that could be written in Rails in Rails because I've got, you know, a decade of experience in managing Rails projects. And I can catch it in little mistakes and understanding how it would scale in production and or under load. And so, yeah, I think you're right. There's definitely value in being able to talk about it with other engineers, but at the same time, I'm not an iOS native developer. And I did an iOS app, and I wrote zero lines of code.

(Joel Beasley at 00:25:51) It was a fairly complicated app that had to—it can't be published to the App Store. For what I needed it to do, I had to kind of work around some things, and it could be a local app for my phone. But I didn't write one line of code. It was 100% just me talking to the thing in Cursor and then it running and then me using it and then providing feedback, and I think we're there. I think we are there for—I mean, I don't think—I know we're there for individual personal use.

(Zach Goldberg at 00:26:20) Mm-hmm. Yeah. And that's this—this is where we're missing the taxonomy, right? 100% agree. For the single purpose app, the individual app, we're absolutely there, right? You know, half an hour, an hour with an AI. You can build your iOS app, Android app, web app, whatever. But quite clearly, like, I still pay—full time GruntWork, the company I work for, still pays full-time software developers to develop enterprise software. And so—and, you know, I'm very confident I could not replace my team with only AIs.

(Zach Goldberg at 00:26:49) But I have a difficult time explaining the line. Like, obviously there's a spectrum here, and what is the label on that—on the x-axis in that spectrum, right? Is it complexity? Is it size? Is it scale? Is it number of users? Is it how long I expect it to last? Like, there's a number of ways to look at that and answer the question, could an AI do this on its own? Or does this still need a team of humans overseeing and collaborating and supervising?

(Zach Goldberg at 00:27:16) We could point to a number of—

(Joel Beasley at 00:27:18) I would label that—like, what's coming up to me right now, I would label that trust to achieve outcome. So there's a number of outcomes that have to happen at your business.

(Zach Goldberg at 00:27:29) Mm-hmm.

(Joel Beasley at 00:27:30) And you can—you know, because you're an entrepreneur like me, and we own businesses and employees and stuff. We know that you can—we know how much load a single person can take, right? Like, you can tell when they're overloaded or not. And so we say, okay. I have trust for that person to achieve these two outcomes. I know if I put a third, that's going to stretch them. Quality is going to drop, but I know that in my mind, I wake up in the day, I know these two things are important to the business. I know Mike's got it. Okay?

(Joel Beasley at 00:27:58) And then you, as an orchestrator, you have many of those items that you will then spread across multiple people, and then you have these direct personal relationships with trust with them to achieve the outcome with the advanced AI. And I think that's what is happening. Yeah. Because if you did trust the AI to achieve all of the outcomes, I think we're not there yet. This is like full self-driving with Tesla.

(Joel Beasley at 00:28:26) Do you have—have you ever done Tesla full self-driving?

(Zach Goldberg at 00:28:28) Yeah. Of course.

(Joel Beasley at 00:28:29) You've got to build up this trust.

(Zach Goldberg at 00:28:31) It's not trust. I think they're—trust in and of itself is multidimensional, right? I trust that if I ask AI to implement feature X, will it implement—you know, and I define half a dozen acceptance criteria. Will it get those six acceptance criteria? I think for reasonable circumstances, there could be some trust there. However, my definition of those acceptance criteria is almost certainly imperfect. There's probably six more AC that should be there that I didn't think of. And if I assign that to my senior engineer, he's going to find those six AC. He's going to think of, okay, we'll build this feature now.

(Zach Goldberg at 00:29:09) And by the way, you didn't think of these other edge cases of, you know, real-world user workflows that might happen. But also, there's other features coming down the road. And, you know, how is it going to interface with X, Y, Z? I have no trust that the AI sees around corners in that way.

(Joel Beasley at 00:29:24) And that's why I focused on trust to achieve an outcome.

(Zach Goldberg at 00:29:28) Mhmm.

(Joel Beasley at 00:29:28) Because the outcomes are very complex. It's not a feature. It's not just like one specific thing. It's this culmination of all of these things in their environment, whether they're interacting with customers and going with their gut after talking to multiple customers. So I think the human's role right now is achieving outcomes with the advanced technology. And I mean, you could probably use the same words ten years ago. But the thing is what's compressing is the number of humans that you need to achieve the outcome.

(Zach Goldberg at 00:30:00) Yeah. And there's still the same problem I've always had of defining the outcomes we want clearly, knowing what outcome you actually want to get to. Understanding what to build is harder than actually building it, and that's more true now than it's ever been before.

(Joel Beasley at 00:30:14) Taste. Like, trusting their taste. When Mike's got the problem, you understand that there's gonna be stuff that you can't even see that are coming from left field, things that are gonna pop up midstream of implementing all this stuff, and you have to trust that he has the right head on his shoulders to make these decisions.

(Zach Goldberg at 00:30:33) Yeah. And I do think the other—so there's trust as a big part of it, but to add another element to the mix—a million tokens in context is enormous. But I think empirically it still pales in comparison to what a human can keep in their head. And the quality of the context that a human keeps in their head, I still think we're several orders of magnitude off. Right? Like, Gruntwork's software compared to Fortune 500, I'm sure is very small in terms of lines of code and total amount of complexity, but it's still huge compared to the individual application and completely terrible, like, unusably poor when thinking about the system at large. Right? So it's still—back to your point—defining an outcome within a context that is understandable within the AI scope.

(Joel Beasley at 00:31:22) Have you looked at the systems that are designed to do that?

(Zach Goldberg at 00:31:26) We've played with it a little bit. I wouldn't go so far as to say that I'm the expert on the larger system tooling that's out there, like Codex spaces, things like this.

(Joel Beasley at 00:31:36) Josh, who is that company that did a sponsorship with us that's specifically doing that? They were building tools specifically for that use case of massive code bases at enterprises to understand them and be able to work with them. I didn't play with it myself hands-on, but I talked to some of their users and customers and stuff, and it was pretty—

(Zach Goldberg at 00:31:58) So do you feel confident whether we're there today, we will be soon enough?

(Joel Beasley at 00:32:03) I think we have—I think we've got all the tools in place that we are, and I think the momentum is there. I think look at the past. How wild since our conversation in 2024. We could not do that personal finance one thing that you did in the hour of 2024.

(Zach Goldberg at 00:32:21) Glorified autocomplete. Like, yeah. Do you remember glorified autocomplete two and a half years ago to now go spit out 3,000 lines of code?

(Joel Beasley at 00:32:29) I keep trying to challenge it more, and it just keeps delivering. I am just so impressed by it. Have you seen Blitzy?

(Zach Goldberg at 00:32:38) Yeah.

(Joel Beasley at 00:32:39) So Blitzy is pretty cool. They have a number of example use cases, but they essentially write—if you have a, you're an enterprise, you're gonna do this product sprint—it'll get you 90% of the way there, like writing all the code and planning it all and everything. And then you just have to go in and do the last couple human things. It's pretty fast.

(Zach Goldberg at 00:33:02) With these kinds of tools, I wonder, what is it they actually do? Right? Where is the value being created here? Is it chaining together existing agents and some workflow glue on top of it all? Like, is there defensible intellectual property there? Or is this really just gonna be part of the next version of Claude in eighteen months?

(Joel Beasley at 00:33:25) Both.

(Zach Goldberg at 00:33:27) Both.

(Joel Beasley at 00:33:28) I do. I think that—you know, that's the fun thing. So one, someone gets a little bit farther ahead, and then they see that, and they're like, oh, that's the thing. And then all the competitors can catch up in a month's time. It's wild. Like, have you seen the release of Gems and then Projects and then the maturity of these systems accelerating at such a degree? It's mind-blowing. So it's almost like whoever can have the coolest imagination, and then you watch everyone run to it like a breath of fresh air. Cool. That's a better word. Then all you have to do is say, go tell the model that you want your thing to do that now.

(Zach Goldberg at 00:34:10) Yeah. Yeah. So to your point—I'm not an investor in Blitzy—maybe buying puts on these small company stocks because to your point it just shows up in your mega cap tech company in three months from now.

(Joel Beasley at 00:34:26) Yeah. But then that's where trust comes, right? So if you're an enterprise and you're like, okay, there is this company, Blitzy, that has done this work with a lot of other enterprises and sectors that are sensitive, like maybe HIPAA compliance or banking or something like that, I would rather bring them in than my team try to do it with Claude because it gives you this sort of—as an executive running hundreds of millions or billions of dollars of value—it gives you this peace of mind back to trust. Like, I would rather have the people who've been doing it since you could do it than start it myself.

(Zach Goldberg at 00:35:03) Yeah. It's interesting. We actually did a survey of CTOs and CIOs at Gruntwork over the past several weeks about their adoption of AI and the build versus buy question. Building is now cheaper than ever. Right? And so has the equation changed? Like, do we see changes in patterns? Relatively small in size—we did about a dozen leaders. The conclusion we came to is there's actually a clear line in the sand where things have changed and where they haven't. And the distinction is read versus write. If there's a problem that a company has that's basically information aggregation, right? Like pulling data from my monitoring system, from Datadog, from AWS, and maybe reading Notion docs as well and synthesizing that into a thing, like a sort of read-heavy use case, vibe code is everywhere. Right? Everybody's got an internal platform with agents that speak all their tools. With writing—we actually go change my infrastructure, go deploy this application—we still see more hesitance to vibe code at enterprise there, at least based on our limited data size.

(Joel Beasley at 00:36:13) But it—

(Zach Goldberg at 00:36:13) It makes a level of sense, right? To your point about trust, writing has—you trust that it's going to work and it's not going to cause problems. And so you don't necessarily want to shoot from the hip on that with something that somebody generated at 9:00 last night. You want the vendor who's willing to put their reputation behind it, a warranty behind it, whatever, an SLA.

(Joel Beasley at 00:36:33) I think you're exactly right. I'm just like you and your personal finance app. You know? If you're just reading data, it's easy to spin it up real quick.

(Zach Goldberg at 00:36:43) Mhmm. But you better believe the very first thing I checked, though, as I started this exploration was, what are the write verbs in these APIs that this application will have access to? Obviously, the assumption is we were looking for zero is the correct answer to that question.

(Joel Beasley at 00:37:01) Startup CTOs. You wrote a book about it. Let's give a shout out. What's the name of the book?

(Zach Goldberg at 00:37:07) The Startup CTO's Handbook.

(Joel Beasley at 00:37:09) Startup CTO's Handbook. Now you wrote that pre-2024.

(Zach Goldberg at 00:37:12) Correct. Yeah. It's sort of a mark of pride that the book was published just around the time ChatGPT became commonly used.

(Joel Beasley at 00:37:19) Yeah.

(Zach Goldberg at 00:37:20) So zero AI in the production of that book.

(Joel Beasley at 00:37:23) And has any of the advice—do you think it's changed? Would you change—are you gonna update it?

(Zach Goldberg at 00:37:29) I do think there are missing chapters. There are new chapters that need to be written about leadership in the world of AI. But I will take a stand that the existing chapters dealing with leadership and people management and how you do hiring and how you think about product development from a—you know, think putting the user first—those things don't change. The nuances perhaps of running a hiring process are perhaps different in a world of AI-generated resumes. But the core values of what it is to think about a hiring process and what you're optimizing for, and when you—you know, performance management internally and leading a high-performance team—I don't think that has changed with the advent of AI. Like, how you interact with humans should not change because the humans are now using faster tools.

(Joel Beasley at 00:38:15) What's the chapter that you think is missing that you would like to add to it?

(Zach Goldberg at 00:38:19) I think it's predominantly how we think about staffing early stages at startups. You know, there's been a common question I get asked: I'm an entrepreneur, I want to start this new company. What's the—I've got $30,000. How do I spend my $30,000 to get a prototype? And the common choice is, well, I can hire somebody expensive in North America or I can go hire offshore for one-tenth the cost. What's the right answer? Right? And I think I would often advise people, spend your money as frugally as possible to get to the MVP because whatever it is you're building, you're almost certainly not gonna still have it two years from now. Right? You don't know what your company is on day one. And so don't overspend and overbuild early on. You know, that's probably changed. Almost certainly you should be vibe coding the first version of your applications. Right? And you should be putting it in front of users, and that should cost basically nothing. And, you know, what is all the nuance to getting there? Right? What is that process of vibe coding and getting in front of a user and iterating to find value early on? And it also comes back to our earlier question of, can you generate value, right, with just a SaaS software application nowadays? And what is your angle to not be replaced, to your point, by some other company who can also vibe code sixty days from now? So I think there's a lot to be said about thinking about value long term and thinking about how you use software to make people's lives better in this world. So I think a few chapters on there is probably warranted as far as your role as a CTO.

(Joel Beasley at 00:39:56) Are you gonna run a prompt after this to have it update the book and deploy it to Amazon?

(Zach Goldberg at 00:40:03) No. I confidently will say that I care about the quality here, and my level of trust for an AI is very low. And so at some point, I'll take a month or two off of work and really think about it and talk to other people and put some blood, sweat, and tears into making something quality there.

(Joel Beasley at 00:40:21) Do you think humans are still gonna be reading books in ten years?

(Zach Goldberg at 00:40:25) 100%.

(Joel Beasley at 00:40:26) Thousand percent. Why?

(Zach Goldberg at 00:40:29) Why does somebody read a book? What are the use cases? What are the user stories for why somebody reads a book? Right? I'm 14 years old in high school, and I have a math test coming up. And I have a math textbook. Right? I think, am I gonna use an AI to teach me math? Or am I gonna read the textbook? There's no right or wrong answer there. Like, both is almost certainly the answer. Why do I read a book? Maybe I want to be entertained. I'm sitting at a beach on vacation, and I'm interested in a story. Right? Humans are wired for story. We enjoy story. It activates all parts of our brains and empathy and connection and identification, all these things. I'm not gonna use an AI to just vibe code a story. I want to go read Brandon Sanderson. I like the way he tells stories. And we could probably—we can enumerate 20 of these. And I think some of them, sure, could be replaced. But, you know, bring our conversation full circle and the nuance, I think, yeah, people will definitely still read books. You disagree? You think books are gone? Save all the trees?

(Joel Beasley at 00:41:29) I don't—I think habits are hard to break. So the people who really like books—I'm not a book person.

(Zach Goldberg at 00:41:38) Mhmm.

(Joel Beasley at 00:41:39) I'm a story person, and I'll do audiobooks and movies and stuff like that. But I haven't ever really been—the only time I've gone to books is when I needed—like, when I was learning programming, I needed to learn how to, so I had no problem going to the books to learn. Now I would just ask the AI. You know? It's like, oh, you don't have to wait for the book. You don't have to wait for the podcast interview. You can just go talk to the AI now.

(Zach Goldberg at 00:42:08) For adult learning, absolutely. Yeah. It's interesting. So like you, I've been an audiobook listener for a long, long time. And I think there's a culture, a subset of the world who believes audiobooks are not as good as real books. And actually, there have been a number of peer-reviewed papers published in the past couple years looking at this question. Right? Do you learn or do you absorb or whatever verb as much information or as well with audiobooks as with paper books? And the answer, surprise, surprise, nuance. When it comes to hard intellectual subject matter that is—what's the—there's the adjective they use—so basically, it connects and builds on each other.

(Joel Beasley at 00:42:49) Uh-huh.

(Zach Goldberg at 00:42:50) Then you want paper. Right? You want, you know, philosophy, mathematics, physics, where to understand physics concept X, you have to also understand A, B, C, D all the way through X. Whereas for story, audiobooks and paper books actually are absorbed and people answer questions and have the same retention, very comparable. And the mechanism that is proposed is with buildable material, you naturally go back in reference. You're looking at this page, it's like, what happened in the prior page? Like, how did we get here? I didn't understand that concept. Very easy to flip back, rescan two sentences, and okay, now you're good to go forwards. Mimicking that process in audiobook—sure, there's a rewind fifteen-second button, but how often do you push it? Right? And does that actually take you back to that one sentence that you missed that was critical to understanding the next phase? And so the hypothesis is that for casual listening, for things you're just absorbing for fun and entertaining, you know, it's okay to be distracted while you're doing it. Audiobooks are excellent and just as good as paper. For things where focus is required and learning is the paramount objective, paper still has an empirical edge. So the research says at this point.

(Joel Beasley at 00:43:57) Yeah. That makes complete sense. Right?

(Zach Goldberg at 00:43:59) Seems intuitive.

(Joel Beasley at 00:44:00) Yeah. That 100%. I think, yeah. Let's see. I still think books are gone. I think they're gone. I think—

(Zach Goldberg at 00:44:11) You heard it here. Books are dead.

(Joel Beasley at 00:44:11) No. No. I think it's gonna take—I think it'll take a couple generations, and they might come back. They might make some small resurgence or whatever.

(Joel Beasley at 00:44:19) But the thing is, I can go learn something interactively with an expert on the topic. And for me, conversationally, that's just how I consume it. So I'd say for people like me, books have been dead for a while.

(Zach Goldberg at 00:44:40) So I disagree with you. However, I will also argue with myself. When was the printing press invented? Fifteenth century, something like this? Don't know.

(Zach Goldberg at 00:44:49) Right? So how long have humans been reading books? A couple hundred years? Right? How long have humans been telling stories and learning things?

(Zach Goldberg at 00:44:55) Ten to—

(Joel Beasley at 00:44:56) Thousands of years. Thousand years. Yeah.

(Zach Goldberg at 00:44:57) Right? So books in the grand history of the way humans communicate with each other is actually a very modern invention. And so is it possible that it's a blip?

(Joel Beasley at 00:45:06) It's possible that it's like a CD, and there's just a better way for it to look. We're all obviously going to neural implant and be able to stream thoughts. So it's not even—WALL-E is the future.

(Zach Goldberg at 00:45:18) That's just it.

(Joel Beasley at 00:45:19) Well, I'm going to be the only person on board that spaceship with a six-pack. I'm not letting my body go.

(Zach Goldberg at 00:45:27) Your discipline is admirable, sir.

(Joel Beasley at 00:45:30) I'll be running that ship.

(Zach Goldberg at 00:45:32) You know, I don't know. If I could have the same life expectancy and just sit in that couch, man, that's tempting.

(Joel Beasley at 00:45:40) No, no. I've been overweight. You feel horrible. It's not—

(Zach Goldberg at 00:45:45) Have you been seriously ill or—we're going to have the pill that allows you to be overweight and still feel good.

(Joel Beasley at 00:45:52) Ecstasy?

(Zach Goldberg at 00:45:54) I'm not naming the drug now.

(Joel Beasley at 00:45:56) Well, this was fun.

(Zach Goldberg at 00:45:59) Yeah. Just hang out.

(Joel Beasley at 00:46:00) Thank you so much for listening. And if you found this episode useful, please share it with a friend or colleague who you think would get value from it. And if you have topics that you'd like to hear discussed on the podcast, either add me on LinkedIn or send me an email: [email protected]. Every time I get an email or LinkedIn message, it absolutely makes my day and inspires me to keep going.