Episode 215 ·
Matt Fornaciari - Co-Founder & CTO at Gremlin
Today we are talking to Matt, the CTO and Co-Founder of Gremlin. And we discuss what the day was like when he decided to found his own company, how Gremlin is using Chaos Engineering to make systems more resilient, and why introspection is the key to evolution.
All of this, right here, right now on the Modern CTO Podcast!

About Matt:
Matt started experimenting with computers as a child largely due to an intense aversion to repeating tasks manually. This experimentation quickly blossomed into a passion and after a formal education, into a career which began with making Amazon.com more reliable. Since then Matt has decided to take what he learned at Amazon.com and apply it to the internet at large, allowing on-call engineers everywhere to rest more easily.
Previously, he was a Senior Platform Engineer at Salesforce, where he led the charge to bolster the experience of viewing and editing each and every record. Before that he improved the reliability and customer experience of the Amazon Retail website, where he founded the ’Fatals‘ team which reduced the number of website errors by half in its first year.
About Gremlin:
Downtime is expensive and can hurt your brand. Gremlin provides engineers with the framework to safely, securely, and easily simulate real outages with an ever-growing library of attacks. Turn failure into resilience with chaos engineering
Transcript
(Joel Beasley at 00:00:00) Hello, my friends. Today we are talking to Matt, the CTO and co-founder of Gremlin, and we discuss what the day was like when he decided to found his own company, how Gremlin is using chaos engineering to make systems more resilient, and why introspection is the key to evolution. All of this right here, right now on the Modern CTO podcast.
(Joel Beasley at 00:00:23) Here we go. This is the Modern CTO podcast. There he is.
(Matt Fornaciari at 00:00:36) What's going on, Joel? How are you? Good to finally meet you.
(Joel Beasley at 00:00:39) Dude, you're the man of the hour. Do you go by Matthew or Matt?
(Matt Fornaciari at 00:00:44) Honestly, I'll answer to Matt, Matthew. My coworkers and friends call me Forney, whatever you like.
(Joel Beasley at 00:00:52) All right, I think I'll go with Matt.
(Matt Fornaciari at 00:00:54) Cool. Sounds good.
(Joel Beasley at 00:00:56) I'm actually out in Colorado doing some hiking this week, and I saw that you enjoy some hiking as well.
(Matt Fornaciari at 00:01:03) I do, yeah. Actually, I just hiked Crater Lake this past weekend. I did Mazama Village up to the Rim Village and back just to go take a little look out there.
(Joel Beasley at 00:01:12) Where is that at?
(Matt Fornaciari at 00:01:14) It is in South Oregon. So I'm in Chiloquin, Oregon right now. It's a very small town, like 60 people. I'm in the same old place on the river right now just to get a little bit of a break.
(Joel Beasley at 00:01:26) Sounds like we both have the same idea.
(Matt Fornaciari at 00:01:28) I think so, but you've got the more mountain man beard going on. I gotta catch up over there.
(Joel Beasley at 00:01:33) It's crazy, right? This is, we're going on, I think, six or seven months since February.
(Matt Fornaciari at 00:01:39) Oh, so this is your COVID beard.
(Joel Beasley at 00:01:43) This is my COVID beard, yeah. It was like the most perfect opportunity to get through the awkward initial beard phase, right?
(Matt Fornaciari at 00:01:51) Yep, no, I feel you. Right on.
(Joel Beasley at 00:01:54) So this is the podcast, by the way, just so you know. We just record the whole time.
(Matt Fornaciari at 00:01:59) Awesome. Are you doing anything in particular while you're out there?
(Joel Beasley at 00:02:03) So my buddy Derek had moved out here from North Carolina, and I was just cooped up in my house for so long with the kids and the family and everything. And usually I travel a lot. So when Derek moved out here two weeks ago, I was like, there's my opportunity. I'll go spend like four days in the mountains and two days with Derek, and it'll be a nice social distance vacation.
(Matt Fornaciari at 00:02:25) Yeah, I think you got entirely the right idea. I'm in your boat as well.
(Joel Beasley at 00:02:31) So how long are you out there for?
(Matt Fornaciari at 00:02:34) I'm just here till the end of the week.
(Joel Beasley at 00:02:36) Okay.
(Matt Fornaciari at 00:02:37) On to the next place for a little bit. So doing a little bit of traveling while—like you said, used to traveling a lot, and so not being able to travel at all has given me a bit of the itch. So trying to scratch that a little bit.
(Joel Beasley at 00:02:48) Are you using Airbnb to hop around?
(Matt Fornaciari at 00:02:51) Yes, sir.
(Joel Beasley at 00:02:52) So useful. By the way, I love your logo, your company logo.
(Matt Fornaciari at 00:02:56) Thank you.
(Joel Beasley at 00:02:57) It's this little green gremlin. Your branding is on point.
(Matt Fornaciari at 00:03:02) Yeah.
(Joel Beasley at 00:03:02) This guy right here.
(Matt Fornaciari at 00:03:05) Yeah, well, this was actually the first decision me and my co-founder made. We put something up on 99designs, and we're like, "Hey, give us a gremlin logo," and this is what came out of it. Our designer tuned it and tweaked it a little bit, but still probably one of the best investments I think we've ever made.
(Joel Beasley at 00:03:21) That's a pretty good site. I've used them before for a logo because it crowdsources, so you get so many variations of it, and you can really find the one that's like, that's the starting point.
(Matt Fornaciari at 00:03:32) Yeah, 100%. And thank you. We're pretty proud of the Gremlin mascot as well.
(Joel Beasley at 00:03:38) So what's the 30-second overview of what Gremlin is?
(Matt Fornaciari at 00:03:42) So Gremlin is a chaos engineering tool. Basically, we go out and proactively break your services so that you can see what happens under duress. The idea is that you want to do this in a very controlled manner. You want to do it following the scientific method, have a hypothesis you're trying to either prove or disprove, go create some chaos to see if your system auto-heals the way it says it does, if you can monitor the impact, if you get paged, that sort of thing. Because as we all know building complex systems, stuff's going to happen in the real world, and it's better to be prepared for it. It's better to have this stuff happen to you on your own terms at 3 in the afternoon as opposed to 3 in the morning, right? So the idea is build more reliable systems by introducing a little chaos on your own terms.
(Joel Beasley at 00:04:27) So let's say I have a product, an app, a web app, and our investors say, "Well, what happens if you get 500,000 users overnight in addition to what you currently have?" We would normally just be like, "Hey, well, it scales because we have scaling set up." But this would be a tool where we could actually simulate that?
(Matt Fornaciari at 00:04:48) Yeah. So this would be the tool where you actually go about making sure that your auto-scaling behaves the way it does, right? There's a bunch of these awesome mechanisms these days to be able to spin up new pods, to spin up new instances if you see increased load, if your CPU spikes, if your memory spikes. But oftentimes it's kind of like, "Cool, push the configuration into production and cross your fingers, hope that it ends up working the way you think it does." This is like, "No, no, let's go test that. Let's go spike CPU and make sure that we do scale from two to four instances, and then once the traffic goes away, then we scale back down." We're always right-sizing our infrastructure so we're not overspending as well on the other side.
(Joel Beasley at 00:05:26) That's interesting. Is it like a library I would include in my project, or is it just its own? How does it work?
(Matt Fornaciari at 00:05:34) So the way it works is we have a client, we have an agent that you install on your hardware alongside where your applications are running. And it basically communicates back with us to our service such that we are able to see the full architecture of your system. And then you can go and select by tag or target or however you like, select what you want to target. So say you have a service running on instances A through Z and you tag them all "service foo," you can just go into the Gremlin service and say, "I want to test CPU on service foo." The control plane will contact all of those instances and say, "Hey, go spike your CPU, go consume a bunch of CPU, go consume a bunch of memory," whatever the attack is, such that you can test out how those applications actually respond in those situations.
(Joel Beasley at 00:06:28) That's fascinating. You're completely opening my mind right now.
(Matt Fornaciari at 00:06:32) This is exciting. What we're here to talk about. I'm excited.
(Joel Beasley at 00:06:35) You guys must be growing fast. This sounds like it solves a really big need that's been out there. A little background about me: I've been programming for just over 17 years, and so you're right. Usually you just put into production, you're just trial by fire. And then when you have moments to be proactive, for whatever the seasonality of your business is, I found myself spending time trying to figure out how to even begin to test these things, how to send traffic to my—I specifically remember this one night, an investor actually asked us. And so I went home and I found this HTTP gem in Ruby, and I was like, "Oh, I'll just put it in a loop and have it send tons of requests and try." But there was no tool out there that's the name brand in the industry that's like, "Oh, you just go to this tool, they've got all 50 different types of tests you can run, you just install the agent, and then you just rock and roll." And that sounds pretty cool.
(Matt Fornaciari at 00:07:36) 100%. Yeah. And that's the thing, is a lot of people don't know where to begin, right? And so what Gremlin actually gives them is a framework to go say, "Hey, these are the couple of best practices you should be doing for this service or this technology, or if you're using AWS, here's some baseline you can do," and we go out and actually test that you're doing all the things that you should be doing. And I mean, this practice—like you mentioned, you didn't quite know where to start, you had to go find this gem and then basically throw your own load tester—but this practice has been around for a while now. In fact, my co-founder and I were doing this back at Amazon like 10 years ago. So it's not necessarily a new practice, but it's definitely one that's come into vogue in recent years. And when we decided to start the company, we're like, "Look, somebody's going to do this eventually, but it better be us. We're the experts in the space. We may as well."
(Joel Beasley at 00:08:28) So I'd never heard of chaos engineering before our production prep with you, and I was like, "This sounds amazing. I want you to explain it to me."
(Matt Fornaciari at 00:08:38) Doesn't it sound amazing? In fact, I used to joke when we started the company that my title was Chief Chaos Engineer, and I feel like that's just the coolest title you can possibly have. But I mean, chaos engineering, it's a practice, really. It's not—it's a mentality of engineering in which you introduce chaos on your own terms, like I mentioned earlier, to an end, to test a certain hypothesis you have about your systems. And as our systems have gotten more and more complex with microservices and moving everything to the cloud, using containers, all of these amazing advancements of technology, we're abstracting away a lot of how these systems work, and we need to be able to test that our assumptions actually are true, that the mechanisms we put in place to do auto-healing, auto-scaling, all of these great recent inventions actually hold when they're under duress. So that's what chaos engineering is. It's crafting a hypothesis, creating some sort of experiment to test that, and then observing the results and figuring out if what you actually think happens happens.
(Joel Beasley at 00:09:48) It's weird. It's like once you hear this is an option, it just is something that has to go on the checklist, right? I could, as a professional, I couldn't not explore this now knowing that it's so easy. I don't know, you have a fan. I'm a fan.
(Matt Fornaciari at 00:10:07) Thanks, man. I appreciate that. I mean, one of the ways I explain it is it's resilience testing for your system, the same way you would do unit testing, the same way you would do regression testing. It's this idea of like, "Cool, now do all the pieces move together well?" And it's testing more of the bedrock of what your application is actually run on, so a lot of the mechanisms around infrastructure instead of just test this code path or this one particular end-to-end, right? And not just that, it tests under duress as opposed to just like, "Cool, let's just test it under the good case and move on." So it's definitely a bit of a departure, I think, from the old school QA and sort of testing modes.
(Joel Beasley at 00:10:51) You mentioned earlier that you worked at Amazon. Is that right?
(Matt Fornaciari at 00:10:55) I did, yeah.
(Joel Beasley at 00:10:56) What did you do there?
(Matt Fornaciari at 00:10:56) That was my first job out of school, actually, which was awesome, but it was also a little bit of a trial by fire. And I really—I honestly, I loved every second of it. In fact, the very first day I got put onto the Availability Team, which was in charge of watching and monitoring all of the Amazon retail website, which was really the money maker back in the day before AWS really came along. Got put on this Availability Team, a very small but very high-powered team. And the first day they tossed me a pager. And I remember it was a toss. It wasn't a hand. It was like a toss of a pager and, "Good luck," give me a little saluting out the door. So one of the first things I got to do was go in and just read about all the outages on the retail website and kind of what caused complex systems to go down. And then as I got a little bit further in, my team was responsible for, "Cool, how do we increase uptime? How do we decrease the latency?" So the above-the-fold page of whatever detail page, list page, whatever you're looking at, loads like that. So it was very interesting. It was a very innovative team for the time, and we actually were doing chaos engineering back then in a little bit more manual way. We were starting to build out tools, but there was this guy, Jesse Robbins. He used to run through data centers and just kind of pull cables out of the wall.
(Joel Beasley at 00:12:21) What?
(Matt Fornaciari at 00:12:21) He was called the Master of Disaster at the time. And so we set about creating a more programmatic approach to doing some of that.
(Joel Beasley at 00:12:29) That is hilarious. What a reputation.
(Matt Fornaciari at 00:12:32) It was a hell of a reputation. Yeah. And then within a couple years, I actually got spun out of the Availability Team, which was mainly focused on uptime. I got spun out to create a team called the Fatals Team, which tracked any sort of 5xx server error. So any time, if you see the Amazon web page say like, "Whoops, sorry, something went wrong," that was something that my team ended up watching. So I'm a year and a half out of school, and they basically are like, "Look, we're giving you a team. Your job is to write a weekly email to Jeff Bezos, or Uncle Jeff as we called him at the time, and the exec team telling them all the stuff that was going wrong." And I just remember we had gone through this huge microservice migration. Everything was breaking left and right. Nobody really knew what was going on, and we were just tracking these things, starting to kind of identify new types of failure modes that were introduced by this migration. And I remember one day I went up to my boss and I was complaining. I mean, you're out of school. I'm just kind of like, "Everything's broken, Tim. What's going on?" And he just leans back and looks me in the eyes and he's like, "Look, Forney, sounds a lot like this is happening to you, and you need to happen to it." And so then I ended up creating this whole program around tracking fatals, clustering them, blocking deployments, and that sort of thing. So honestly, my time at Amazon was a tour of duty, but it was an amazing experience in just learning how to build reliable systems.
(Joel Beasley at 00:14:03) It sounds like the culture has also stayed with you a bit.
(Matt Fornaciari at 00:14:07) Just a little bit. We've taken a couple of those things and brought them over to Gremlin over here. So the customer obsession, the customer focus is definitely the utmost for us.
(Joel Beasley at 00:14:18) And then the one that stuck around a lot at our company is the Day 1. We have, I guess, our own interpretation of it, but there's just something so fascinating about how our default programming is to go do something new, right, when you could just take what's working and improve it. And so we've been really focused on that at the company, and it's like, how's the Day 1? Like, let's get back to basics. Like, what can we do right now to improve our core services?
(Joel Beasley at 00:14:45) And then it just compounds, and you look back six or seven months, and it's just mind-blowing how effective of a strategy that is. No wonder Amazon's as big as it is.
(Matt Fornaciari at 00:14:55) 100%. In fact, yeah, I still espouse that idea quite a bit myself. I think on our fourth anniversary of the company, I sent something to the effect of, like, amazing work. We built an incredible team. The product, you know, I love everything about this company, but remember, our work is just beginning.
(Matt Fornaciari at 00:15:11) Like, it's always back to basics. How can we get better? How can we make the customer experience a little bit better? You know, rely on the things that have gotten us where we are.
(Joel Beasley at 00:15:20) Yeah. It's a good move. It's what the experts do. They drill the fundamentals over and over in new ways, and they come out like this incredibly polished athlete or business person.
(Matt Fornaciari at 00:15:30) 100%.
(Joel Beasley at 00:15:31) What was the day like where you founded Gremlin? You were like, we know we have something, we're doing it, we're 100% in, we're going to file the LLC or the corp or whatever it is. Like, what was that day like?
(Matt Fornaciari at 00:15:46) Well, it was a day that has completely and totally burned into my memory. I think my co-founder and I—I think he came over. He lived in San Jose at the time. I was in San Francisco, and I had a big whiteboard in my front room, which attached to my bedroom, which then became my office for quite some time. But I remember the two of us just sitting there whiteboarding for probably four, five, six hours. And so, you know, we'll fill the whiteboard up, take a screenshot, fill the whiteboard up, take a screenshot, fill the whiteboard up. And we eventually got to the point where we're just like, look, are we doing this? I think we should do this. You know, it's two in the morning. We've had a little bit of scotch. We're just like, yeah, let's do it. I think this is the time. And it honestly, it's one of the best decisions I've ever made. I've really enjoyed every single second of doing this, and we're going on five years here. So I feel incredibly lucky to be able to say that.
(Joel Beasley at 00:16:36) Was that guy you mentioned, Jesse, was he an inspiration for the name? Because as you described him running around the data center ripping cords out, I visualized him as little gremlin in the movie doing that. Like, just—
(Matt Fornaciari at 00:16:48) So, actually, the gremlin, like, the etymology of it goes back to World War I. Pilots that were running long, long back-to-back missions would actually get really sleep-deprived, really bleary-eyed, and they would swear to control that they saw little things tinkering on their wings, like, messing with their engines and stuff like that. And so the etymology of gremlins is actually something that messes with mechanics. These little, like, just mischievous, but not necessarily malevolent, beings that kind of go in and muck with electronics and machinery and that sort of thing. And it just—we toyed around with a couple of other ideas, but we were like, nah, we gotta do it.
(Matt Fornaciari at 00:17:30) So it's, like you said earlier, it's been a great sort of mascot mentality to sort of build our brand around. So I really, I'm glad we did. I'll say that much.
(Joel Beasley at 00:17:43) So where's the business at today? Like, what stage are you in?
(Matt Fornaciari at 00:17:48) Do you mean in terms of funding? Do you mean in terms of growth? I guess—
(Joel Beasley at 00:17:52) In terms of—
(Matt Fornaciari at 00:17:53) Ways I can answer this.
(Joel Beasley at 00:17:54) Yeah. I left it a little bit ambiguous because not necessarily funding. I don't like measuring companies by funding because it's just so inconsistent how people see different rounds across the globe. But definitely in the sense of, like, where you feel it's at. Is it, like, you have a product, you have a fit, you understand your customer, you make those calls, you hear the same things over and over, you're constantly improving, growing teams of teams of teams? Are you trying to figure out international expansion? Just, like, you know, where is it to you?
(Matt Fornaciari at 00:18:28) Yeah. Totally. I mean, I think the way I've described it to a lot of people is, you know, the very beginning when we were first starting out the company, it was sort of the question was more, what the hell is chaos engineering? Like, there was not a general understanding of what this practice even was. And around year two, two and a half, it really morphed more into, okay, well, why would I do that? Like, I've already got so much chaos. I'm already doing so much in terms of fire drills, and I'm just, you know, I'm already pegged to the wall. This doesn't make any sense. We answered that question, and then it became, alright, tell me how to do it. So I think in terms of where we're at in the journey of sort of moding chaos engineering, it's become much more of, like, cool. I get it. I know what it is. I buy into it. Just tell me how to do it. And I think that's where we are in the journey.
(Matt Fornaciari at 00:19:09) It's figuring out how to take all of our expertise and keep pouring it into the product. You know, we've got a fantastic toolkit that you can do just about anything with, but what we really need to teach people now is, like, here's how you perform the practice, the program. Here's how you build out a reliability mindset. And really, I mean, I mentioned it on a publication a bit ago, but, like, make it into a habit. And not just a habit, but into an identity.
(Matt Fornaciari at 00:19:42) Like, make this, like, I am an engineer that creates reliable code. I am a reliability engineer. Somebody who really thinks about these sort of things on the daily. So to kind of get back to your original question, like, yeah, I think we've got the product, we've got the fit, and now we're really just—we're continuing down that education route of here's best practices, here's how you do this, these are what the experts do, and you want to be an expert, right? And really promoting that sort of idea, that practice on the day to day.
(Joel Beasley at 00:20:14) Yeah. Yeah. So you were talking a little bit about your article and your culture. Right? So I had actually read that article where you talked about the culture of resilience through adopting good habits. Me, personally, I'm a huge fan of monitoring your twenty-four hours and your habits are creating your future. And what really set me on that path was this guy James Clear. Have you ever come across Atomic Habits?
(Matt Fornaciari at 00:20:37) I have. That's the book I was talking about. That's the book that kind of got me going down this thought process. I was out on a run one morning listening to Atomic Habits and was just like, oh my gosh. I guess at the core, what we're really selling here is a new practice, a new mentality, a new way of sort of thinking about developing software.
(Matt Fornaciari at 00:20:56) And so it was, you know, we're not just selling a product, we're selling a new practice. We're selling something where it's like, hey, you gotta think about reliability as a whole when you're architecting, and you need to really sort of get in there. So, I guess, the part that really resonated with me was just that establishing good habits is all about sacrifice upfront without any of that sort of instant gratification. And so that part was the part that was like, oh, shoot. It really is. Like, you've got to be willing to defer some of that gratification and sort of fight against human physiology, which is like, I want gratification now. So how do we do that? And so that's what we're working on doing at Gremlin. It's, like, what are sort of these—I think James Clear calls them gateway habits—these things that you can do in two minutes to really get some value out of it. Right?
(Matt Fornaciari at 00:21:45) So how can we get you into the product, get some value, get you out of the product, and you go, cool. That was a great use of time. You know, this sort of analogy he uses that resonated with me is, you know, since I'm a runner, was he talks about, cool. You want to run a marathon? Your gateway habit is put on your shoes every single day for a week. Right? That's the intro. And so that's what we want to get people to do. Come in here, tell us what your, you know, your CPU monitoring threshold is, and we'll validate that your alarms go off. Something like that. Something very simple to kind of get you in and get you hooked on that.
(Joel Beasley at 00:22:17) How do you, personally, how do you ground yourself in the habits? And I'll share an example, like, for me. So—
(Matt Fornaciari at 00:22:30) Sure.
(Joel Beasley at 00:22:30) I go to the gym pretty much five to seven days a week. Right? And on the days where I'm lifting, which is probably about five of those days, I track, you know, all my sets and everything so I can constantly increase by a little bit, which helps track the progress. But what I do while I'm sitting there in between sets, because I've got timed rest periods, is I'll actually write the thing I'm—like, the goal I have or the thing I'm trying to do, and then what habit that I'm currently using that'll get me there. So, for example, I'll write, like—there's five things that are currently on my—and I've never shared this before. So it's personal. We might cut it out, but we'll see how it goes.
(Joel Beasley at 00:23:06) Let's see what the producer thinks.
(Matt Fornaciari at 00:23:08) Yeah. Alright.
(Joel Beasley at 00:23:09) So it goes like this. It goes nutrition, and my habit for that is tracking my macros in MyFitnessPal. After nutrition, it goes fitness, and my habit for that is showing up at the gym. After that, it goes family, and my habit for that is this thing I call Saturday Sheet. It's this spreadsheet we made called the Beasley Games 2020, and we track—I saw I hired a salesperson, I saw how he tracked the sales team. We got way better results that way. And so I started doing that with my family and the things we want to accomplish.
(Matt Fornaciari at 00:23:39) I love that.
(Joel Beasley at 00:23:40) Yeah. And then I have art, and my habit for that is piano. So I take these piano lessons through this app called Simply Piano. And then my last habit, number five is cash flow. And for that, I attend our sales meetings and meet with the financial advisor for the company.
(Matt Fornaciari at 00:23:59) Nice. That's awesome.
(Joel Beasley at 00:24:00) So I write them down. Like, I write that list of five and connect those dots five times a week at least. And I do it at the gym during that session, and it just burns into my mind.
(Matt Fornaciari at 00:24:13) Yeah. No. I think that's a great way to do it. I mean, I personally have started using—well, not started. I've been using it for a couple months now, but the app Streaks.
(Joel Beasley at 00:24:22) Oh. I don't—
(Matt Fornaciari at 00:24:22) I don't know if you're familiar with it, but basically, you put in what the habit you want to do with this, and then every day you have to check it off—but it's got—or not check it off, and then it becomes kind of obvious. But there's actually a bit of psychology around repetition in these things. And as you start to build up a streak, it becomes much harder for you to say, oh, I'm going to break it today. So meditation is one of those things that's been super hard for me to kind of get into, but I can see I have a twenty-three-day streak of meditation going on. Am I not going to do the twenty-fourth day? That doesn't make sense. Right? So things like that. I mean, I think the idea that James talks about is making the habits that you want to espouse very obvious, attractive, and easy. And so those are the ways that I try—basically, anything I'm trying to pick up, I try to make it as accessible as possible.
(Matt Fornaciari at 00:25:13) So, you know, if that's running, I put my shoes by the door with my shirts and my shorts ready to go as soon as I wake up in the morning. If it's drinking a bunch of water, like, I have a big water jug, I fill it up and put it on my desk in the morning and hope that it's gone by the end of the day. So it's making these things just immediately ready for you to sort of lean into, I suppose. So that's—
(Joel Beasley at 00:25:36) Yeah. I do that stuff too. What was his example in his first talk? He did it with an apple. He put apples on his kitchen countertop instead of some other type of snack, and he noticed that he just somehow—he just increased his apples intake. And so for me, this concept of curating your environment, like, being very intentional about the objects that are, you know, immediately available to you will—can actually—well, it's not can. They will. It absolutely does shape your future.
(Matt Fornaciari at 00:26:05) Yeah. 100%. And then, you know, eventually, you get these habits to be such a part of you that you have an identity shift, and you start to be—you start to—I no longer think, like, I run sometimes. Like, my thought is I am a runner. Right? Like, I have run several marathons. I'm like, I am a runner. Like, that's part of who I am now. And so there's less burden to it. You know?
(Matt Fornaciari at 00:26:28) To bring it a little bit back, that's very much what we want to get people to in terms of reliability. We want to give them the easiest, most obvious, attractive, and easy way to get into this product and get into this practice and then become, you know—not just say, like, I build reliable software, but, like, I am a reliability engineer. I am an engineer that builds reliable software sort of mentality.
(Joel Beasley at 00:26:51) I don't think that'll stick, but I think, like, I'm a gremlin would stick.
(Matt Fornaciari at 00:26:56) Okay. I'll work on that one instead. We'll send out little gremlins and everything too.
(Joel Beasley at 00:27:00) Right? Like, that's an identity, and that's, you know, doing this whole founder business thing, finding out—I heard this one sales guy had a profound impact on me how he talked about how sales happens, but he talked about this concept of identity. Like, when people can buy your brand and then they can become a part of that and that connects with their identity, it's just a whole different type of sale than a functional transaction. And, yeah, I could totally see people—if you have a shirt that has the gremlins on it, I would be very grateful to buy one. I know a lot of people don't sell their company shirts. They kind of give them away.
(Matt Fornaciari at 00:27:36) We'll get you one. Don't worry.
(Joel Beasley at 00:27:38) I will wear it too. I really will.
(Matt Fornaciari at 00:27:41) We'll get you one after this. I'll make sure somebody sends one out to you.
(Joel Beasley at 00:27:45) I love it. When did you start running?
(Matt Fornaciari at 00:27:47) I started running when I lived in Seattle when I was working at Amazon. So that was probably seven years ago, maybe eight years ago. And it started off with just, you know, five Ks and stuff like that and started to push myself a bit longer and longer. The whole reason is really that's where I find a little bit of peace of mind, a little bit of quiet to kind of mull over some of the, you know, the daily considerations and whatnot and really kind of dive deep into them. When everything else is distracted and I just have, you know, just have my mind—I don't usually run with headphones or anything like that. I just—
(Joel Beasley at 00:28:20) Oh, really?
(Matt Fornaciari at 00:28:21) Mull over whatever we got going on. So—
(Joel Beasley at 00:28:24) When do you run? Like, in the morning, afternoon, mornings?
(Matt Fornaciari at 00:28:27) Yeah. Yeah. I can't run in the afternoons. If I don't get it done in the morning, it's not going to happen.
(Joel Beasley at 00:28:32) That's funny with me too. So I get up, I run, make breakfast, then I lift, then I go to work. Because if I try to be like, oh, I'll work out in the afternoon—now there's—do you lose, like, as the day progresses, you lose discipline almost?
(Matt Fornaciari at 00:28:46) Well, so I heard once that you have a finite amount of willpower each day, and you can put it towards whatever you're working on, but it diminishes over time. And then when it's out, that's it, you know? That's the day. So I've found that if I—I tend to get up at like 5:30 in the morning, get my workout in before anything's up, and then I also just feel, you know, I feel quite accomplished for the day as is.
(Joel Beasley at 00:29:11) Yeah, I run about a mile to a mile and a half in the morning, and it's really just to wake me up. I try to get out of bed and onto the sidewalk in like five minutes. So I put my running clothes on next to my nightstand and all of that—my starter habit. So it's just easy. But I've never gotten into running the 5Ks. And then I had—there's this cool scheduling service called Calendly, and I had Roy, their CTO, on. And we're casually talking. He's like, oh yeah, I run. The guy regularly runs like 22 miles. And I was like—because I was like, maybe we'll get together, we'll run, you know, go for a morning run, because to me that's like a mile, mile and a half. That's fine. I'm not winded. That's just for me to start my day. And he's like, no, I do like 22-mile runs. He runs these long races and like hills and stuff. And I was like, I felt so small.
(Matt Fornaciari at 00:30:04) I used to do that, and I've sort of faded off from that. In fact, this morning I was like, you know what? I'm not running today. I did a little kayak instead. But I've been trying to switch it up a little bit more just to not pound the pavement so much, so to speak, just give the body a chance to recover sometimes.
(Joel Beasley at 00:30:21) Yeah, I was wondering about that too. I'm in my mid-30s, but I was like, I wonder how long these knees are going to go because I've been running seven days a week for like three years. I just turned 32.
(Matt Fornaciari at 00:30:31) I'm starting to get there too. I feel you.
(Joel Beasley at 00:30:33) Nice. Life is good though, right? It is. Oh, here's something we could talk about since we're so close in age. Okay, so I have noticed this distinct—I like to look at myself like a video game character and abstract out and objectively try to view myself. Doesn't happen by default, but you can sit down for 30 minutes and run the thought experiments. But what I've noticed is my mind going from like 18 to 22 to 27 to like 32 plus—I can tell the maturity happening. You can almost feel it. You can feel your mind actually changing how it's processing information, and you can feel this concept of experience stacking up, or you notice a pattern that's happened three or four times in your personal life, and you just choose—you have the maturity or the wisdom to just be like, I'm not going to go down that path this time. I've done it three or four times. It was horrible every time. The last two times I did it and I knew I shouldn't. And so now I'm not going to do it anymore. So I've been noticing this trend. Have you noticed that within yourself in the past couple years?
(Matt Fornaciari at 00:31:40) Yeah, 100%. I mean, when we started the company, I was of a very different mindset than I am now. You know, it was still very much perfectionist engineer-oriented and was very much like, cool, now we've got to do it exactly the right way, and it's not getting out the door until then. And, you know, this is just one of the concrete examples. But as I've garnered some of that experience over time, it's like, no, we've got to get features out the door, get them to customers, get them to start iterating and using it so that we can decide, you know, is this the right thing? Can we pivot a little bit? That sort of thing. But beyond even that, you know, changing just mindset around how I respond to things, how I evaluate things—if I take time to sit back and say, cool, I read through that document before throwing all these comments in there, just to give it some time to digest as opposed to, you know, more of the spur-of-the-moment, off-the-cuff, got to respond immediately sort of thing. Taking a bit more time to be thoughtful, to be strategic as opposed to strictly execution-oriented has been, I think, one of the largest sort of turning points or changes over time for me. And beyond that, the things I love more, I just poured more of my soul into. And the things that I'm like, this doesn't really serve me anymore, you know, those things I've come back to pay from. So just getting a little more experience, I think, helps guide with that.
(Joel Beasley at 00:32:58) It gets me excited because, you know, the story of my life has been patience and not having it, right? And so as I'm getting this patience and seeing how this different mindset can yield such better results, it makes me excited about—if life has gotten so much better with age now, I'm excited for the next 20 years, you know?
(Matt Fornaciari at 00:33:21) Yeah, 100%. Yeah, I'm excited for what comes next. I feel like also the past couple of years have just been, you know, that exponential sort of curve in terms of figuring myself out a bit more, you know?
(Joel Beasley at 00:33:32) You know, I was just thinking about that yesterday. As I was driving through the mountains, I said I have this repeating habit that just happens sometimes, or this repeating thought that occurs, and it's like me reviewing life post-life. Like giving my review for the game creator of life, like on how it went, right? It's just this weird thing that my mind does sometimes. But I'm always like, you know, the past couple years have just been—this is good because it's inherently difficult. It's like you have to overcome yourself, get to know yourself, and then it's really like a one-versus-one game. And it's just interesting to think about. I don't hear a lot of people talk about it. So if you ever come across any good books or narratives or discussions, you know, send them my way.
(Matt Fornaciari at 00:34:22) Yeah, 100%. I mean, introspection, I think, is pretty key to some of that evolution as well, right? And that's honestly part of the reason I hike so much. I spend so much time in the woods and that sort of thing, because that's where a lot of the other noise sort of dulls and you can spend a little bit of that time being introspective, thinking about, cool, how did this go? Is this something I want to repeat in the future? Did this go really well? Should I double down on this? You know, the same way we were talking about earlier. It's still day one, but let's go back to basics. Let's go build the core things and sort of work and help us.
(Joel Beasley at 00:34:55) Yeah. You mentioned actually this concept of creating do-not-repeat items. Could you expand on that for me?
(Matt Fornaciari at 00:35:01) Yeah, 100%. I mean, the do-not-repeat items are more in the aspect of building reliable software, but I mean, I think they're a general good note for life. The way we use it in terms of incidents and reliability is we take learnings from these incidents that we have, and we pull out the sort of learnings and we say, cool, we're going to go fix this because this is a do-not-repeat incident. And all incidents are do-not-repeat. I think I wrote an article way back when—you probably haven't seen—but the title of the article is just "Won't Get Fooled Again." It was, we're never going to fail the same way twice or we're not learning. And failure in and of itself isn't a bad thing. It's how you learn. But if you don't learn from it and you don't take those learnings and actually automate them and figure out a way to not make them happen again with respect to computing, then you're not learning as much. So what happened that time is our disk filled up—and this is something pretty much everyone has run into if you've been an engineer. Our disk filled up, and then we couldn't write any more logs. So we started throwing 500s, and everything fell over. And it was 4 o'clock on a Friday afternoon, and everybody's ready to go home, go on the weekend. And all of a sudden, boom, we get hit with this. Our service is completely out. We've got to go in there. It takes us 30 minutes to figure out that the logs can't be written because the disk is full. Well, that Monday, we wrote the disk fill gremlin, and we said, cool, we're automating this against our system. It runs every day between the hours of 2 and 4, and it's never going to bite us again. The mitigation that we put into place is we're going to consistently test it so that it never becomes a problem to us again. And that's sort of the idea of DNR issues—do-not-repeat issues. So the things that you take away and the things you implement to make sure that you never fail the same way twice. And that's, I mean, that's one of the cores of chaos engineering right there.
(Joel Beasley at 00:36:58) So in Gremlin, would I have like a list of all the learnings?
(Matt Fornaciari at 00:37:03) Absolutely.
(Joel Beasley at 00:37:04) That's actually cool.
(Matt Fornaciari at 00:37:05) So what we have in our application is something called scenarios, and we're actually building another layer on top of it called use cases. But the idea is that this is a failure scenario, and there's a set of actions that you take to test this. So you make this hypothesis. You say, I hypothesize that my disk, when filled, will kill that instance or replace it with a new one. Then you have an attack that basically goes in, fills everything up, and then make sure that that instance gets killed and that another one takes its place. So you essentially start to build out this library of failure modes that you become immune to. You basically inoculate yourself against each of these things as you run into them if you're going with a reactive approach. So you turn every incident into a learning, into a scenario that you're now immune to. And then on the proactive side, as you're going out and testing things proactively to see like, hey, does this work? Does this work? Does this work? As you sort of figure out, yes, this mechanism does work, you turn it into another scenario. So you build up this whole library of things that, you know, your system is resilient to, that make you more reliable. And then you can share those across different teams, across the company, and everybody gets to learn together.
(Joel Beasley at 00:38:19) That's so much better than putting it in a Google Doc somewhere of like retro notes.
(Matt Fornaciari at 00:38:25) A little bit. Yeah. I've seen every manner of post-mortem and retro notes.
(Joel Beasley at 00:38:30) But this is—
(Matt Fornaciari at 00:38:31) It tends to get lost in the shuffle.
(Joel Beasley at 00:38:32) Tell me if I'm wrong because my mind's connecting the dots or trying to. So it kind of reminds me of testing, how people would test in the console—that's not writing a test. Like, you know, developers before they learn about what testing is, they'll test in the console. And then it's like, hey, hey, hey, hold on a second. Just write this test and it'll save you a huge amount of time. But it sounds like your scenarios, as you're referring to them, they have the test in them. It's like it's not just a note. It's like that is the test, but you can actually run that again, and it's all there.
(Matt Fornaciari at 00:39:07) Not just run it again. Run it consistently. Because the way that these systems run, constantly components are changing. You may go through a migration. You've got just all these moving parts as things get more and more complex, and you want to make sure that you're never regressing. There's a book called "Drift Into Failure"—I think Sidney Dekker is the writer—but the idea is that you never want to just suffer due to code rot or, in this case, infrastructure rot, or these sort of things where you just over time become more and more resilient or less and less resilient. So by automating a lot of this testing, you're basically setting a baseline of like, no, this is the core things that we run against our system to make sure that we're reliable. And to your analogy, yeah, you go and automate that. You come up with a set of base classes—that's test-driven development. Well, if you take a bunch of these scenarios from Team A, which is great and has, you know, service Goo, which has a ton of reliability, you give it to Team B, which is spinning up service Bar—now this becomes reliability-driven development. And they can't release the service until they pass scenarios one, two, three, and four, right? Until they've got their baseline monitoring, until they've got their baseline alerting, until they make sure they can auto-scale—a bunch of these kind of core best practices that define reliable systems.
(Joel Beasley at 00:40:25) Watch out, Bezos. Matt's team, you guys are going to end up being the domination in the world. It's just this stuff—I just get excited and geeky. As you were describing it, it reminded me of an immune system, right?
(Matt Fornaciari at 00:40:39) Exactly.
(Joel Beasley at 00:40:39) And this concept that I recently heard about called antifragile. And I hadn't—I can't remember who I heard. I think I heard from Rand, like Rands in Repose.
(Matt Fornaciari at 00:40:50) Yep.
(Joel Beasley at 00:40:51) Michael Lopp. But yeah, he talked about this concept of antifragility, and it just sounded really, really cool. But that brings me to another question I have for you. So, you know, when you're talking about filling up the disk and you mentioned earlier, you're having to teach a lot—to explain these concepts and understand not only how to do them, but how to do them in the context of your system. How has that shaped your company? Do you have people that specifically make educational content?
(Matt Fornaciari at 00:41:19) 100%. We have one of the best advocacy teams I've seen. You know, it's run by Tammy Butow, former Dropbox SRE, who's just amazing. We've got Jason Yee and Ana Medina on that team as well, as well as a guy by the name of Patrick Higgins, who went to go work on the Elizabeth Warren campaign after working for us for a bit, then came back. And they're all just—they're so passionate about teaching. They're passionate about building up these bootcamp programs and getting people to sort of figure out how to build reliable systems instead of just writing some code, right? How to think about this when you're architecting things from the ground up. So they're very much into that. Our marketing team also does a fantastic job of building out like, look, use these technologies. Here are the four things you have to do in order to be resilient, to be reliable. So we do a lot of work. You know, I would say that it's a team effort. Everybody does such a great job, and there's no one person. But it's one of the sort of necessities of category creation—sometimes you've got to go out there and teach people. And so we've leaned really hard into that, and we want to do all that we can to teach people, not just people in industry right now. You know, this quarter we're going to spend a little bit of time with some groups that support younger kids that are just coming out of high school or some of the underrepresented minority groups to basically be like, cool, you want to become an engineer? Here's some stuff that you might not be taught in your regular curriculum in school. Here's some things that, you know, you want to be thinking about as you go out into this world and really start to enter the industry.
(Joel Beasley at 00:42:56) Well, yeah. It's like your top of funnel. I mean, Apple put the code playground on the tablet for the kids. You know, they can code little—
(Matt Fornaciari at 00:43:02) So—
(Joel Beasley at 00:43:02) Get people coding or interested in understanding, you know, the product. And it's not nefarious at all. It's very useful because just, like, here are the real world things you're going to run into, and let's learn.
(Matt Fornaciari at 00:43:14) 100%.
(Joel Beasley at 00:43:14) Man, I wish I would've—I've gotten to interview people, and they, like, accidentally, you know, just or just by chance, like, got on, was on a good team from the beginning, learned best practices as day one. And I'm just like, oh, man. Because I did it wrong for, like, so long and then got frustrated and learned how to do it right.
(Matt Fornaciari at 00:43:33) I mean, I felt the same way. I didn't really do internships like the way some of my colleagues did when I was in college. I worked at a small, like, contractor, with a small firm down the street from where I grew up and, you know, spent my afternoons just kinda hanging out writing some code. I got to write some cool stuff. I got to write some, like, the check-in technology for the Clinton Global Initiative and stuff like that, but it wasn't "let's learn best practices, here's a mentor," that sort of thing.
(Matt Fornaciari at 00:44:00) It was very much trial by fire. As you mentioned, I consider myself very lucky and very fortunate to have landed on the team I did at Amazon. I got to see so much and really got to see not just a great organization in terms of being able to get out code and really have that customer focus and that drive, but also a really fantastic operations organization. Like, being able to operate this thing at such scale. And I think while I was there, we spun out China and India. We spun out a bunch of new regions as well.
(Matt Fornaciari at 00:44:32) And to see all that kind of happen—honestly, I don't think I appreciated it enough when I was there, and I still appreciated it a hell of a lot. So—
(Joel Beasley at 00:44:43) I'm curious to know about, like, how your sales org is set up, how you get customers. Are most people—do they go through a free trial or is it B2B, SMB, or enterprise? Like, what do you target?
(Matt Fornaciari at 00:44:55) Yeah. It's primarily enterprise, but we do have a lot of folks that come in through the free trial. We have a free offering that we launched about a year, year and a half ago, I wanna say now, that, you know, has out of the box—it has being able to shut down any host and being able to black hole any sort of traffic. You know, we actually give you a playground. We give you two hosts to go play around with.
(Matt Fornaciari at 00:45:15) And for a lot of people, that's all they really need to get to that moment. And they're like, cool. My company can use this. Let's talk. So there is a little bit of a bottoms up, but for the most part, you know, enterprise is sort of what we're targeting.
(Matt Fornaciari at 00:45:27) And that's just because, you know, when you have these larger companies, the downtime, it just costs a lot more for them. At Amazon, we were able to see if, like, hey, if we lose—if we do, we have a 20% order drop for this amount of time. If it's twenty minutes, we lose $4,000,000 or something. I'm making these numbers up, and this is a long time ago. But it's easier for them to kind of make that value connection a little bit sooner.
(Matt Fornaciari at 00:45:52) So some of the SMB, you know, occasionally will see, like, look, we've already got enough fires. We've already got enough problems going on. And that's, again, another education opportunity. That's for us to say, like, cool. You're in the middle of a migration to Kubernetes. Well, you can test your reliability as you move. Right? This is a whole new technology for you. But, you know, again, time is one of the biggest sort of issues for developers.
(Matt Fornaciari at 00:46:16) Right? There's no shortage of things to ever do, and so getting them to prioritize reliability engineering, chaos engineering, is always—you know, if they don't have monitoring or something like that, there's always a bit of, like, well, cool. Let me get this in place first and then we'll talk.
(Joel Beasley at 00:46:31) Yeah. That's interesting because at the larger companies, you're gonna have, like, dedicated reliability engineers. Whereas, you know, at a company maybe with, like, 50 people, that might be one or two people's side job or something, just a role that they kinda own. But it's really just, you know, checking the logs. But it sounds like what you're doing is you're making software that's for that role.
(Matt Fornaciari at 00:46:53) Yeah. A 100%. But it can be used by just about anybody. You know, a reliability program is—I mean, it's always great. It's a little bit of kind of command and control a little bit, which is unfortunate.
(Matt Fornaciari at 00:47:05) Like, I always draw the analogy back to the FMEA's team at Amazon because I wrote this software that essentially would start to cut tickets to people, and it all had my face on it, and so it made me the bad guy. If you go and do that with a reliability program, that makes the reliability team the bad, you know, the bad people sometimes. So, I mean, really, yes, if you have an SRE team, they can use the software. They can build out a program and hand it over to new feature teams, application teams as they roll new software out. But, ideally, you know, nobody knows their applications better than the actual application developers, and so we want them to understand how to use this as well.
(Matt Fornaciari at 00:47:39) And it's a different persona, a different identity, you know, to bring it back to what we're talking about a little bit earlier, but there's 100% a path for them as well to be like, cool. I use these technologies. I have two, three microservices. Let me run through X, Y, and Z. You know, according to Gremlin, according to the experts, these are the things we gotta do.
(Joel Beasley at 00:47:58) Do you guys have a podcast?
(Matt Fornaciari at 00:48:00) We did have a podcast for a little bit. We were interviewing a bunch of folks in the, uh, in sort of in the industry, and, you know, we just—we didn't have the time to sort of continue doing that particular avenue. We're just still a small company, and there's just so many things to do. You know? So after I think it was about eight or nine episodes, but it was called Breaking Things on Purpose.
(Matt Fornaciari at 00:48:21) If you wanna go find them, listen to some backlogs, there are some pretty awesome interviews with some really interesting folks on there.
(Joel Beasley at 00:48:28) That's good. I'm glad that you, like, even if you're not putting out new episodes that you left it up. I get so frustrated when people, like, oh, yeah. We did, you know, 10, 15, and we took it down. I'm like, what are you doing?
(Joel Beasley at 00:48:38) Leave it up, you know.
(Matt Fornaciari at 00:48:39) You got 10 episodes.
(Joel Beasley at 00:48:40) It's like, leave it going. No. It's good because the—I'm actually kinda interested in the site reliability engineers and, you know, what they do on a day-to-day basis. Because I never worked at a company that was that big that we had dedicated reliability engineers.
(Matt Fornaciari at 00:48:59) It's a really interesting role, and I think, again, this is one of the roles that kind of varies from company to company, you know, depending on sort of whether or not you have a centralized platform team or if these are just the folks that run around and, you know, enforce best practices or if they're running a major migration to, you know—I mentioned Kubernetes earlier—to something like Kubernetes off of bare metal. You know, they're running a program like that so that you can have a little bit more of those resilience mechanisms baked in. It's just a different role, you know, sort of wherever you go. And so I encourage you to talk to many of them, not just one, because I think you'll get more of a wider view of sort of what the responsibility can be.
(Joel Beasley at 00:49:39) What I wanna know. Do you guys write—have you written a book? Like, or is there a book written on Site Reliability Engineers?
(Matt Fornaciari at 00:49:46) There are a couple books that have been put together, a couple of websites as well just on site reliability engineering. They kind of all take a similar viewpoint, although, you know, they try to publish or they try to sort of espouse some pillars, some best principles. There's a little bit of, you know, is that the case? There's a little bit of conversation around cool. Is it people and processes more so?
(Matt Fornaciari at 00:50:08) Is it the actual applications and services, the infrastructure? Yeah. There's a lot of material out there, and there's more and more by the day.
(Joel Beasley at 00:50:18) Is there a book on chaos engineering specifically? Like, is there a book titled Chaos Engineering?
(Matt Fornaciari at 00:50:24) I believe there's a book on chaos engineering. Yes. I think it's an O'Reilly book.
(Joel Beasley at 00:50:28) I'm gonna Google it real quick.
(Matt Fornaciari at 00:50:30) Go for it. So the practice—I mean, the practice started way back when in a bunch of different places, but I think this was one that came out of Netflix after sort of the Chaos Monkey explosion in popularity. So—
(Joel Beasley at 00:50:42) Yeah. That's like the big image that comes up. It's got the chaos monkey. It's got, oh, your ads. What is chaos engineering?
(Joel Beasley at 00:50:49) I'm not gonna click on it. I'm not gonna cost you money.
(Matt Fornaciari at 00:50:54) Oh, I appreciate it. But—
(Joel Beasley at 00:50:56) Then it's got a Wikipedia entry, and then you guys should get—if you're not on the Wikipedia page, you guys gotta get on there.
(Matt Fornaciari at 00:51:02) I think we are right now.
(Joel Beasley at 00:51:04) There's not—you know what? It's even—as a note here from my producer, he said he couldn't find competitors to you guys, and I was like, because they're the best. The people don't even wanna compete. They're just like, Gremlin solved the problem. We're not even going there.
(Matt Fornaciari at 00:51:19) I don't know. That's quite true. I mean, I think we have competitors, but in a very—maybe not necessarily in the traditional sense. One of the things that we see a lot from some of these enterprise companies is sort of the conversation of build versus buy. So like I said earlier in the podcast, my title used to be Chief Chaos Engineer, pretty cool title.
(Matt Fornaciari at 00:51:39) Turns out a lot of people want to be chaos engineers. They really like breaking things. And so, you know, oftentimes—well, not oftentimes, but occasionally—we'll see some companies that are like, hey. We think we can build this internally. We're gonna go ahead and just try to spin this up.
(Matt Fornaciari at 00:51:54) It's, you know, the thing that we end up seeing usually is, like, cool. Really easy to break stuff. Really hard to put it back together the right way. And if you wanna do chaos engineering in a very safe, secure, and simple way, which are our three product principles, by the way. You know, it's a much more difficult problem to solve.
(Matt Fornaciari at 00:52:13) And so you'll oftentimes see some of these companies be like, hey. Let's go ahead and try this. And what they'll probably end up doing is there's, you know, myriad sort of open source solutions that are very specific, very targeted. We'll do one or two different things. They'll try to sort of cobble those together into a full-fledged solution.
(Matt Fornaciari at 00:52:33) But what ends up happening is, you know, one, things end up breaking because that's how software works. And then, two, you know, there's more and more sort of failure modes that they want to include, and so it's tack this on and bolt this on and tack this on, and it becomes a little bit unwieldy pretty quickly. So a lot of that is sort of where we're actually seeing the competition right now is more internal for these particular companies and then trying to take some of these particular solutions and sort of mush them into one big thing. Beyond that, you know, we've seen a little bit from some of the cloud providers come out. I think Amazon Aurora, I'm not sure if you're familiar with Amazon database, they came out with the ability to inject a little bit of failure via SQL-based code, which is, or SQL, like SQL syntax, which is kinda cool.
(Matt Fornaciari at 00:53:21) It's a very definitely a very interesting way. But, you know, nothing is sort of holistic. And, I mean, that's largely because, you know, some of these larger cloud providers, like, they just do everything. Right? So these are all—our native offerings end up being sort of, look, you can start to use this.
(Matt Fornaciari at 00:53:39) This is more for beginners. If you want something dedicated, you know, that's when you start to go towards the people that really focus on it. So—
(Joel Beasley at 00:53:46) Yeah. And I don't know. I'm just—I wish you guys were publicly traded because I'd buy some stock.
(Matt Fornaciari at 00:53:53) I'll let you know when we IPO. Don't worry.
(Joel Beasley at 00:53:55) Please do. We did it, man. We made a podcast. How do you feel?
(Matt Fornaciari at 00:54:01) That was great. I actually really enjoyed that. I'm always kinda like, what am I going into? How do these work? And, you know, luckily, with Modern CTO, I actually went a little bit into your backlog.
(Matt Fornaciari at 00:54:11) So I was like, okay. This will be fine. Joel's a great guy.
(Joel Beasley at 00:54:14) Oh, nice. Thank you.
(Matt Fornaciari at 00:54:16) It feels great, man. I really appreciated you having me on as well. I feel very fortunate for the opportunity to chat with you.
(Joel Beasley at 00:54:22) Thank you so much, guys. Have a great day.
(Matt Fornaciari at 00:54:23) Thanks so much.
(Joel Beasley at 00:54:24) Thanks. Bye, guys. Thank you so much for listening. And if you found this episode useful, please share it with a friend or colleague who you think would get value from it. And if you have topics that you'd like to hear discussed on the podcast, either add me on LinkedIn or send me an email [email protected].
(Joel Beasley at 00:54:46) Every time I get an email or LinkedIn message, it absolutely makes my day and inspires me to keep going.