John Kodumal - CTO and Co-Founder at LaunchDarkly

(Joel Beasley at 00:00:00) Hello, my friends. Today we are talking to John, the CTO and co-founder at LaunchDarkly, and we discuss how to make use of their feature flagging systems to deploy code safely, the build versus buy dilemma as it relates to internal tools, and lessons learned from going through hypergrowth. All of this right here, right now on the Modern CTO Podcast. Here we go. This is the Modern CTO Podcast.

(Joel Beasley at 00:00:36) I saw that you like to go outside. You climb rocks, ice, boulders, small buildings. What's up with that?

(John at 00:00:43) Yeah, it's actually been a hobby of mine for a long time, maybe about 15 years, but I'm an avid climber. I climb basically anything I can get my hands on, like sport climbing, trad climbing, ice climbing. Last trip I took pre-COVID was an ice climbing trip with some folks on my team.

(Joel Beasley at 00:01:03) That's so cool. What's the one thing you need to care about when you're ice climbing?

(John at 00:01:08) Ice climbing is one of the ones where you can't really fall because you've got a lot of sharp things on you. You got crampons on your feet—they're sharp. Your tools are sharp. And when you fall, you're kind of at a minimum going to sprain an ankle falling ice climbing, and things can get a lot worse because, you know, imagine a taut rope with sharp objects kind of coming near it, and you have kind of a recipe for a bad situation.

(Joel Beasley at 00:01:37) Yeah, I will leave that to you and the experts. And I also saw that you invested in Rookout, and I know Liron over there.

(John at 00:01:47) Oh, that's amazing. Awesome. Yeah, they're a really interesting company doing some really interesting, hard technical stuff.

(Joel Beasley at 00:01:55) Right? Our team found that, I think, on Crunchbase. It's like, yeah, he's invested in one thing, Rookout. And I was like, no way, that's so cool.

(John at 00:02:01) Yeah, I've actually done a handful of angel investments. I think that the Rookout one might be the only one showing up on Crunchbase, but I've done quite a few.

(Joel Beasley at 00:02:08) Yeah, dude, what other things are you investing in? I want to know.

(John at 00:02:12) I invested in Coder. Not sure if you're familiar with Coder, but they're a cloud-based development platform. They help you streamline the process of creating dev environments for your teams that are always up to date, lets you spin up and work easily. Clubhouse—not that Clubhouse, the other Clubhouse—the issue tracking software, and a handful of others.

(Joel Beasley at 00:02:40) Oh, very cool. Yeah, I always like to—I mean, this is our world, right? I always like to talk to smart people, figure out what cool products are coming out, and I was excited.

(Joel Beasley at 00:02:48) I was talking about it with Jason over at GitHub, and then I found out that the CEO over at GitHub was also invested in Rookout. And I was like, this is such a small world.

(John at 00:02:58) Yeah, yeah, it's always interesting to see. I end up co-investing with some of the same circles because I do a lot in the developer tools space because it's the space I know best. And yeah, I've found that a lot of folks are getting the thesis around dev tools specifically and end up investing with a lot of the same people.

(Joel Beasley at 00:03:21) And you were at Atlassian before, right?

(John at 00:03:23) That's right. I was at Atlassian, I think, for six years. I joined when I think they were about 150 people, and when I left, it was just before the IPO. They were about, I think, 2,000, 2,500, something like that, people. And I left to start LaunchDarkly.

(Joel Beasley at 00:03:41) Dude, that is so cool. I was talking with Archana, and the thing I typically will remember, like one key thing from all these interviews I do. And she had this funny thing about, like, instead of eating your own dog food, drinking your own champagne. And I was like, that is classy. That is a much better way to say it.

(John at 00:03:58) We—that is a better way to say it. I like that, the champagne thing. And a funny thing was we call our—we have a separate server, a separate copy of LaunchDarkly for dog fooding, but we call it cat food. And there's a fun story behind that. We had a dog food server, and then we didn't deploy to it for like six months—this was really early in the days of LaunchDarkly—until we got to the point where we didn't know how to deploy to it anymore. So we literally had to, you know, burn it down and start over from scratch and make a new name for it, and we called it cat food instead. So we have a practice of cat fooding within LaunchDarkly, which—there you go. Champagne is still much better than that. Drinking your own champagne.

(Joel Beasley at 00:04:40) I've never had a server named Champagne, but it would sound classy though, right?

(John at 00:04:44) Yeah, that does sound pretty classy.

(Joel Beasley at 00:04:47) But I would be interested to know what the contents are within that. But if I came across the cat food server, I'd be like, I would ignore it. Security is—no.

(John at 00:04:56) You didn't know what you'd see. Yeah.

(Joel Beasley at 00:04:59) All right. So was LaunchDarkly like a divestiture spin-out? How did that come to be?

(John at 00:05:04) It is something that came about when my co-founder and I—we've known each other for a long time. We've known each other for about 20 years, and we at some point decided we were going to start a company together. We were just looking for the right opportunity. And the idea for LaunchDarkly came out of some of the things that I'd seen at Atlassian, where they kind of built an internal tool for feature flagging, for feature management. And it is just something that we created from scratch after I left Atlassian, after my co-founder Edith left her last gig.

(Joel Beasley at 00:05:36) How did you originally meet your co-founder?

(John at 00:05:39) We met in college. So, without revealing our age, we were in college at a time where there were still computer labs specifically for the math department. And so we were in a differential equations class together at Harvey Mudd College. And they had these old VAX machines, VAX VMS machines that people use to do their differential equations homework. And so we hung out some in that math lab. And then she went up to the Bay Area to work. I went up there for the Bay Area for grad school at Berkeley. And after that, we just stayed in touch. We were good friends. We never actually worked together before founding LaunchDarkly.

(Joel Beasley at 00:06:19) So what was the point? What was it like? You have this tool, this internal tool, you're going to spin it out. Was it the inspiration for it, or how closely are you connected with Atlassian, just so I understand?

(John at 00:06:31) I actually hadn't used the internal system at Atlassian when we started LaunchDarkly. It was something that I had heard one of my team members complain about. He had mentioned that at Atlassian they called it dark launching. And I made the mistake of thinking that was an industry standard term when it turned out that was mostly used at Atlassian and not very many other places, but that's where our name came from. Anyway, one of the people on my team was telling me about the first time he used the dark launching platform at Atlassian. And he mentioned all these pain points and frustrations, and it was a lot of the classic build versus buy story. There had been a team that had built this, and they'd really only thought about a subset of use cases. And they were based in Sydney. Atlassian is an Australian company at heart. And the Sydney team hadn't really thought of some of the needs that we had for our stack. We were based in San Francisco. And so it felt like an opportunity there, but I actually never used it directly. So it was something where I sort of intuited what needed to exist in that product in order for it to succeed.

(Joel Beasley at 00:07:46) That's pretty cool. What—tell me about what the product is, like the high-level overview.

(John at 00:07:50) Yeah. I think one of the critical things that it is at its core is it's part of the continuous integration, continuous delivery story. And it comes out of an explicit recognition that there's a difference between deploying software and releasing it to your end users. And those are two things that had historically in CI/CD practices been conflated, kind of mushed together into one thing. You're deploying—that means, you know, you take your service, you deploy it out, and you put it on a machine, and then you shift a load balancer to point to that server, and it's live and everybody's seeing it. And one of the things that you can recognize is that if you can separate those two steps, if you can think about deploying as just putting the artifact on a server and maybe releasing as exposing it to users and you treat those as separate things, you can—it just unlocks so much potential. You can, for example, reduce the blast radius of a release that you're doing. So, you know, you launch a new feature and instead of everybody seeing it immediately, you can roll it out to like 1% of your users or just a handful of beta testers. And if something goes wrong, you can just turn it off, and you've minimized the risk of that deploy, which is essentially one of the most important parts of continuous integration and continuous delivery. But the use cases are much broader than that. Now your PMs can sort of turn on some new piece of functionality just for a subset of the testers that they care about before releasing it more broadly. Or even in the long term, you can have some functionality that's exposed to some users because of plans or entitlement reasons and other functionality not available to others. And then finally, once you have feature flags in place, you sort of have this natural distinction between a control group and an experiment group or a test group. And so you can do your A/B testing or optimization or experimentation on top of the platform as well.

(Joel Beasley at 00:09:41) Yeah. That was probably the hardest thing for us to get going at the outset. People understood feature flagging. A lot of people had been exposed to feature flagging, but really only in a limited way because the tools—a lot of the homegrown tools, a lot of the open source tools—weren't very mature. They really restricted what you could do with feature flagging. And so once we came along and once we provided something that was, you know, much more first class or real commercial offering with a lot of the capabilities you needed, the use cases grew out of that and the need to educate people on those use cases grew as well.

(Joel Beasley at 00:10:26) Yeah, because I mean, my background is software engineering. So I was super excited to have this conversation because, you know, everyone—when you have a word or concept like that, feature flagging, everyone's going to have their own local decentralized understanding of it, right? And so I instantly thought, okay, well, my first questions are, like, are there types of features that you can't flag? For example, like features that require data migrations, right? Like, how are you doing—how does that happen? And is that something that LaunchDarkly solves, or is that something that's like, if I want to use feature flags more intensely, I have to learn a different way of structuring my code? Or can you talk to that? Am I making any sense?

(John at 00:11:06) Yeah, no, that makes perfect sense. And we get a lot of questions like that. With respect to database migrations, it's something where feature flags can sort of be the building block to enable something like that. We talk a little bit about this in some of our content, but one of the ideas that you can do is sort of have a code path that uses the old data model and a code path that uses the new data model. But in addition to that, you sort of need to enact a system of like dual writes or do a migration that's not destructive so that the data exists in its old form and its new form and updates are done to both the old version of the data and the new schema. But what you can do is use feature flags to sort of control which data model you're reading from, the old or the new, and control which code is running—the code that knows how to access the old data model or the code that knows how to access the new data model. And you can seamlessly shift back and forth as long as you're writing to both models and keeping them both in sync. And then you can do things like consistency checks. Hey, does the old system and the new system—are they agreeing on what the data should be? And if so, then maybe the new data is probably correct, so maybe we can cut over to that. It is something that you can do. It requires a little bit of advanced planning. Your broader question was, are there things that you can't really do with feature flagging alone? And there definitely are. An example that comes to mind is like certain kinds of infrastructure changes, like changes to a configuration file. They exist below your application code layer, and feature flags don't really manage those sorts of things because they live in your application code. So, let's say you changed a configuration file and you wanted to canary traffic between the old configuration file and the new configuration file. Or another example might be you've changed EC2 instance types to see how a new instance type that might be cheaper, might have different performance characteristics, performs. That's really hard to do with a feature flag. There, you're looking at something like, you know, a load balancer or canary launch, that kind of level that exists above the application layer, more at the router or the load balancing layer. And you do have capabilities to do that, and they're complementary to feature flagging.

(Joel Beasley at 00:13:18) What's the most ideal—and give me an in-the-weeds example of very straightforward feature flag?

(John at 00:13:26) Very straightforward feature flag. So, site maintenance mode, right? You just let everybody know that something's not right and you're working on it. The Twitter fail whale from history is an example. I know a lot of engineering practices have moved to a world where you want to bend and not break, and so you never want to use this feature flag. But it's a valuable thing to have in your back pocket of, there's an incident happening and rather than show a 503 page or just some raw content, you want to show that the site is down and that you're working on it. So you could feature flag that. Put in a code path that says render this static page that says things have gone wrong if things go wrong. And hopefully you never have to use that, but it's there if you do need it.

(Joel Beasley at 00:14:12) That's interesting. So you're like, instead of running over to do some redirection, right, and some higher privileged concept for directing to a status page or a down page or something like that, it's in there with the feature flags. And that's typically where you're going to be when things go wrong, right? So I'm flipping feature flags on like new things I'm trying and all of a sudden, whoops, I broke something. Now there's a switch right down in that same interface for me just to put up a wall.

(John at 00:14:39) Yeah, the site maintenance mode is more of an anomaly. It's not something that you hope to use every day. An example of something that might be more common is the ability to feature flag off specific features that might have a performance impact but that aren't essential to your site. A great example of that—let's say you run a search service like a Google or Bing or something like that, and there's some problem with autocomplete where it's erroring out. So if somebody starts typing a search and the autocomplete service is down, you still can use the search product. You just want to turn off autocomplete. And so you might feature flag that and say this non-critical service—it's very valuable, but it's not critical. I can still use the product without it. Let me feature flag that off because it's impacted and provide the user the best possible experience when some sub-service is in a degraded state.

(Joel Beasley at 00:15:27) But at the same time, if I didn't have autocomplete to my normal customers, but that was a new feature I was rolling out, I could enable it for—you call it a blast radius—I can enable it for a certain set of users and see how that happens before rolling it out to all of them.

(John at 00:15:43) Yeah, that's the sort of positive use case of I'm building something new and I want to see how it performs. I want to make sure it's working well before I do a broader rollout, and I want to make sure it's not negatively impacting performance. And I want to do that in a gradual way that's not, you know, it's just available to everybody and all of a sudden I'm hitting peak load kind of instantaneously with some new piece of infrastructure and some new feature that I launched.

(Joel Beasley at 00:16:07) So in this context that we're talking about now of application layer, right, that would mean that it doesn't really matter if it's on Heroku or AWS. It's going to be compatible because you're manipulating the application layer.

(John at 00:16:21) That's right. And that has been a tremendous advantage and value of the technique. It's sort of agnostic to the infrastructure, your deployment model, what cloud provider you're on, because there's so much fragmentation in the rest of the space, the market. So for us, all we have to do is support all the programming languages that are out there, which there's a lot. But when you look at that, it's much smaller than trying to support every possible ingress controller or every possible load balancer, every possible cloud provider. The fragmentation problem ends up being something that doesn't affect us as much. We just have to write SDKs effectively for all the programming languages that are in common use.

(Joel Beasley at 00:17:03) Nice. I hope you don't take my rapid questioning as interrogation. I'm just really excited.

(John at 00:17:09) No, this is one of these things where, you know, when we first thought about this idea, we didn't really understand fully what the impact was and just how powerful it was going to be. And so you can probably tell from the enthusiasm in my voice, there's just so much potential with what we're doing. The potential there is to kind of transform the way people do software development. And that is something that I didn't really see at the outset when we thought about building a SaaS offering for feature flagging or feature management.

(Joel Beasley at 00:17:42) How's your go-to-market? Right? Your cofounder, are you guys talking to CTOs, VPs? Are you going in directly, just like, here's an open source tool and getting people in it? How are you getting awareness of this?

(John at 00:17:55) You know, our customers are kind of all over the map. They range from Fortune 100 companies with thousands of engineers that are looking to roll out a solution across their entire organization to small startups with just a handful of team members that are just finding their product-market fit and getting their first customers on board. I think one of the things that has been very valuable for us is thinking about building a product that works for any team, you know, from a small startup to a large engineering organization at an enterprise company. One of the things that helps us there is we kind of view the organization of engineering teams as fractal in nature. So at a small startup, there's a single small team. As that company grows, they'll stamp out that squad as a template, and then there'll be maybe another fractal layer on top of that as you get growing the teams of teams. But by building a product that sort of works for a single team, we can support the smallest fractal unit within a large organization as well. And then it's just important for us to build the support for those larger teams going up the stack.

(Joel Beasley at 00:19:03) What are some of the executive level values? Like if you're talking to CTOs about implementing, maybe they're not writing software every day. Maybe they have 500 engineers.

(John at 00:19:18) There's a handful of things. It depends on where the organization is at in their software maturity. If they're an organization that is just going through the transformation of becoming a software-driven organization, we can talk about ourselves as fitting into the continuous integration and continuous delivery story. Sometimes when you're talking to an organization that's already sort of been there, done that and has a rapid cadence of deploys and pretty mature practices, we can talk about things like reducing incident impact through reducing mean time to remediation. With our platform, you can flip a feature flag within hundreds of milliseconds. And so, you know, even at great organizations, sometimes making a rollback or deploying a code change can take, you know, minimum minutes, but more likely order of hours. And so if we're able to come in and say, you know, a change was deployed and it broke, and we can remediate that error or fix that issue within seconds instead of hours or days, that's incredibly valuable to large organizations.

(Joel Beasley at 00:20:28) Yeah, because I mean, we're sitting there watching Jenkins run all of our tests. You got to push stuff. If you can just flip a flag and it's—that's fantastic. That's so interesting. I'm like, when I was writing code every day, I remember we were just barely experimenting with feature flags. Right? Like, oh, let's make this menu item something we can turn on and turn off. And then there was some Rails project—that's my primary toolkit. There was some Rails project that had some slightly more advanced features. Right? So, for example, the blast radius stuff. But I haven't looked at that market in a long time. And then when I was preparing for this episode, what you guys have done, it is so mature. It's on steroids. You've taken it so far. I was like a kid in a candy shop scrolling through your website, looking at all the stuff you guys have done with feature flags.

(John at 00:21:22) Yeah, there is so much stuff out there that we're just getting to. I mean, it seems like this boundless well of things that we can build, and everything that we can build is just incredibly impactful to the folks trying to use it. So we're doing this really novel stuff with edge compute now where we can do the feature flag determinations at the edge. And that means if you're on a mobile phone and getting a feature flag change on that mobile phone is near instantaneous. We're dropping things down by an order of magnitude in terms of getting the flags out to client-side contexts. And that's just something like, if you're building this in-house, it's really hard to think about doing that and then making that investment unless you're maybe a Netflix scale or an Uber scale company. And even at those companies they've built—they've got dedicated teams working on this, teams of 20 to 30 engineers at times working on the feature flagging platform, which seems like such a surprise when you just think about it as a conditional statement that's in your code.

(Joel Beasley at 00:22:24) So when you guys are selling into companies, I always like to look at sales objections. Right? And I'm curious, what are the most common sales objections you get?

(John at 00:22:34) Build versus buy is one of the first ones, because it is something that seems so easy to do on the surface. Until you kind of explain the value proposition and the downstream needs that an organization will have as they adopt this within a larger organization or engineering team, it seems like, you know, why do I even need a UI for this? Why isn't this just a table in my database? We get questions like that sometimes. And usually when we show the product, we show the capabilities, a lot of those objections get handled. I think one way that I think about it is we're something that impacts change on your production service, right? And most mature organizations have pretty substantial mature change management practices around anything that touches production. And a lot of what we've done is build that for you. Right? So you've got audit logging, role-based access controls, scheduling, approvals capabilities, all of that stuff is baked into our platform, and it's all stuff that you're going to have to build yourself if you decide to go the homegrown route. So build versus buy is definitely one of the most common objections we see.

(Joel Beasley at 00:23:45) Nice. Buy it. It's so cool.

(John at 00:23:48) Yeah, I mean, we have a pretty large team working on this, and you don't want to be building this yourself. You want to be focusing on your business and delivering value in your product directly, not building an internal tool to flip feature flags or to manage feature rollouts.

(Joel Beasley at 00:24:03) Yeah, another perspective too is I want to support it because what you're doing is you're out there helping all of these different people. So your company is actually becoming a leader in programming related to feature flags. We're talking about releasing content. And when you can get out there and see how all of these different companies are doing it, that's valuable to me. So I want your company to do good.

(John at 00:24:27) Exactly. I mean, we have thousands of customers using our platform today at some pretty large scale, and we've heard all their feedback. We know what a 5,000-person organization needs in terms of feature management. And we're going to get there and we're going to build it, you know, so that if you're a smaller customer, when you grow into that large organization size, it's already there for you because we've already thought of what you're going to need six months down the line or a year down the line or even beyond that.

(Joel Beasley at 00:24:54) So your organization's pretty mature then?

(John at 00:24:57) I think so. Yeah, this is my first company that I've ever founded, but I've had about a 20-year career almost exclusively in building developer tools or thinking about developer productivity. And so even from the initial stages of building LaunchDarkly, we knew we were going to be developer-focused, and we built things ground up with that kind of maturity in mind, knowing what the eventual requirements were going to be.

(Joel Beasley at 00:25:23) And then what's your—I want to know from cofounder, executive, your company's growing. I want to know what your day-to-day looks like.

(John at 00:25:32) It's all over the map. It's got a lot of meetings in it, but, you know, roughly speaking, I sort of see myself as wearing two hats. One is the CTO and one is a cofounder. With the CTO hat on, I'm thinking about all the different touch points between technology and the business that we're building, whether that's the product—I'm a very product-focused cofounder—or the technology as it impacts our employees, the ability to deliver tools for them or to unlock their ability to be more efficient in the things they do on a day-to-day basis. I spend a lot of time thinking about that. From the cofounder hat perspective, I think about the company that we're building and making sure that we're building an amazing company, not just for our customers and the product that we're delivering to them, but for the folks that are working at LaunchDarkly, trying to build the environment where they can do their best work and seeing how I can impact that as a cofounder.

(Joel Beasley at 00:26:27) How are you going to change the world just by creating a great culture and inviting everyone to come work for you?

(John at 00:26:33) You know, I think there's this old, almost clichéd, software changes the world. And one of the things I've thought about is how much better software makes the world, right? The fact that software has so positively impacted society, and our role kind of is to indirectly benefit society by making teams better at building software. And that's how I think about the impact of LaunchDarkly and the customer base that we have, the kinds of things people are doing with LaunchDarkly. I can't name all our customers, but what I can say confidently is it'd be pretty hard for you to get through a day without using a piece of software that is powered by LaunchDarkly in the back end.

(Joel Beasley at 00:27:16) Nice. I know, it's frustrating. You work so hard to get the Fortune 100 clients, and then in the contracts, they don't let you use their brand names for marketing.

(John at 00:27:25) I'm okay with it. You know, I would love to use those names, but at the same time, I have the internal satisfaction of knowing the positive impact that we have at some of these businesses.

(Joel Beasley at 00:27:34) Yes, there's the bright side. So tell me a little bit about, what does it mean to be working at LaunchDarkly? What's the culture like?

(John at 00:27:43) Culture is something that we focused a ton of effort on, and it's been a real challenge over the past year. Like a lot of other companies, the culture shift for us has been pretty significant. We are an Oakland-based company. You know, if you're not familiar with the Bay Area, there's sort of San Francisco, there's San Jose, there's Oakland. You know, six years ago when we started LaunchDarkly, there wasn't that much tech in Oakland. And so we prided ourselves on being part of the Oakland community and not, you know, creating a walled garden between LaunchDarkly and Oakland and the part of Oakland that we were in, but rather really feeling like we wanted to be a part of it. And that was a really a real key part of our cultural identity. And as COVID hit and location stopped—physical proximity to Oakland and people being in the office stopped being so prominent in our daily lives—it really felt like there was a gap there that we had to fill in terms of building the culture at LaunchDarkly. So that was a real challenge for us, in addition to all the, hey, now everybody's remote, we're all in the deep end here, let's figure it out, in addition to all of those challenges.

(Joel Beasley at 00:28:54) Was it a hard transition or was everybody pretty well suited for it?

(John at 00:28:58) It was both hard and easy. Easy in some of the operational aspects from the perspective of running the service. I think a lot of the best practices and controls that you put in place to run an always-on mature service, they're very highly compatible with the idea of working remotely. Like, if you have 24/7 on-call, you need to be able to manage the service remotely from your home anyway. So that part was not a big shift. The hard parts have been sort of the unanticipated impacts of being so isolated for an entire year. Hiring, you know, basically a third to half of our workforce has never been together because we've grown so quickly during COVID that so many of us just had never experienced this six-year-long culture of working together and being in the office. And so when we come back together, it's going to be fascinating. Our culture is going to shift into something new that never existed before. And I'm not really sure what that is. And that's kind of scary. It's exciting, but it's also really scary.

(Joel Beasley at 00:30:08) We started doing quarterly get-togethers. So we just fly, travel everyone in, get together, go to a restaurant or something. And because we were all in the office before COVID, and then COVID happened and our business model changed slightly. So we lost about half of our people. And then we've grown significantly since then. But the new hires are all across the United States. Right? So we've got this Slack office culture and these, you know, stand-up Zoom. We're about 15 people. We're not as large as you guys. You guys are huge. But that is where we're at. So we've got those cultures, but then when we all get together, it's like this new thing emerges and it's fantastic. It's like, this is what we're like in person. And it's different than online, but in a good way.

(John at 00:30:57) Yeah, it's just impossible to capture the nuance of face-to-face interaction through Zoom. There's sort of this, what was that person's intent when they said this? And then everything you say feels so much more deliberate because only one person can sort of talk at a time. So you can't really get into a room and just chitchat with, you know, and then socialize with a group of 10 people.

(John at 00:31:19) And so all of that social interaction is missing. All of that wealth that we build up through those small social cues is just gone, and it's been replaced by all of these postage-sized stamp interactions on a computer screen with each other, which are great. I can't imagine what the pandemic would have been like before Zoom existed. You know, we're really lucky that the technology got to the point where it's at now when this happened.

(John at 00:31:46) Otherwise, using the technology from ten years ago would have made this incredibly painful.

(Joel Beasley at 00:31:52) I agree. My emoji game, though, is through the roof since COVID. I used the bowl and spoon emoji today to talk about eating oatmeal.

(John at 00:32:02) That is next level. Yeah. We have a specific Slack channel for new emojis. It's like the new emoji room, and that thing goes off sometimes. Sometimes there'll be like 40 new pig-related emojis. It's like, pigs? Really? That's what we're—okay. But yeah, we have a really strong emoji game within LaunchDarkly.

(Joel Beasley at 00:32:21) I love it. I didn't know until COVID that you can make custom emojis inside of Slack. Like, you could take someone's photo. There's a photo of me in some interview I did this, like, "Yeah." Right? And so they took a screen grab of that and made it an emoji, and they use it when we make a sale or someone's excited. It's hilarious.

(John at 00:32:45) You can do really interesting things too, you know, aside from all the humorous, awesome things you can do with emojis in Slack. We're starting to build this practice of triggering actions based on emojis. So during incidents, you know, we'll have an incident channel, and you can star a Slack comment and that'll get pushed into our incident management SRE tool of, like, this is a follow-up item. Really cool stuff. There's so many ways to use that within Slack.

(Joel Beasley at 00:33:11) That's like a responsible way to use it.

(John at 00:33:13) I know. I know. I had to be a killjoy, take all the fun out of the emoji game and, like, well, actually, there's a really useful thing to do with emojis here.

(Joel Beasley at 00:33:21) It's like, thanks, Dad.

(John at 00:33:24) Yeah. Absolutely.

(Joel Beasley at 00:33:25) So what are you learning right now as a leader? Like, you're leading this company, you're going through this growth, you're expanding. What are you focused on?

(John at 00:33:34) I'll focus on a very concrete thing, which is a thing that I've learned more as a founder is that you set the framework for the culture and it evolves. And sometimes it can evolve beyond what you originally intended. And that can force you as a leader to react differently into situations based on what the culture has become. Sounds kind of hand-wavy, but a concrete example of this is we recently communicated a change in one of our on-call processes, and it honestly didn't go over very well. We did not do a great job communicating this.

(John at 00:34:07) And so I looked at this and I thought, well, this is, you know, at all the companies that I've ever been at, this is how a change like this would have been communicated. And from a sort of quote-unquote "best practices" perspective on change communication, we did all the right things. We worked with stakeholders, and we rolled it out, and we created a Confluence page describing all the details. And what I realized is that we have intentionally created a very collaborative culture at LaunchDarkly, one where people want to be involved in decisions and want to have input into the process. And we failed to respect that in this particular incident—in this particular situation. And it's just one of those things where I realized that the culture makes demands of me as a leader that I have to react to.

(John at 00:34:55) I have to change my leadership style to reflect how the team wants the culture to operate here. And it goes even beyond what Edith and I, my co-founder and I, initially created or developed. And that's incredible. It's a challenge every day that all the leadership at LaunchDarkly has to live up to, and we don't always do it. And it's fascinating to me, and I love being challenged and pushed in that way.

(Joel Beasley at 00:35:25) Yeah. Well, it's hard. That's why I like doing the founder path. It's difficult, but it's fun and rewarding.

(John at 00:35:30) It is incredibly hard. It is the hardest thing I've ever done. And I always say it's my one at-bat because just it's been so challenging and so hard that I can't imagine trying to do it again from scratch.

(Joel Beasley at 00:35:44) Dude, it's tough. It is super, super tough. But I'll tell you what, if you had to do it again from scratch, you'd have a few minutes of self-pity, and then you'd be like, nope, that's not who I am, and you would just do it. You would just do it again.

(Joel Beasley at 00:35:56) Yeah.

(John at 00:35:56) Yeah. Well, I'm thinking I probably won't ever be in that position, you know, given where LaunchDarkly is and the opportunity sitting in front of us. But I think my personality—you're probably right. I probably would just roll up my sleeves and go if I had to.

(Joel Beasley at 00:36:12) Well, because, alright, so, you know, our business is at the point where it's just snowballing and it's self-growing and, like, I'm aware that this is a system that will continue. Right?

(John at 00:36:22) Mhmm.

(Joel Beasley at 00:36:22) And so there's some comfort to that because at the beginning, it's very uncertain and it's very difficult at the beginning. And so when you start getting that revenue happening and it's consistent and it's growing consistently and things are doing well, you're like, okay. But still, challenges appear in all areas of life. Right? So it's not like all the challenges are just isolated into business revenue. So your personality, your ability to handle and weather those challenges and respond to them—you are still exercising that muscle.

(John at 00:36:52) Yeah, absolutely. I do have this sort of—there's a thing about my personality where I view all challenges as sort of existential crises. You know, like, if we fail at this, we will no longer exist. And that's really, really valuable early on because you're kind of looking at every decision in an early-stage startup with extreme urgency, like this decision could lead to us failing in this way or that way, or if we don't get this right, we're not going to succeed. And it's an odd thing.

(John at 00:37:22) Maybe I can't tell if it's a good thing or a bad thing, if it's helpful or if it's not helpful, but I still look at many decisions as like, if we don't get this right, we're not going to survive or whatever. And that's clearly, very clearly, not true at the size that we're at now. But I still kind of think about it from that perspective. It's probably not a good thing. I should probably not be behaving that way or thinking about things that way.

(Joel Beasley at 00:37:45) I think it's like a spectrum. Right? You've got the people that are like, only the paranoid survive. You got people that are like, everything's going to work out. And so the hard part I have found is not living at one extreme or another. It's sliding across the spectrum given the context of the situation. That tends to be the most difficult. Whoever created this game called life, that tends to be the most difficult thing that keeps you on your toes. Right?

(John at 00:38:07) Yeah. I think that's a very Buddhist way of thinking about it. Right? Like, everything in moderation. And so that would be a useful thing to apply in my thinking, I think.

(Joel Beasley at 00:38:17) So have you surrounded yourself with a nice group of mentors who've done this before? How do you talk? How do you dish? How do you get this stuff off your chest privately?

(John at 00:38:27) Great relationship with my co-founder is kind of first and foremost. I think, you know, we are in a state now where we can be incredibly candid with each other. Sometimes we can get frustrated with each other, but we always fall back to, you know, we both are trying to get to the same destination. We might just have different ideas on how to get to that destination. But that leads to an incredible sense of trust and an incredible ability to sort of share things.

(John at 00:38:53) Because, you know, when you co-found a company, when it gets to a certain size, even with your most senior leadership, you can't necessarily have that. You can't necessarily be that candid. So having at least one person that understands the business very, very, very thoroughly, understands what you're going through, and can be a person that you just vent to if you need to is incredibly important to me.

(Joel Beasley at 00:39:16) Yeah. Venting is an interesting thing. The ability to bounce ideas off or just to hear yourself talk them out with someone else that's a stakeholder that understands the situation. It's incredibly important because as you hear yourself talk it out, you are learning a lot about the situation.

(John at 00:39:36) Yeah. There's a term for that in programming where you sort of explain the problem to someone else and you sort of explicitly say, I'm not looking for you to give me feedback or even brainstorm with me. I just want you to listen to the words coming out of my mouth because I'll figure out the solution through the act of talking through it. It's like teddy bear programming or something like this. I can't look it up, but it's something that I've heard about in a few different contexts.

(Joel Beasley at 00:40:01) Yeah. Well, we've all experienced it. I'm glad they put a word on it.

(John at 00:40:05) Yeah. It's a really incredible phenomenon, especially for people that think in a more auditory fashion.

(Joel Beasley at 00:40:15) Now, has anyone ever given you some leadership advice that really, really stuck with you as you've grown over the past couple of years?

(John at 00:40:24) That's a good one. I'll have to think about that for a second or two. There's a number of them. I'm trying to bypass some of the generic ones, like, you know, find leaders that you can trust or things like this.

(Joel Beasley at 00:40:38) And it's okay because some of the generic ones are the ones that are most true. Like, I love clichés because the reason the definition of the cliché is that it's a cliché. Like, everybody understands it. It's been said ad nauseam. Right? Like, everybody understands it, and they understand it because it's true. And so I don't discount clichés.

(John at 00:40:58) Yeah. I'll go with, you know, one of the things that I took from Atlassian and the founders of Atlassian, Mike and Scott, was the value of transparency. And the way I think about it is the more information you can share with people within your organization, the better they're equipped to make decisions that are the best decision for the business because they have that context. Another way to think about it is, you know, as a founder or as a senior leader in an organization, a lot of times the reason you can make good decisions is because you have more context than anyone else. And so to the degree to which you're able to share or distribute that information, that's the degree to which you're able to empower everyone within an organization to make good decisions.

(John at 00:41:45) And that requires enormous amounts of transparency. It can be counterintuitive. Sometimes it feels like sharing a piece of information when, you know, it hasn't been completely decided, when it's only been partially hammered out, can lead to more frustration and sort of indecision or second-guessing among people, especially when you have to arrive at the right decision. But I believe that if you provide that transparency and are very clear about where you're at in a specific decision-making process, you'll make the whole organization more efficient.

(Joel Beasley at 00:42:17) Yeah. Setting the tone up. One thing I learned from a past guest was setting the tone of a conversation. So going into it and saying, you know, exactly what you want out of it. Like, this is just going to be an ideation session. Like, we're not making decisions in here. We're just bouncing ideas. And so for the next, you know, 15, 20 minutes, we're just going to explore ideas. But before that, one of the mistakes I was making as a leader is in my head, I'll be like, I'm going to go explore some ideas. And then I go grab one of my executives and I just start talking with them about some ideas.

(Joel Beasley at 00:42:47) And to them, they're like, what is this guy doing? We're doing these other projects and he's talking about all this other stuff. And I was like, oh, I needed a moment of their time for this specific task, you know. And so by setting that upfront, it helped.

(John at 00:42:59) Yeah. Yeah. I like that. I'm also very explicit about venting. You know, back to the earlier topic, it's sort of like when I'm venting, I'm going to tell you in advance that I'm venting. So I'm not really looking for anything other than to get something off my chest. I'm not looking for a solution or feedback on what I did wrong or anything like that in that particular context. So I find that very valuable as well, just sort of being explicit about what you're looking for when you say something. I'm looking for a decision here or I'm just here to vent about something or I just want somebody to listen. Those are all useful things to be clear about at the start of the conversation.

(Joel Beasley at 00:43:34) Yes. Yes. Well, I'm curious. I've got a question about the size of your company. So I get to interview people of all sizes from, you know, single founder, just building their first product, maybe with a team of two or three, up to, you know, Fortune 100-type companies. And so I'm always interested and I've kind of become a geek to talking with these people and understanding the differences in the organizational structures and sizes. At your size of organization, typically, that's right when they start doing career paths, things of that nature, because people want to see forward. They're getting disconnected far enough from the founding team that, you know, you're becoming a more mature organization. Where are you at? Have you done that yet?

(Joel Beasley at 00:44:21) The career progress paths type stuff?

(John at 00:44:25) We have. And that was something that happened in the past year, say, where we created explicit levels and more explicit pathing. And it is largely rolled out for the roles that we have at scale. So engineers, AEs, those types of folks. Where it can be lagging sometimes is in the roles that don't scale as proportionally to the organization size.

(John at 00:44:50) Like, for example, a guideline that I've heard is like one to one-and-a-half to two-ish percent of your staff should be dedicated towards security. Right? So, you know, even at a midsize company like ours, that's only a few people. And so we haven't been as quite as on top of providing career pathing and our strategy for those roles that, you know, where there's not hundreds of them at LaunchDarkly, for example.

(Joel Beasley at 00:45:18) How did you recognize—like, take me back to the executive conversations of when this became a problem in the sense that you realized the need that we have to do this.

(John at 00:45:30) There were a few triggers. One of them was the need to hire with more clarity at scale. Another was thinking about equity and equality within the team and making sure that people's salaries made sense relative to each other. You know, at a certain size, people start sharing their own salaries and, you know, at early-stage startups, things are all over the map. You know, you're struggling to attract talent and you don't have the leveling system in place.

(John at 00:46:00) And so you're like, I don't know. What do you think this person would take if we made them an offer? And so you end up with titles that are all over the map, comp that's all over the map. And once you get to a scale where people start sharing that, they're kind of expecting you to have more structure in place, and they're surprised when there isn't. And so, you know, the minute a salary gets shared that's substantially lower than somebody else's—

(John at 00:46:26) It feels like there's maybe some inequality there, and sometimes that's more a factor of, you know, just we didn't have HR when this person was hired or situations like that. And so you go through this deliberate process of recognizing you need to do it, and then the longer process of recalibrating everyone and putting everyone on a more level playing field.

(Joel Beasley at 00:46:49) So did you end up doing, like, a specific software with that career pathing, or did you do it through, like, Google Docs? How did you actually do it?

(John at 00:46:58) We used an external consultant, someone that had, you know, knew kind of what they needed at a stage-appropriate size, someone that had access to industry data. For example, it's pretty standard to use things like Radford for benchmark comparisons on salary. And so we brought in folks that had experience with that. And, you know, from a tools perspective, to be honest, we didn't really have great tools for this. There's still a lot of spreadsheet-driven organizations from that front.

(John at 00:47:32) And I've talked to peers or folks that are at companies that are a few stages bigger than us or ahead of LaunchDarkly, and it is kind of surprising how the tools are not as mature as you'd hope, and they're still a little bit more Excel in the wild than you'd hope.

(Joel Beasley at 00:47:50) No, exactly. That's why I ask around. And one of the reasons why I'm always curious about this type of stuff is when I find that a lot of things are still being, like, Google Doc or spreadsheet, whatever it may be, that typically is a sign of, like, what I call market pressure. So it's just not a big enough problem to solve with outside funds yet. Like, it's good enough, you know.

(Joel Beasley at 00:48:13) As Bill Nye—he's one of my favorite people—like, he has this concept that made me feel way better about life. When he was telling me, like, DNA and stuff, he's like, things move forward when they're good enough. They don't have to be perfect. They just have to be not that bad to get extinct.

(John at 00:48:28) Yeah, yeah, yeah. You know, I think one of the things that's challenging is people don't really know how immature some of those tools are, and people do not, or don't receive it well when you make a mistake that involves comp or benefits, right?

(John at 00:48:43) And so you never want to make a mistake on that front. It should just be like water. It should just work whenever you need it, and it should always be right. And sometimes these tools, or the lack of tooling, causes a lot of things to be hand-entered. People, when they don't realize exactly what the processes look like at an organization our size, or even organizations that are bigger than us, they can be sort of really surprised to learn that, hey, there was a manual entry error here.

(John at 00:49:12) But that happens far more frequently than you'd imagine.

(Joel Beasley at 00:49:15) Yeah. Dude, this was awesome. I very much enjoyed you. This is great. Thank you so much for listening.

(Joel Beasley at 00:49:25) And if you found this episode useful, please share it with a friend or colleague who you think would get value from it. And if you have topics that you'd like to hear discussed on the podcast, either add me on LinkedIn or send me an email, [email protected]. Every time I get an email or LinkedIn message, it absolutely makes my day and inspires me to keep going.