Maximizing the Velocity of Tech Teams with Gahl Saraf, Co-Founder & CTO at Raftt

(Intro Narrator at 00:00:00) Today we're talking to Gahl, the co-founder and CTO of Raftt, about increasing developer velocity, leading during a time of national crisis, and more. You're listening to Joel Beasley, Modern CTO.

(Joel Beasley at 00:00:18) But first and foremost, I'm sure you're getting this question a lot, right? How are you doing right now?

(Gahl at 00:00:24) So this week has been quite a shock. It's been quite disturbing as well. I am personally doing fine and my family is fine, which I'm really thankful for. But a lot of people are now nowhere near as fine, both if they're personally affected or they know people firsthand. I mean, it's over a thousand people right now have been announced as murdered or kidnapped. And so almost everyone in Israel knows someone, if not directly, then through their friends, through their family. It's pretty bad.

(Joel Beasley at 00:00:55) Yeah. And are you located in the city?

(Gahl at 00:00:58) Yeah, I'm located in Givatayim, which is right next to Tel Aviv. It's right in the center of Israel. So it's one of the safer places generally, and we haven't felt it directly here geographically speaking. But it's been such a shock and such a huge event that its rippled effects have reached everywhere.

(Joel Beasley at 00:01:19) Is most of your team in Israel, or are you guys all remote?

(Gahl at 00:01:24) Yeah, our whole team right now is in Israel. So it's also been interesting and it's been difficult to support the team as well as we can.

(Joel Beasley at 00:01:35) Yeah, that's my next question—leadership, right? Like, you are the face of the company. You're one of the co-founders, correct?

(Gahl at 00:01:42) Yes, that's right.

(Joel Beasley at 00:01:43) Yeah. So you have to lead this team through it and somehow focus on work. How do you even go about that? We need people to do work. You have to focus on something, but at the same time, there's a lot going on around us. So how do you deal with that?

(Gahl at 00:02:00) Yeah, I think to be honest, making sure that the team is able to work effectively was not top of mind in the beginning of this week, and I think it's going to be a little bit more important as we go along. But the first priority was to make sure that people are okay physically, and then to make sure that they're able to be in a safe space mentally. And we're seeing this affect the team in different ways. Some people want to go back to work immediately, and other people are needing to take some time to figure out where they stand and how they can operate in this new environment. And it's something that is really important for us to support. We see the team as a bunch of people that we've assembled really carefully, and we want to work together for the long term. We want them to be able to be effective and healthy. And so optimizing for being effective at work this week of all weeks is something that we can put to the side for a bit. I will say that in a general sense, right now everyone's working from home, and this is pretty common in these types of situations because people have family. We don't necessarily have the right facilities at work to, for example, bomb shelters. And luckily, the COVID years have made us really adept at working from home. So this is not a big deal. People often work from home. It's totally fine, and that's the steady state occurrence. It's just not always that everyone is from home. We've talked amongst our team leads and leaders in the organization and tried to encourage them to reach out more to their team members to figure out how to still have some sense of cohesion in the team, so people can also feel comfortable reaching out when they need help.

(Joel Beasley at 00:03:40) Yes. You mentioned that people have different responses to trauma, right?

(Gahl at 00:03:47) Yep.

(Joel Beasley at 00:03:48) And so you, being the leader and coaching those that have others in their charge—you just deal with it on a case-by-case basis? Do what's right for the person at that moment in time?

(Gahl at 00:04:01) We've tried—there are certain things that I've been trying to do broadly and certain things that I think it makes more sense to do personally. So I'm trying to make sure that everyone in the team is feeling safe and feeling that they have the place to reach out and the people that will listen to them. This is on a broad sense, so this means touching base with them, making sure that their managers are also available for them. And in a more personal sense, for the people that are struggling a little bit more, reaching out more often, making sure that they're fine. Unfortunately, we can't really meet them physically. Many of them have been scattered across the country to stay with family and so forth. But we still try to keep a sense of connection.

(Joel Beasley at 00:04:47) Yeah, you would definitely want to be with family, right?

(Gahl at 00:04:50) Yeah. Yeah.

(Joel Beasley at 00:04:51) Yeah. Well, I want to talk about your company too, because this interview was scheduled weeks ago before any of this, and I definitely just wanted to start by touching base, let you know there's nothing but love over here on our side. So let's talk about your company. How do I say it? Is it Raftt? Is that how I say it?

(Gahl at 00:05:08) It is Raftt. Yeah.

(Joel Beasley at 00:05:09) Okay. And why Raftt?

(Gahl at 00:05:10) That's a really good question, and it's one we get quite a bit. The idea was in the whole DevOps landscape, you have a bunch of nautical-themed words—things like Kubernetes and Helm and a bunch of different words, containers. And then Raftt was the idea of a really nimble, really quick-to-set-up ship that would be able to help developers do what they need to do really quickly, without slowing them down or bogging them down or being too heavy or difficult to steer. That was the idea, and we stuck with it. We added the T for disambiguation. It turns out that RAFT is a name for a protocol, a consensus protocol for distributed systems, which is really cool, but also unrelated to what we're doing. So we needed something.

(Joel Beasley at 00:05:57) So how do you spell it?

(Gahl at 00:05:58) So it's R-A-F-T-T. Yeah. I see you have our logo right behind you.

(Joel Beasley at 00:06:05) Yeah. So I just want people listening to know they can go to raftt.io, and it's primarily—your one sentence on it is it's a Kubernetes—

(Gahl at 00:06:14) Yeah. Our goal is to give developers fast iterations and rich feedback on top of Kubernetes-based environments. What we're seeing over and over again with all kinds of companies that we're talking to is that as companies have made the shift to using modern DevOps deployment strategies—things like Kubernetes or even serverless or all kinds of different solutions—developers got left behind, and they're experiencing more and more pains. And as a result, their iteration speed is going sky high, and they're not able to be effective, and we want to solve that.

(Joel Beasley at 00:06:46) Do you have firsthand experience with this?

(Gahl at 00:06:48) Yes, of course. So I can give a bit of background. My technical journey started at 8200, after which I joined a startup called Onavo. And at Onavo, we were a very slim startup. We were very small, and pretty quickly we were acquired by Facebook. And one of the things that I saw at Facebook was that the team I joined a bit afterwards was scaling really fast. It started from around 10 or 15 people when I joined, and after about two years it was over 100 people supporting another couple hundred around the company. And so this was a really intense period of growth for the team and for the platform that it was building. And the platform was a decade old. It was a monolithic Java app, very complex, and there was a lot of pain around people that were ramping up onto it and trying to work effectively. We were working locally. There were all kinds of problems because of that. And this was even a light case, because we were working locally and it was just a single monolithic service, so it was actually relatively simple. And the pains, once people are working in microservices-based environments with multiple dependencies or cloud-based dependencies—the local setup becomes much more complex and the pain goes much higher. And this is where we come in, and this is the pain that we wanted to solve. And as time has gone on, we see more and more companies adopting these types of architectures, not always necessarily thinking through the effects they will have on the entire engineering organization. Does that make sense?

(Joel Beasley at 00:08:25) Yeah. Yeah. My background is software engineering, so that completely makes sense. When you're talking to the business side of things, though, the people that are either product managers or VPs or engineering leaders—when they're thinking about the business value side of things, they're like, "Okay, it's great. It makes tech better, but how does it affect us in a financial sense, or how does it affect us in a personnel sense? What does it do for our core business?"

(Gahl at 00:08:52) So what we're saying is that in companies of a certain size, there's starting to be a realization that we really need to invest in the way that developers are working with our infrastructure. Because the pain is hitting us in many different areas. It's everything from business value not being able to be delivered on time, to attrition in developers, to a general slowdown and a lack of innovation. And these things are really painful for the organization as a whole, and as a result, companies are adopting modern platform engineering practices. And we're seeing this pop up all over the place, where the DevOps organization, which is mostly in charge of production and had an additional role—which it never did really well—to support developers, is splitting off and spawning a platform engineering team. And this is a really interesting evolution because it allows a small team of individuals to support a really, really big engineering team. And they're in charge of making sure that the business is able to continue operating effectively by making sure that developers are able to continue working effectively. And a lot of the goals that they have are making sure the developers are able to be effective. And these teams are really interested in solutions that are able to alleviate the pains felt by developers, both because developers are bugging them constantly because things aren't working well, but also because they're trying to figure out what's the best way that they have to support their developers. There's a bunch of things that they can and need to do, but one of the bigger parts of them is making sure they can iterate and get feedback really well from the changes that they make. And so things like owning CI/CD, owning ephemeral environments, owning the way developers iterate on top of these environments is exactly in the—

(Joel Beasley at 00:10:48) So the DevOps team is responsible for this. So those are the people you want to reach. You say, "Hey, DevOps teams, Raftt exists. It'll make your life easier. Come check it out."

(Gahl at 00:10:59) So it's somewhere between the DevOps and the platform teams. It depends on how mature the company is in their life cycle, I guess. The larger the company is and the more modern it is, it'll usually have some kind of platform team. And we're even seeing older, slower-moving companies such as banks, healthcare, financial institutions also start going in this route along with adopting modern DevOps. And that means that they become prime targets for Raftt because they can gain a lot from adopting these technologies to accelerate the development.

(Joel Beasley at 00:11:34) Okay. So if technical leaders are listening, they would go talk with their DevOps team about this.

(Gahl at 00:11:43) Yes. You could talk with your—yeah, either the head of DevOps or the relevant persona. And if you're using Kubernetes-based environments, or in general if you're using modern DevOps deployments, then this should be something that is on your radar, and you would want to evaluate to see if it could save a lot of time for your developers.

(Joel Beasley at 00:12:01) So how are they making that a number in the sense that—you know, I asked how this would express itself: attrition, developers, lack of innovation, slow iteration times. What's the number that they can actually pull that's a hard number to say, "Okay, this is the hard number that we have, and we know that Raftt is going to cut that number in half or by 25%"? What's that number that you guys are looking at?

(Gahl at 00:12:29) So the number that we're looking at most is how long it takes for a developer to get feedback from a full environment. And usually this means going through some kind of CI/CD process. So the developer will write some code, commit, push, run through CI/CD, and actually look in the environment to see whether it worked or did not work. And this whole cycle can easily take tens of minutes. And this is something that developers need to do multiple times per day just to verify that their code is working well. What Raftt does at its very core is cut that down from tens of minutes to one second, because it short-circuits the process and it makes it pretty much instant. So developers can make, instead of just five or 10 iterations per day, they can make literally hundreds or even thousands of iterations. We're not going for a 10% marginal increase in developer productivity, because this is something that organizations are kind of wary of. We're targeting orders of magnitude more iterations, which means orders of magnitude more output potential for developers.

(Joel Beasley at 00:13:33) How are you doing that?

(Gahl at 00:13:34) So we're getting a bit technical here, which I'm fine with, of course. What we're doing is we're taking existing environments that are deployed on Kubernetes, and we're allowing the developers to connect directly to them and make changes directly on top of these production images in the production-like environments, such that the changes that they make, that they write locally, take effect immediately. They can write code in their favorite IDE. It syncs immediately to the environment. There's hot reloading and live sync of the code. And they can even debug out of the box from their IDE that process that's running on the remote environment. And this means that they get the ideal and rich feedback they're used to, but also in a full environment that matches their production experience. So you get high quality with a really fast iteration speed.

(Joel Beasley at 00:14:27) That's crazy. And then so you're doing that while working, so you're not actually going through the CI/CD pipeline?

(Gahl at 00:14:34) So we're not going through the CI/CD pipeline for the fast iterations for development. We're not touching the process that code needs to go through to reach production, though. That stays the same. So to reach production, eventually you will be running through CI/CD, after which it will be deployed to some kind of staging usually, and eventually it will reach production. And that's a process that we don't want to touch. It's very important for a lot of companies—the tests and the compliance security verifications they have in those stages. But those stages aren't important for just iterating in the developer stage of the lifecycle. So while I'm still iterating on my code, I'm still trying to figure out what's wrong, I don't want to have to go through all of these stages.

(Joel Beasley at 00:15:18) That's interesting. So normally, you're writing all of your code—it's like back in the day, right? You're writing all of your code and then you deploy it to the CircleCI pipeline.

(Joel Beasley at 00:15:32) That pipeline is actually booting up an instance of the environment and running the test in it. And then if that comes back successful, it will deploy or whatever your setup is. But you're saying rather than having to wait till you run through the CI/CD pipeline to actually get feedback from a live production environment or the simulated production environment that you have for your CI/CD, you can simply have that in real time while you're working, just like it's on localhost, just like I'm running a copy locally.

(Nagal at 00:16:02) So it doesn't need to be local. It can be remote. It can be wherever it's most convenient for you. We're seeing a lot of companies adopt open source technologies such as Argo CD or Flux. There's a whole bunch of both solutions and companies that build products that allow really efficient and easy to set up orchestration. And that means that it doesn't matter what kind of orchestration solution the company has.

(Nagal at 00:16:26) We can connect to it and give the same exact experience. This also provides kind of a consistent orchestration and framework they're using, which can abstract away a lot of the complexity, such that developers don't need to deal with all of these finicky details just to be able to iterate on their application code.

(Joel Beasley at 00:16:48) How do people try it?

(Nagal at 00:16:49) You can go to our website. You can try it for free. We're happy to help. So feel free to reach out, even if it's not even a commercial prospect. We have extensive documentation, and we really look for people to try the product and figure out if it works for their use case.

(Joel Beasley at 00:17:09) That's awesome. You remove all the barriers. You let the engineers just go use it to see how it works for them.

(Nagal at 00:17:16) Yes.

(Joel Beasley at 00:17:17) Where does the free model stop and the paid model start?

(Nagal at 00:17:22) So our pricing is generally license based, per seat. We see the companies that we're working with that are using our product are pretty much using it daily. This becomes a core part of the developer workflow. But at that point, it becomes very obvious that a seat-based pricing makes the most sense. I want to encourage people to use the product as much as they can, and I don't want to cause weird incentive schemes due to an odd pricing or price per minute or things like that.

(Nagal at 00:17:47) So generally, that's where it kicks in at a certain number of seats. Generally, at companies after three seats, we'll reach out and see if it's relevant for them to become a commercial prospect. We're happy—we've been working with companies of all kinds of sizes, so we're happy to work with everyone.

(Joel Beasley at 00:18:06) How long ago did you start, or when did you get your first customer?

(Nagal at 00:18:10) Okay. So we're working with design partners from really early in the company. We've been running for about two and a half years. We've been working with design partners from about a couple months after we started, so that was really early. Our first paying customer was about a year and a bit ago. So almost a year and a little bit after we started, I guess.

(Joel Beasley at 00:18:31) Nice. That's exciting.

(Nagal at 00:18:32) That was exciting. It was a really big moment. I mean, it gets less exciting the more customers you have, right? But it is actually still exciting.

(Joel Beasley at 00:18:42) It's exciting to get customers, and then you're like, oh, this is work. But, yeah, it's still exciting.

(Nagal at 00:18:48) It's still exciting. It is. Yeah.

(Joel Beasley at 00:18:50) Yeah. Okay. So for you guys, did you bootstrap the whole thing, raise capital, use some of—you worked at Facebook—did you use some of your Facebook savings there to help start it?

(Nagal at 00:19:01) No. No. We raised capital right as we started. We raised from Aleph, which is a large Israeli VC, as well as Cardamom Capital. And that was early 2021.

(Joel Beasley at 00:19:14) Nice. Nice. That is exciting.

(Nagal at 00:19:16) It was.

(Joel Beasley at 00:19:17) I'm pumped up. I like what you're doing because I haven't been writing software for about three years or so. But before that, it was north of 15 years—

(Nagal at 00:19:26) Okay.

(Joel Beasley at 00:19:26) Right, where I was doing it all day, every day. And I remember towards the end there, the last couple years when CI/CD pipelines became popular, like Jenkins and CircleCI and all of those. That's what I'd do. I'd hit run, and I would go walk away. I would wait for the results to come back from that situation. Then they started getting some efficiencies, but the idea that you could take it down from 10—and then as your environment grew and got more complex, it just even took longer to build.

(Nagal at 00:19:57) Mhmm.

(Joel Beasley at 00:19:57) So the idea that you guys have managed to take this down from tens of minutes to like seconds is phenomenal.

(Nagal at 00:20:03) Yeah. I totally agree. This is what we do. But we're seeing this everywhere. And to be honest, the big problem with long iterations isn't unique to developers as well. It's kind of common all over the place, and it's actually really easy to focus on developers when talking about this, but I'm seeing this in all the functions. Like, we were working on marketing as well. And in marketing, the cycles are insanely long. Like, you do some kind of paid campaigns and it usually takes you days at minimum to figure out what's going on, whether it's working or not. And only then can you make small corrections, and then it still takes again, days.

(Nagal at 00:20:39) And I see huge potential—and it's not something we're going to do—but I see huge potential for companies that what they're doing is removing barriers to really fast iterations across the board. And it really is everywhere. I mean, once you're able to crack that, I think it makes every single function in the company much, much more efficient. So even things like Calendly, what they were doing for booking meetings, right? They identified a cycle that was taking, I don't know, several email messages and overall some tens of minutes, and bringing that down to a one minute book through a web portal was pretty cool. It's kind of a really interesting model.

(Joel Beasley at 00:21:21) The moment I saw them, they were real early when I started as a customer with them. But I became a huge fan of their product because it prevented the back and forth, like, hey, can you meet at this time or this? And it was just, hey, send me your Calendly link. If not, here's mine. Because you want to make it easy on them.

(Joel Beasley at 00:21:40) So that's actually a Calendly trick I found is I'll say, you know, hey, do you have a Calendly link? Go ahead and send it over to me. I'll book a time that works for you. If not, here's my Calendly. Because it's kind of rude if you're just like, here's my Calendly link. Go do some work and book some time. Yeah. And especially when you want something from them.

(Nagal at 00:21:55) Yeah. Exactly. I've noticed this kind of balance as well. You have to do something that's kind of in between. It's not yet fully socially acceptable to just send people links. I feel like we're getting there, though.

(Joel Beasley at 00:22:06) We are. The next generation will help us out a lot.

(Nagal at 00:22:08) I hope so. Yeah.

(Joel Beasley at 00:22:10) I'm excited for people reducing cycle times in marketing. You know, when we do marketing, I'm sure you do marketing at your company, at least some retargeting to stay in front of people. We do pre-marketing to our email list, right? So they'll see our ads before they get our emails. We do little creative things here and there. But every time we do something, the conversation with me and Juan goes like this: We'll do it. We'll wait a week. We'll make two adjustments. After the week, we'll wait two more weeks. We'll make a couple more adjustments. We'll get enough data, and we'll figure out if this is viable. And now we're a month down the road, you know, trying to figure out this specific test. Meanwhile, the backlog of things I want to test, there's like in the tens, right? There's like 10 different ideas I want to test, but it just takes so long.

(Nagal at 00:22:54) Yeah. Yeah. I totally agree. Again, marketing is actually like a really easy point because it's one of the areas where you never have full support internally. You're always—I've always found myself eventually working on marketing as well because there's so much to do and so many possibilities, and it's really difficult to navigate what the right thing is. And so people run a lot of tests, but they often don't have the throughput to run quite as many as they would like and end up running a very few, which don't cover the entire surface and kind of do random things as a result.

(Joel Beasley at 00:23:27) And that's why you'll see the direction in marketing is headed towards—if you've seen Facebook's latest update, I don't know how much you're in there—but now you'll have different, like, oh, here's five different headlines and here's five different images. And it'll say, don't worry, we'll figure it out for you. And I say, well, yeah, our values are misaligned there. Our outcomes are misaligned. My outcome is to get the highest maximum return for the lowest cost, and your outcome is how much money can you take from us. Just being fair.

(Nagal at 00:23:58) That's true. Though I think in the long term, that they're more aligned than it would seem. I mean, if it doesn't work, you would very quickly stop advertising on Facebook, right? And then they get zero of your money.

(Joel Beasley at 00:24:11) Yeah. No. No. A hundred percent. I don't—I'm not a Facebook conspiracy theorist, but—

(Nagal at 00:24:17) It's fine. It's fine. But it—

(Joel Beasley at 00:24:20) Yeah. No. It's one of those things where it's like, you have to think deeply upon it to get some real insight, but I get what you're saying. And then there's also the competitive aspect, and I've actually seen some of the Facebook performance improve in the past year.

(Nagal at 00:24:37) Okay.

(Joel Beasley at 00:24:38) So that's a positive. As far as cost and things go, I have seen an improvement, and I follow a couple—there's one mentor to the—he's like a mentor to our company, and he owns a fairly large digital marketing agency in Florida. And so I follow and talk with him every couple months about what's going on because he's managing ad spend for like hundreds of millions of dollars, right? So they see everything just on a whole different scale.

(Nagal at 00:25:05) Sure.

(Joel Beasley at 00:25:06) But, yeah, they definitely were improving this past year. But let's talk more about Raftt and you guys. Now, here's a good question for you. Do most companies know their cycle time?

(Nagal at 00:25:18) I think this is something that they're pretty aware of. Yeah. I mean, it depends who we're reaching in the company. The developers are definitely aware of this, and so are their DevOps. The interesting thing is that this is actually costing them money as well, right? Because CI/CD is really expensive. And this means that if developers are needing to go through CI/CD every time they're making an iteration, then they're ending up spending a lot more than they would have to if they could just not do that. And so they're not only aware of how long it takes, they're also aware of how much it's costing them. And that's a pretty cool realization.

(Nagal at 00:25:52) A big part of reducing iteration speed is making people able to make mistakes. And this is also something we found out recently, or not recently, but while working on Raftt. And kind of we started from an angle of, I want to make developers as effective as possible. But what we quickly realized is that one of the things that's limiting them and limiting their growth is that they're too afraid to do things wrong. And the reason that they're afraid is because it takes so long to correct a mistake. When you make a mistake and then it takes me an hour and I kind of realize that if I make a mistake, it's going to take me a couple more days. Or even, or even in the worst case, I kind of mess something up, and now my environment is crap, and I have to fix it, and that's going to take a long time. And this really, really slows people down, and it stops them from making changes that could be dangerous. And one of the great things that's coming about in this huge revolution of the DevOps and platform teams is that things are becoming more standardized, and it's becoming harder to make mistakes. But if the iteration speed doesn't kind of go along with it, then we're still stuck in a place where people are trying to optimize what they're doing to make sure that things don't go wrong.

(Nagal at 00:27:05) And so we're seeing that developers feel much more free to explore and experiment if they can do that really quickly because the pain of getting things wrong goes down to zero.

(Joel Beasley at 00:27:19) ChatGPT feels no pain.

(Nagal at 00:27:22) Oh, we can talk about Gen AI too. I feel like—

(Joel Beasley at 00:27:26) How long until— Go ahead.

(Nagal at 00:27:28) I feel like the— You know, there's the Gartner hype cycle for a bunch of things, right? And I feel like Gen AI is right past the peak of the hype cycle right now. It was super compressed, right? Like, it started here about like six months, eight months ago, I don't know. But it went up really fast. Yeah. And now it's going down. It's going to go down really fast as well. I don't know. I don't know if you've had the chance to play with it for writing or any kind of really creative stuff. To be, like, from my perspective, it's kind of fallen flat no matter what I've done with it. What? You can— You read the text, and it feels like AI wrote it, right? You can always tell. And kind of, it's no matter what's going on. You can read the code, and you see that AI wrote the code too. This is all—

(Joel Beasley at 00:28:12) You can— Well, if you can tell, then you need to map your algorithm. What is their success rate? At OpenAI, I think their success rate's like 20% of them being able to detect if it was written by AI or not.

(Nagal at 00:28:26) Yeah.

(Joel Beasley at 00:28:26) I think it's just— Here's— Here's where I believe— And because I got caught up in this too. I was like, yeah, that sounds like ChatGPT because it's well written. No. Like, people don't speak that well. No. Like, it's just so well done.

(Nagal at 00:28:45) No. No. It's overly verbose. It doesn't notice that it's repeating itself. There's like really common patterns that people just don't do. You'll have like a paragraph that's really just one sentence, and it's a paragraph. I don't know why it does that. So there's like— And there's ChatGPT has recently introduced hints that you can give it so it improves its writing or sounds more like you, right? And that helps a bit. I played with it quite a bit too. And to be honest, we actually did some work internally. We thought, hey, this is really cool stuff. Maybe we can try to find a good use case in the DevOps landscape for something that's Gen AI driven, right? Maybe it can give you insights while you're developing or help you debug things that are going on in Kubernetes. And we dug into this for quite a bit. And to be honest, this did not go nearly as well as we hoped. It was— Things like it would just— It would get things wrong, and it would kind of fall over itself in trying to help you, but kind of end up repeating itself and not doing the right thing. And it was really hard to get it to be consistent because it was often going in random directions that were sometimes okay and sometimes really problematic. And it was kind of— It's a limit.

(Nagal at 00:29:53) So I think— I really believe in this technology in the long term. I think we're going to— I think we're going to fall down the hype cycle. Anyway, that's my take on this, and before we analyze—

(Joel Beasley at 00:30:07) I agree with you on falling down the hype cycle because that's just how it goes. But I would say, and I've talked to people with a wide range of experiences, I'd say you're on the side that hasn't had an awesome experience with it. I am on the side that there are very specific things that we use it for and we use it in very specific ways. We have a Slack channel that's just about prompting GPT for very specific things that we do at the business.

(Joel Beasley at 00:30:40) And if you get those prompts right, it can—in my business, and I'm not saying in your business, I'm not saying in a DevOps business at all, right, because we're a media company now—but in my business, using those things, like, for example, we create the subjects headlines for a clip, right? We can take the top 15 most viral clips in our sub-niche, put them in, say, "Hey, these are the top 15 titles. Here's a quick summary of what the conversation was about." Or we can take the transcripts, have them summarize it down to a paragraph, put that paragraph in, and then have it come up with 10 different headlines, and we can pick which one we like the best. Now, there's only so much creative energy somebody has in a day.

(Joel Beasley at 00:31:25) So when you're down on your creative energy or you're all spent and you need to now do this task of creating these headlines for these 10 or 15 clips, you can do it without hardly thinking. You're just kind of sitting back and getting these custom suggestions. But we did all the legwork of figuring out how to prompt it, how to prompt it from zero. That's another thing people get wrong a lot because they'll be prompting, and then they'll go along and take all these notes because they're in the singular conversation. And then they go back and try that in a blank conversation. It doesn't come anywhere close to it.

(Joel Beasley at 00:31:57) But, yeah, we've learned a handful of very specific ways to use it for our business, none of them which is programming. But have you tried Copilot at all?

(Nagal at 00:32:07) I have. I have tried Copilot. I was actually in the beta a year ago when it didn't cost anything at the time. And to be completely honest, my experience there was a bit the same. I felt that it was missing context often. And if I was doing something that's really, really templaty or kind of grunt work, then it would do a really good job, right? Call an API and fetch some stuff, parse the JSON, things like that. But once you do something that's complex and has a lot of interconnect, it's really integrated to your existing system, then it just doesn't have that context. It isn't able to have that context, and then it doesn't work as well as you would want.

(Nagal at 00:32:45) Or it ends up using things that are not in the idiom that you're using in your codebase. I haven't tried it in the past couple of months, though, and people have told me that it's improved.

(Joel Beasley at 00:32:55) No, I haven't. Just so you know, I haven't been playing with it in the flow of work. That's why I asked smart people like you who do use it from time to time because I just see the videos, right? And I'm like, well, that's actually pretty cool.

(Nagal at 00:33:10) It's kind of similar, though, to the GPT Engineer type tools or the GPT Researcher, and those are really—they're brilliant. They're really, really awesome. But what they're always doing is starting from zero. And it's much, much easier to start from zero because then you can pretty much write whatever you want and, generally, there is no additional context to have. So it's much easier to write any code from zero, right, than to adapt new code to a really complex system. That's kind of where I feel like there's some kind of delta. To be honest, though, we do use some of these AI tools internally, so it's not like I'm kind of a purist here.

(Joel Beasley at 00:33:45) You're a purist. We've said no generative AI. It's a horrible thing. It all is not on team AI. No. I'm kidding with you. A hundred percent. I'm excited to see when they get to the point—well, I'll back up. I was talking to some of my other engineering friends that are actively engineering, and I was like, well, why don't you just dump your whole codebase into GPT? Like, get your own instance of it, first of all, not the hosted OpenAI version. They say they don't keep your stuff in private mode. I don't trust them. I just don't. And get your own local model and then train it on your codebase and explain to it and give time and put customer stories into it. Like, really train the thing up on what you're trying to do and then ask it for some help.

(Joel Beasley at 00:34:26) I think that in the future, we're headed more in that direction because you're right. It needs all of this back context to even begin to start to help a little bit.

(Nagal at 00:34:41) Yeah, I totally agree. And I'm sure that there are companies that are starting up now or in the past year or so that will be doing exactly that. And there are also alternative models that at least purport to have much larger context sizes, so you actually can give it more context. Though, from what I've read, it's kind of a mix of whether it actually behaves like it has additional context or is too weighted to recency. I don't know. I'm not trying to be an AI expert here. So it is super interesting, and I'm following along really closely, but I'm currently a skeptic.

(Joel Beasley at 00:35:12) Yeah. Well, I'm always a little skeptical at everything. I was looking up how to boot up my own version of it, and this was four or five months ago. It said it needed a minimum of—at the time, whatever project I was looking at, I looked at a couple. The smallest one I could find needed a minimum of 48 gigs of RAM for me to boot up the model.

(Nagal at 00:35:33) Oh.

(Joel Beasley at 00:35:33) And I was like—

(Nagal at 00:35:34) That's so wild.

(Joel Beasley at 00:35:35) I know.

(Nagal at 00:35:36) Yeah. I think there are projects that are running LLAMA, the smaller instances of LLAMA, on a regular computer with relative—or even on a video card—with relatively low amounts of RAM. I think there's adaptations of it that work pretty well. We've played around with them as well, kind of where we want to give some kind of on-prem solution that would give, I don't know, DevOps a device and we were looking at something like this. But, again, these models are generally less intelligent than the OpenAI models because they're so much smaller.

(Joel Beasley at 00:36:09) So LLAMA, L-L-A-M-A, that's—I found Hugging Face has a bunch of information about it. So that's a model that's lighter weight that I could actually boot up locally and play with.

(Nagal at 00:36:19) Yeah. You should be—I mean, you still need a relatively recent computer probably for it to work at an effective speed. But, yes, you should be able to boot it to work with it locally.

(Joel Beasley at 00:36:29) Are you training the actual model from nothing, or are you using some pre-trained model and then you're just processing it?

(Nagal at 00:36:38) So it really depends on what you want to do. It's a regular LLM. You can use it for regular prompt-based tasks. You can also fine-tune it, and I'm sure you can do some more extensive training. But, again, this is kind of where my AI knowledge peaks. So that's all I know.

(Joel Beasley at 00:36:52) Me too. Yeah. No. That's interesting. Thank you for making me aware of that because now I'm going to go play with this. Because I really wanted to understand, like, there are certain things I want to do, but I just would only trust it if it was a local model.

(Nagal at 00:37:10) Yeah. I think a lot of companies are in the same place. It's really dangerous to send all your data somewhere.

(Joel Beasley at 00:37:16) Yeah. Because one of the things I thought would be interesting was if I dump all my QuickBooks data into it and then asked it to act like—and then dumped specific financial adviser type books that I trust their styles into it—and then ask them to think like this person in order to understand my business better or look for areas of improvements. But, I mean, I'm not going to go screwing around with that and throwing my QuickBooks business data into any online service, you know.

(Nagal at 00:37:45) And that makes sense. Yeah. Yeah. Again, I'm sure that this is going to be a product pretty soon, like, your automated financial adviser type thing. And eventually, people will be trusting online services to do this stuff as well. It'll take some more time, and maybe we'll find out there's additional privacy models we haven't yet explored. That's also something that can be pretty interesting. And it's also actually relevant for my use case as well. I mean, a lot of companies are really, really hesitant to run their code on someone else's servers, right?

(Nagal at 00:38:13) And they're kind of, "Okay, we'll trust the public cloud providers because Amazon, Google, and Azure are too big to fail, and they're probably doing things okay." But not even so, not all companies, right? Banks, financial institutions, and so forth are pretty wary. But even still, they won't run it in someone else's account in those cases, even though there's not necessarily much of a difference. Interesting use case for a more private computing standard or method.

(Joel Beasley at 00:38:39) For people that do have those standards, like banks, can they still use your product? Is there still a way for them to use it?

(Nagal at 00:38:45) Oh, of course. Yeah. Raftt can be deployed on-premise, inside your account. It can be deployed everywhere pretty much, because we're depending on Kubernetes as kind of a standard infrastructure layer, then it can be deployed anywhere that there's a Kubernetes cluster. There's no real requirements beyond that.

(Joel Beasley at 00:39:03) I love it. What else—so we know people can go and try it for free and get a couple seats and see what it's like in their environments and with their workflows, and it's R-A-F-T-T dot I-O.

(Nagal at 00:39:17) That's right.

(Joel Beasley at 00:39:17) What else did we not cover? What else do we need to get out there to the world other than Israel is awesome?

(Nagal at 00:39:23) I guess one of the things that I like talking about and we haven't had a chance to touch on yet is the importance of tech debt. And this kind of goes two ways. One of the things that I've seen over and over again is that engineering organizations really quickly accumulate a lot of tech debt, and it becomes something that organizations like to manage quite a bit. There'll be extensive discussions on the kinds of different tech debt there are and whether what kinds we should be working on, how much time should we allocate to tech debt. And the first realization here that organizations kind of should process is that tech debt is a good thing, right?

(Nagal at 00:39:59) It means you're not going as slow as you could be. It means you're making trade-offs where they make sense. So you're prioritizing tasks effectively. And that's a really positive thing from my perspective. And the second part is that there are different kinds of tech debt. And most important, the most important kind is the kind that can make you go faster. And this really ties back to everything that we're doing. And there is a kind of tech debt—and I've noticed it internally, even at Raftt—where there are tasks that we know we should be doing, and eventually we do them. They end up taking, I don't know, an hour, and they cut down engineering time by 10 minutes every day. And there's kind of trade-offs like these that we obviously were not making the right decision over a long period of time, because these types of tasks are really effective at making my engineering org go faster.

(Nagal at 00:40:52) And because they're kind of bucketed all together in the same, kind of under the same name of tech debt, which is amorphous, it's difficult to value, and it's not clear to me when I should be doing these tasks before other tasks that maybe have some kind of business value, I don't know when to actually assign them to people and when to make them—when to work on them. And it's really simple to get a framework that helps you evaluate this. And the framework is pretty much for each task, figure out how much time it could save me potentially per day across the org, and first, how long it would take. Now, there's an XKCD that perhaps you're aware of that kind of a table between how long should I be investing in a task versus how much time is this going to save me per day?

(Nagal at 00:41:43) And it's kind of a convenient way to think about it. And for many things, it becomes really obvious whether this is something that we will never be doing ever, because this has no value, or whether it's something that we ought to do immediately. And we should be doing this before anything else because within a week it's paid for itself. And of course I should be investing that time. And this is something that I've seen engineers struggle with a lot, both as developers themselves and as engineering leaders. And it becomes kind of a point of contention. Why aren't we doing these things? Or how much time should we be investing? Like, the product organization will kind of be annoyed that engineering wants to invest so much time in tech debt when there's a really simple, a really concise framework with which everyone can agree what is important and what is not, because everyone is really aligned at a certain point. Everyone wants things to go faster, and this kind of makes everyone talk in the same language. And this kind of ties back again to where Raftt meets organizations, because one of the hard things is reaching developers and when they really want to use a tool like Raftt, because leaders or product owners can view this as kind of yet another thing that the engineering wants.

(Nagal at 00:42:56) And the right way to frame it is this will make your engineering team able to operate at a five times faster speed. We iterate more, and you'll get better feedback. You'll have higher quality code. And all this really ties directly into the business value that you want to reach from your R&D organization. And it's—the goal should be to create a common language between all the relevant stakeholders to make adopting such a project or working on tech debt or really anything else a much easier decision.

(Nagal at 00:43:27) And that's kind of what we're striving for from a messaging perspective with Raftt. And it's something that I look around and I try to do in general. It's not always with our engineering organizations, right? It's in general, with every kind of partner, we're looking for the way to speak to them in a way that makes sense to them, and we can talk about the same thing and agree about the goals. And almost always it's possible.

(Joel Beasley at 00:43:50) No, that's great. Have you written a blog post or anything on these thoughts here, or no?

(Nagal at 00:43:54) I've written a blog post on several different thoughts, but not necessarily about this one specifically. Do you think it's worth it?

(Joel Beasley at 00:44:01) I do. Yeah. Okay. I would share it. If you write it, I'll share it.

(Nagal at 00:44:04) That sounds good. Then I can write it.

(Joel Beasley at 00:44:07) Yeah. Just ask ChatGPT. Just a moment. Can you have a meeting with Copilot and write up an article on tech debt?

(Nagal at 00:44:15) Yeah. I'll do that.

(Joel Beasley at 00:44:16) Well, Gahl, man, we did it. We made a podcast. How do you feel?

(Nagal at 00:44:20) That was amazing. Yeah. Thank you. It was a really good experience too.

(Joel Beasley at 00:44:23) Thank you so much for listening. And if you found this episode useful, please share it with a friend or colleague who you think would get value from it. And if you have topics that you'd like to hear discussed on the podcast, either add me on LinkedIn or send me an email, [email protected]. Every time I get an email or LinkedIn message, it absolutely makes my day and inspires me to keep going.