Jonathan Ross - CEO & Founder at Groq

(Joel Beasley at 00:00:00) Hello, my friends. Today we are talking to Jonathan, the CEO and founder of Groq, and we discuss how Groq's processors will make compute functionally free for everyone, knowing which culture items to focus on as an organization scales, and why hiring engineers with high empathy is the path to innovation. All of this right here, right now on the Modern CTO Podcast. Here we go.

(Joel Beasley at 00:00:27) This is the Modern CTO Podcast. What did you do at Google X? I'm curious. We were geeking out, so we're just a bunch of nerds here, right? It's probably stuff you can't really talk about, but in our production prep meeting, we saw that you were a rapid evaluator, is what the title was, at Google X working on moonshots. What is that?

(Jonathan Ross at 00:00:55) All right. So a little while ago, Google reorged, and Larry Page was pretty big on puns. So if you've ever wondered why it was PageRank, it's not just ranking web pages. It's Larry PageRank, right? And so he had an opportunity for another pun. He apparently met with Warren Buffett, got the idea that he wanted to have a large conglomerate, and so he renamed Google into Alphabet as in a bet on alpha, like financial alpha. So the rapid eval group, we were the group of about 12 people whose job it was to come up with new alphabets.

(Joel Beasley at 00:01:32) And how did you do?

(Jonathan Ross at 00:01:34) Pretty well. There was a couple of really great people that we got to work with. I think a couple of them made it into the later stages. I was only there for about a year, but yeah, we did everything from biomechanical engineering. Fortunately, they shot it down, but I had this idea of potentially trying to securitize lawsuits and things like that. So it was actually for social good, but the media view of it would have been terrible, and they were right.

(Joel Beasley at 00:02:06) What are some of the top two or three ones you remember the most?

(Jonathan Ross at 00:02:11) Well, I don't think any of them were canceled, so I don't know that I can talk—well, except for that one. I don't think any of the other ones were canceled. Actually, one of them, the mechanical engineering one, some patents have issued. And so there was actually a way to do solid state clutches. And what I mean by that is there's ferromagnetic fluid clutches where you can, but you have to apply power, right? And here, you could stop applying power and it maintained its state, but there were no moving parts. And there was just this guy that I was working with on the team who had experience with ferromagnetic clutches. We had this idea. He literally built it in a weekend. He was awesome. That team was great.

(Joel Beasley at 00:02:53) That sounds really neat. You get to work on such a wide variety of things. So how did you come up with Groq, and what does Groq mean?

(Jonathan Ross at 00:03:02) Ah, okay. So that's two questions. So let's go with what Groq means first. So it comes from a science fiction book, Stranger in a Strange Land. It means to understand something deeply, and that fits our culture. It also fits machine learning, and I happen to have the domain name. And when we were founding the company, I'm a little bit of a domain name hoarder. I had about 100 domain names that I wanted to consider. That was on the top of the list. I shared that one with some people, and it was the first one on my list. I'm like, but I have 99 more. But everyone agreed that was the right one and was the first one on the list. So as for what Groq is and what we do, so we build machine learning accelerators, and machine learning is incredibly expensive to run. And it was actually a limiter at Google. So for example, the Google TPU, right, it was actually started because, well, actually, there wasn't enough compute at Google. Sounds crazy, but they had actually trained these ML models, and they couldn't deploy them in production because they were too expensive. And so as a 20% project, I started building an FPGA acceleration of that, which is a reconfigurable chip. And anyway, so in building that, we ended up solving Google's problem, right? But then we realized the rest of the world still had that problem. So we wanted to solve that for everyone else.

(Joel Beasley at 00:04:43) And when will compute be free for everyone?

(Jonathan Ross at 00:04:47) Yeah. So we like to say that our goal is to make compute free. And we don't mean free as in it doesn't cost any money, but free as in we give you so much compute per dollar that it might as well be free.

(Joel Beasley at 00:05:00) Yeah, I love it. All right, I get what you're saying, right? Because it's like it's going to be free, and I'm sitting there. I'm like, what star is he going to harness the energy from? Like, how is he going to make it free for everybody? Who's paying the power bill, you know?

(Jonathan Ross at 00:05:12) Well, if you look at it, it's relatively free compared to what it costs, say, 10, 20 years ago, right? And when it comes to compute, there's always problems that are unaffordable. They're almost like options. They're not in the money yet. And once you provide enough compute, you can do these things that you couldn't do before, right? Speech recognition was completely unaffordable until ML accelerators were developed, right?

(Joel Beasley at 00:05:39) Yeah. And this is what you think about all day?

(Jonathan Ross at 00:05:44) Pretty much. And the team. It was a great team.

(Joel Beasley at 00:05:47) How are you guys growing? Where are you at right now?

(Jonathan Ross at 00:05:50) So we're at 120 people. And if you'd asked six months ago, we would have been at 60 people. So we're growing very rapidly. And we're focused on—so if you know much about semiconductor companies, they tend to require a lot of people, right? It's very expensive. And so what we've been focused on is how do we build chips with the smallest number of people possible. And there's a reason. It's not just to be cheap. It's if you want to be innovative, it's hard to be innovative if you have too many people, right? Because you have to bring everyone along with you. So we're very focused on what we call talent density. And so a lot of the people here, they span multiple disciplines. In chips, there's about 30 different specialties that you need in order to make a chip. Many of the people here know multiple of those specialties.

(Joel Beasley at 00:06:40) So are you working alongside with Google? Like, they're going off on their TPU thing. You guys are making custom chips. How does that play together?

(Jonathan Ross at 00:06:50) So, you know, at Google, we built TPU. It solved an interesting problem. One of the reasons that I decided to start Groq, or one of the reasons or things that we decided to fix when starting Groq, I should say, is that actually, while the Google TPU solves a bit of the compute problem, it was very difficult to program. And actually, I remember when, you know, the AlphaGo competition was about to happen, that team emailed the TPU team and asked, is your compute as fast as we've heard? And sort of like in Ghostbusters where, you know, I think Ray says, if someone asks you, are you a god, you say yes, right? That's just you always must say yes. Well, if someone asks you is your chip as fast as we've heard, you must say yes, right? So we said yes, and then they dug into it and realized we might be able to help them and they're like, well, we've been advertising this competition we're going to do, you know, and we realized we're going to lose, and we need more compute. So in about a month, the team actually put together this version that ran on TPU, and that's the version that actually played Lee Sedol. So it was on TPU and afterwards they had that play the GPU version and it won 99 out of 100 games. And since Lee Sedol won a game, right, that means they were right. They would have actually lost. So the thing was, while I was doing that, I actually volunteered to help the team that was porting some of the software to the TPU. And even though I was a software engineer, even though I, you know, helped design TPU, even though I implemented a lot of it, I actually couldn't port that software easily. It was very difficult. And that's when I started to realize that actually, this wasn't just a problem in TPU. A lot of people on the team used to work on GPUs, and it was even harder there. So when we started Groq, we thought, well, what if we start with the compiler first? What if we spend the first six months doing nothing but trying to take models implemented in TensorFlow, compile them down to something that would run on a chip, and then design the architecture to suit that. So a big difference between what we're doing and what Google's doing is we actually focused on software first, but it is very different architecture. It's not like a GPU, FPGA, or CPU.

(Joel Beasley at 00:09:18) When you did this, did you just self-fund, or did you raise capital? How'd you get started?

(Jonathan Ross at 00:09:24) So we raised capital. We raised an initial $10 million seed. That's not enough to build a chip, but that was enough to start working on the software, prove out a lot of what we were doing. Then we raised some subsequent rounds.

(Joel Beasley at 00:09:36) So are you selling the chips yet, by the way?

(Jonathan Ross at 00:09:38) We are. So we've deployed to multiple government labs. We actually have two very large lighthouse customers in very different spaces, actually. One's autonomous vehicle, and one is actually finance. And the uses couldn't be any more different. One of them does inference. One of them does training. One of them cares about latency. One of them cares about throughput. One of them cares about energy, and one of them cares about power, right? They're just complete opposites.

(Joel Beasley at 00:10:08) Sounds like a lot of the quantum computing discussions I was having recently with like the head of Honeywell Quantum Computing and Microsoft and all of these different—there's so many different types of quantum computers, and they have different strengths and weaknesses. Are you getting into quantum computing at all?

(Jonathan Ross at 00:10:23) So we're not in quantum computers, and I actually do think quantum computing has a place. And it's interesting. It solves a different problem than I think most people realize. And there's actually a bunch of different types of interesting computation. I used to have to vet all the stuff that would be shared with Alphabet. Like, do we want to look at this thing or that thing? So I've looked at optical. I've looked at analog. I've looked at quantum. What's interesting about quantum is it's funny that so few people know about this. I'm surprised it's not popular culture at this point. But I think it was in the sixties or seventies, a mathematician actually found—you know, that whole E equals MC squared thing where energy is matter and matter's energy, right? It turns out there's a relationship between information and energy. How so? So it's called Landauer's limit or Landauer's principle. And back, you know, a hundred fifty years ago or so, there was this paradox. And it was called Maxwell's demon. And it was, you've got a chamber filled with gas, and the molecules are bouncing around. You've got a divider in the middle. You've got a little door, and there's—it's called Maxwell's demon because Maxwell proposed it, and there's a little demon that sits next to the door. And whenever a fast-moving particle goes through this way, the demon opens the door, lets it through, and closes it. And a slow-moving particle goes this way, demon opens the door, lets it through. So pretty soon, this side is cold, this side is hot. You just created energy, and that violates thermodynamics. So this mathematician showed that actually, information is also energy. And oddly enough, when you erase information, you actually have to spend energy. So there's no free lunch. And the thing is quantum computing can't escape this. This is a fundamental physical law. It's actually been verified by experiment. So there's a relationship between energy, matter, and information. Now, what quantum computing does do, what's interesting about it, isn't that it allows you to make computations, you know, that you could not have made otherwise. What it does is it actually allows you to sort of condense that computation down in a way, in sort of a very dense area that you would have never been able to achieve before with standard techniques. And so networking is basically free between the points, but you still have to spend all that energy, right? They have to cool it down. It's really expensive.

(Joel Beasley at 00:13:03) It's expensive to cool, yeah.

(Jonathan Ross at 00:13:05) Yeah. But that's still costing you energy, right? And the funny thing about Landauer's limit is the amount of energy you pay per bit erased depends on the temperature at which you erase the bit, which is one of the reasons why quantum computing works much better when it's cold.

(Joel Beasley at 00:13:23) Dude, the world is so fascinating. It's—we get so many advancements you can't even keep up with them every day. I mean, I subscribe to a couple different newsletters that try to aggregate like technological advancements on a daily basis. But it's just—it's things that you would hear about once every 10 years are happening 10 times a day now.

(Jonathan Ross at 00:13:43) And a lot of things are improving exponentially. Some things are improving doubly exponentially. So as an example, the performance of chips has been improving exponentially. But also, the ability to connect more chips together has been improving exponentially, which gives you double exponential growth. This is what Google fundamentally did, right?

(Joel Beasley at 00:14:05) Yeah. I'd seen—I think a wafer-size chip came out a few months ago. It was huge. And it was because they could get everything so close together in such a large area. It was just incredibly fast. I don't know if it's in production or if it was just a one-off chip, but I'm sure you saw that. It was huge.

(Jonathan Ross at 00:14:23) I, yeah, I saw that. Actually, that was something we looked at at Google, and it's an interesting idea. And the goal of it is to actually get data to be able to move more easily from, you know, one processor to another, right? Because that's one of the big problems. That's what quantum computing, for example, really solves inherently, right, in a more deep way. But the reason why we didn't go that route was it's sort of like taking Manhattan and making it 50 times larger, right, without adding more streets. Because one of the fundamental limits in chip design today, they're getting, you know, denser and denser transistors, but they're struggling to add more wires, right? So when you put more on a single wafer, you're stuck in two dimensions. And it's literally just like taking Manhattan, making it 50 times larger. It takes forever to get to the center. It's hard to get things in and out. And so what we've done is we actually developed our own interconnect, which allows us to move more data, lower latency, and scale up much larger.

(Joel Beasley at 00:15:29) When you take these chips, do you actually like ship them to your customers, or are they just buying data warehouse space?

(Jonathan Ross at 00:15:37) We do. We actually, we've got pallets of boxes, and we ship them on pallets. It's actually nice to have a physical thing you're sending out the door. And, you know, we always send a picture to our team when we ship one, and they all get excited. And it's pretty awesome to have a physical thing you're selling.

(Joel Beasley at 00:15:56) How did you learn chip design? When did you get into that?

(Jonathan Ross at 00:16:00) So that's an interesting story. So I actually learned chip design from an online Flash video game. I was in an undergrad program, and I needed a job. And so I was interviewing at a finance company, which, it was pretty small. It was actually smaller than Groq is today.

(Jonathan Ross at 00:16:22) But the CTO at that company was asking me about how to improve the latency of the systems that they were working on as part of his interview. And I started talking about how you could make a chip. You could probably make an FPGA-based chip. Right? It wouldn't be as hard.

(Jonathan Ross at 00:16:41) And then I made a comment like, "Oh, but the hard thing is making timing," which I had no idea what that meant. It actually means something in the industry. I meant something completely different because I'd been playing this video game, and those words just made sense. And he's like, "That's exactly right." And he pointed to a shelf, and he had an FPGA in a box, and he's like, "You're hired. I want you to program this FPGA to handle this stuff." And so as a result of that, I actually had to learn how to program FPGAs, and I had never taken a hardware course in my life.

(Jonathan Ross at 00:17:13) Our Chief Legal Officer has more formal hardware development training than I do. And so I just treated it like software.

(Joel Beasley at 00:17:23) That's awesome. That is so cool. I love that. You're a very smart person, my friend, but I'm sure people tell you that all the time.

(Jonathan Ross at 00:17:30) Well, we like to hire people who are smarter than ourselves, so I like to feel dumb. It means we're doing the right thing.

(Joel Beasley at 00:17:37) Me too. Isn't life easier, though, when you do that? At first, it sounds like, oh, that's a good idea, but then when you do it, you're like, oh, I get it now. It makes everything easier.

(Jonathan Ross at 00:17:48) It makes everything easier, but also, you have to learn to go with the flow. Right? Because when you're hiring people who are really, really smart, they're going to come up with things that you would never have thought of that are going to challenge your assumptions, and you have to go with it. And you have to shift from the sort of command and control of "Do this, do that" to framing the problem in a way where they can come back with something that surprises you.

(Joel Beasley at 00:18:18) Yeah. Is that one of the leadership lessons that you've learned going through this founding a company thing?

(Jonathan Ross at 00:18:26) Well, I'll definitely say, I think I've learned all the wrong ways to lead, and I found a couple of the right ways maybe. So, yeah, there's a couple of things. I think one of the big ones is there's often a tension between what I jokingly refer to as federal and state issues, which is, you know, you've got to leave people alone to do their own thing, while you also have to, as a company, set some standards and have some collective goals where you don't just do whatever someone wants. But a good example of this—you can break it into local and global maxima. Right?

(Jonathan Ross at 00:19:05) Your job is to, as a leader of the company, the person setting the culture, determine what things you're going to globally optimize versus the things you're going to allow to be locally optimized. A great example of something that people only in the last couple of decades have figured out you can allow to be locally optimized is dress code. It really doesn't matter what people wear to work. You're going to have a great company either way. Right? So you just let people wear whatever they want. Right?

(Jonathan Ross at 00:19:31) On the other hand, when it comes to, say, hiring, if you let people just hire whoever they want, then you're not going to have a standard. You're not going to actually be able to raise the bar because there is no bar. And so you have to pick the things that you want to globally optimize, and you have to pick the things that you're going to allow to be locally optimized, because anything global takes active energy to do. Right? You only want to pick a couple things.

(Joel Beasley at 00:20:01) Thinking about these types of things, is that what encouraged you to move from CTO to CEO?

(Jonathan Ross at 00:20:09) Yeah. So, actually, I can give the nerdiest explanation of this ever.

(Joel Beasley at 00:20:16) Do it.

(Jonathan Ross at 00:20:16) Okay. All right. Don't indulge me. So as a software engineer, I started off writing in the traditional languages like C, C++. These are imperative languages. You just tell the compiler and, therefore, the chip what to do. Right? It's like command and control.

(Jonathan Ross at 00:20:32) And then you start moving into functional languages where it's more, how do you compose these things that exist and get something better? One of my favorites is Haskell, but you can also do this in other languages. And then you start moving into the more mathematical ones, like automated reasoning ones, like SMT solvers, SAT solvers, optimization things where you just say what the end result should be, and it figures out a program to generate something that satisfies your constraints.

(Jonathan Ross at 00:21:15) And so as you start to grow, as you start to improve as a leader, you start to realize you become more and more abstract in how you deal with things. You still achieve the same things, but you can do it while specifying less. And that allows you to do more things and do more ambitious things. Right? And so the trajectory kind of just went in that same direction where it went, okay, well, what's after automated reasoning? How about people? Right?

(Jonathan Ross at 00:21:31) And people are capable of doing some pretty amazing things. And so it gives you a capability that you wouldn't have if you were just doing things as an individual contributor.

(Joel Beasley at 00:21:46) I like that. I like that. How do you think about growing your team, like your specific team as a leader?

(Jonathan Ross at 00:21:55) Okay. So the first thing you have to ask yourself is, what is it that you're actually trying to accomplish? So we talked a little bit about global optimization versus local. And as a company, we've decided that we're going to be an innovation-first company. And what that means when you're picking something as a cultural item is it's not something you get to compromise on. Right?

(Jonathan Ross at 00:22:19) So if, for example, we discovered that the chip industry was a horrible industry to be innovative in, we might make the most innovative garbage bags. Right? Wherever we can be innovative. Now, fortunately, the chip industry is a great one. But when you're growing your team, this is super critical.

(Jonathan Ross at 00:22:34) So when we started getting to around 60 people, we realized it was getting very difficult to do counterintuitive things because you had to bring a whole bunch of people along with you. So we started looking at the problem, and this is where that global optimization versus local optimization came from. And whenever we interview someone, we have them make a pledge. And the way it works is we talk about how we're an innovative company. We want you to innovate. That's important, but that's not the most important thing.

(Jonathan Ross at 00:23:03) Actually, the most important thing is that if you join Groq, you actually agree that you're going to help other people innovate. It's not just about you. It's about helping other people innovate. And sometimes it's your turn. Sometimes it's not. Right? The difference between a local and a global maximum is inherently, it means sometimes you don't get the thing you wanted to do.

(Jonathan Ross at 00:23:21) That means that sometimes three times in a row, you might get the short end of the decision stick. Right? But you commit to helping us do the innovative things. And then sometimes you'll be happily surprised because it's your idea that gets selected. Right? And so that's helped us scale up and stay innovative.

(Joel Beasley at 00:23:52) Well, you're setting the expectation upfront with really bright, driven people. Right? Because those people, they want the project. They want it to be their project. That's just the personality type I know because I am one of them.

(Joel Beasley at 00:24:06) And so, what I—as you were talking, it reminded me, in my life, I play multiple parts. So in some areas, I'm playing to the strength and I'm the leader. And other areas, like outside of work, like in my fitness world, I surround myself with people who are really, really strong, and they're leading, and I'm just following, because I think it's good practice for me.

(Jonathan Ross at 00:24:33) Well, and I think there's another ingredient, though, and I think it's something, you know, I've seen you exhibit. But I think you can't have innovation—so there's a difference between innovation as an individual and innovation as a team. And any movie you watch about the tortured innovator, you know, Tesla or whoever. Right? They have the idea, and they're just struggling to get other people to understand it. And somehow that story has become a sort of victim story of that person. That person's a victim to the rest of the world, but that's not really what's going on.

(Jonathan Ross at 00:25:07) What's going on is sometimes IQ doesn't match EQ. Sometimes someone can be super creative, and they can't actually communicate with others. So we've actually found that one of the most important things when we hire someone as an engineer is that they have empathy. How do you figure that out? You talk to them.

(Joel Beasley at 00:25:27) What type of questions do you ask?

(Jonathan Ross at 00:25:29) You want to get interviewed now?

(Joel Beasley at 00:25:33) Sure. Let's find out.

(Jonathan Ross at 00:25:35) Okay. Well, the first thing I do is I say, I'm going to ask you to come up with as many creative uses as you can for a household item. You're going to have two minutes. Right? We'll do it right now.

(Joel Beasley at 00:25:48) I'm going to get this timer.

(Jonathan Ross at 00:25:48) Yeah. I'm going to get my timer up, and we're going to interview you. We'll, you know, maybe you'll get a job offer out of this. Okay. So here's the way it goes. So you'll be asked to come up with as many creative uses as you can for a household item. I'll name the item, and I'll say go, and I'll start the timer.

(Jonathan Ross at 00:26:18) Now as an example, this is not the item, but as an example, if I were to say bricks, you could say you could build a house or a wall, but those wouldn't be creative uses. Everyone knows how to do that. Right? So if, on the other hand, you said break a brick in half and use the rough edge to sand something down, use a brick to store potential energy, use a brick to raise the level of water in a bathtub, those would be three creative uses that would be worth three points total. Right?

(Joel Beasley at 00:26:37) Okay. I'm getting it. Okay.

(Jonathan Ross at 00:26:40) Okay. So are you ready?

(Joel Beasley at 00:26:42) Using the object for other than its intended purpose counts as creative.

(Jonathan Ross at 00:26:46) Yeah. And the more creative, the better. But you shouldn't have to explain it. You should just say it and go on so you can get as many as you can. All right. Are you ready?

(Joel Beasley at 00:26:56) All right. I'm going to fail, but, yeah, let's do it.

(Jonathan Ross at 00:26:59) Okay. Here we go. Household sponge. Go.

(Joel Beasley at 00:27:03) A sponge? Yeah. Well, a lot of the items that you described, like using the rough edge of the sponge to sand something, because you gave that example.

(Jonathan Ross at 00:27:15) Uh-huh.

(Joel Beasley at 00:27:15) What else could you do? Well, you could—can I have more than one sponge?

(Jonathan Ross at 00:27:20) Go for it.

(Joel Beasley at 00:27:20) You could put some sponges into a shoe and use them as a sole. Yeah. You could take a sponge and—what could you do? You could make art out of it. You could do some art with it. You could take the sponge, and you could—

(Jonathan Ross at 00:27:39) So I want you to think like a kid. What would a kid say?

(Joel Beasley at 00:27:42) Okay. What would a kid say? A kid would eat the sponge. Yummy. Yeah. A kid would throw the sponge. A kid would rip apart the sponge. A kid would put the sponge in their nose. A kid would take the sponge and put some paint on it and use it to make a design. A kid would take the sponge and throw it at their sister. That's what Lachlan would do.

(Jonathan Ross at 00:28:12) Okay. Time's up. Did you notice how much faster you got when I said think like a kid? Yeah. So it turns out, one of the best ways to improve creativity in a group is just tell people, "Come up with the worst possible ideas you can." Right? It's just, give me the absolute worst. And the more grown, the better, you know, the more points you get.

(Jonathan Ross at 00:28:32) And usually by the fourth or fifth idea, you're like, "Actually, that's kind of a good idea. No points for you." Right? But then you get good ideas out of it. And it's just, people are really afraid to say, you know, creative ideas. They're afraid of being wrong. When you say think like a kid, there's no harm in a kid saying something that's wrong. Right?

(Joel Beasley at 00:29:00) Yeah. We do that in our meetings. I say, "Let's get all the bad ideas out first," just because of experience. I just learned that you've got to get past the stupid ideas. It's just something in life. When I hung out with some of my friends that write music, they say, "Let's just get all the bad cliché, cheesy lyrics out first and let's just roll with it," you know.

(Jonathan Ross at 00:29:20) Yeah. And so, you know, innovating as an individual, it's all about sort of tamping down on that gut feeling of fear. Right? Once you get past that, then you start working on innovation in teams, and that's where all that other stuff we talked about comes in.

(Joel Beasley at 00:29:36) Yeah. Yeah. I like that. That was fun, the sponge thing.

(Jonathan Ross at 00:29:41) Wasn't it?

(Joel Beasley at 00:29:42) What are better uses of sponge? Give me better ones. I obviously didn't do—I don't know a lot about sponges. You're going to interview me now?

(Jonathan Ross at 00:29:49) You're going to interview me now?

(Joel Beasley at 00:29:50) Yeah. Give me the sponge. Go. What can you do with the sponge?

(Jonathan Ross at 00:29:52) All right. So you could use it to transport water. You could use it to fill it with water and kill a spider by throwing it. You could then freeze it with the water, and now you've got a hard object that you can throw. You can use it to deliver water slowly to a plant. Like you said, you can use it to cushion feet, but you can also cushion a blow. You could use it as padding. Right? You could fill a seat that has holes in it.

(Jonathan Ross at 00:30:26) You could probably also use it to—oh, you'd probably use it to stop sound, so it's great for a recording studio. Right? You can stack things on top of it. You could isolate from vibrations in the ground. So you'd probably put a microphone stand on a sponge, and therefore, it would dampen it. Right? Dampen the oscillations. You can insulate from electricity. You can—oh, you can use it to thermally insulate. So you could avoid leakage of temperature and stuff. I could keep going if you want.

(Joel Beasley at 00:30:54) Yeah. You got two minutes.

(Jonathan Ross at 00:30:56) Okay. Well, I hope you're timing because I wasn't.

(Joel Beasley at 00:30:58) No. All right. We're good. We're good. We're good. We can stop. No. That's great, though. Do you agree or disagree that this is a skill that can be developed?

(Jonathan Ross at 00:31:09) It can definitely be developed. It's a little bit like being a basketball player. Right? You can definitely get better, but some people are seven feet tall. Right? And the thing is, in sports, one of the reasons that there's such a difference between the best of the best and the next best is that you can only have so many players on the court at a time. Right? That's not true of innovation, though. Right?

(Jonathan Ross at 00:31:41) So you don't have to be the best of the best. Right? This is why Kaizen works in manufacturing. Right? So you can have a plant full of people coming up with ideas, and it can really meaningfully contribute to the company. So you should have everyone innovating, and everyone can improve. And you shouldn't expect everyone to be a seven-foot innovator, if you know what I mean. Right? You just need them to be better at it, and you'll have an improved product. Right?

(Joel Beasley at 00:32:05) When you're interviewing, do you care if the individual—or do you somehow probe if they've done an exercise like this before?

(Jonathan Ross at 00:32:12) I always pick a different item. And the other thing is, one of the things that you want to look for—and actually, a little bit of advice to people who are trying to hire great people and people who will innovate—we look for something that we call, and we'll come up with a better name in the future, but we call it "booking the win early." And the way it works is you'll notice that someone can say, "Hey. I have a way to make the chip twice as fast."

(Jonathan Ross at 00:32:42) Right? And most people will hear, that's a way to make the chip twice as fast. A subset hears, if we don't do that, the chip will be half as fast as it could be. And those people, you want to hire because for them, it's like taking something away from them if you don't do this amazing thing. They are super diligent about following up on every idea.

(Joel Beasley at 00:33:06) I'm learning a lot. This is good. I've got some unique insight from you. You did bring up something though that I was curious about. You used a manufacturing reference and that kind of triggered this conversation I had.

(Joel Beasley at 00:33:19) And I was also curious to know about how you guys do it. But I talked to this guy named Erwin. He's a CTO of a company called Optessa. And what they do is planning, scheduling, and manufacturing. Think about Ford putting their cars together, everything coming in from different times, different places, paint colors, line, queue length, optimization for production in these systems. Right? Lots of manufacturing. And then I thought you guys make these chips. And I don't know if you make everything in-house yourself, but how do you coordinate and pull everything together to put your chips together?

(Jonathan Ross at 00:33:52) Got it. So actually, a question that every technology company should ask themselves, and I actually learned this one from my chief of staff. So if Estelle's listening, thanks for this one. Companies are often designer, manufacturer, or a combo. And we learned this because when Estelle joined, we were trying to do OKRs, objectives and key results. That's how we planned things for the company. And I came from Google, Estelle came from Intel, and we realized we had a disconnect and we were arguing for like an hour. And she was like, we have to get to 100% of our OKRs. We have to hit everything. Right? And I'm like, no, no, no. If you hit everything, it means you were phoning it in. Right? You should only hit 70%, because that's what Google does. Right? We were products of our old environments. Right? We didn't understand that maybe there was a better way. And so it was actually Estelle who came up with this. She's like, oh, I get it. After like an hour of arguing, she came from the memory portion of Intel, and they had to build memories the same way every time, identically, repeatably over and over. Right? But we were at a different end of that. Right? We use a fab. Right? We don't build the chips ourselves. We actually design the architecture. We design the software. And so we're a design company. And if as a design company, if you do the same thing twice, you've done nothing. So it's a pretty fundamental disconnect in how you approach engineering. Right? It's weird that we call those two different things engineering because they're so different from each other.

(Joel Beasley at 00:35:31) So how do you put everything together?

(Jonathan Ross at 00:35:33) Super glue. Super glue? Yeah. So great people. Right? So yeah. You have to design something that can be manufactured. You have supply chain. You build chips. Every chip is like a potato chip. They're all different. They're special little snowflakes. They've each got their own. One molecule is a little over here. So this particular transistor is a little slower than that one. And our job is to provide you a chip that behaves the same way no matter what's happening in that physical process. Right? And so that's part of the manufacturer problem of how do we build something that behaves like a digital thing. Right? But what we spend a lot of time doing at this company is, okay, once it's that digital thing, what digital components can we put together and make into something extraordinary? Right? Not like all the other digital things that people are building. We actually came up with something, which is sort of the thing that comes after digital. Right? And that may sound weird because digital is baked into our lot. What could be the thing that comes after digital? But remember, there was a time before digital. There was a time before you could actually have permanence of objects. Right? If you had a picture, it would fade. Right? If you had a written note, yeah, someone might copy it exactly, but eventually, someone's i is going to look like an l, and it's going to get copied incorrectly. Right? And so digital changed things for us, but it wasn't always here. And just like with digital, where digital allows you to repeat the same thing over and over again. Right? It allows you to get a predictable result, a correct result. Right? Imagine photocopying an image over and over and over and how it would degrade. Right? With digital, that doesn't happen. You get the same result at the end. Our chips are deterministic. And a lot of people use that term, but what we mean is when we compile a program for our chip, it runs for 43,517 nanoseconds according to the compiler. Then it's going to run for 43,517 nanoseconds plus or minus zero. There's no variation in the runtime. And so whereas digital made correctness repeatable, what we've done is we've actually made performance repeatable. Right? Literally, a program runs for exactly the same amount of time, and we know it at compile time.

(Joel Beasley at 00:37:53) That's pretty smart. That's pretty interesting.

(Jonathan Ross at 00:37:57) But it's about scale. Right? Because, you know, the problem we were having at Google was we were trying to make it double exponential. Right? The chips were growing exponentially. We were adding exponentially more servers. Right? The data centers were getting bigger. We were getting more of them. And the problem we started to have was unpredictability in how the servers behaved. Right? If one of your servers is, you know, one one-thousandth of the time too slow, if you're calling out to a thousand servers, then you're always too slow. Right? So that's a problem with scale. And so what we've done by bringing in determinism is actually we can have thousands of chips working together on a problem, and we still know exactly how long it takes to finish. Right? It's like going to a restaurant with or without a reservation. If there's no reservation, you don't know how long you're going to wait. If there's a reservation, you just sit down, and you save a lot of time.

(Joel Beasley at 00:38:50) Yeah. Because I was just watching the time here. I want to be respectful of your time as well. I cannot get the sponge out of my mind. I was trying to think about it. It's almost like you're getting people to do a stream of consciousness. Like, it's just getting that going. Creative stream of consciousness. Yeah.

(Jonathan Ross at 00:39:06) And it's more about what is this person capable of rather than what have they put on their resume. Right? So I think in interviewing, a lot of people look at the resume, and they're like, that fits what we're looking for. Great. And what we're trying to do is, here's the thing. If you want to build an innovative product, you can only hire people who are innovating on themselves. Right? You can't hire someone who comes in and says, I'm an expert. Just listen to me. Do what I say. Don't ask questions, and then expect to get an innovative product. Right? So when we hire people, we're looking for people who are looking to grow. Right? And when we say grow, we don't mean like, oh, you get 10% better at something. We mean like uncomfortable growth, like phase change growth. Like, when you were a kid, you probably wanted to be able to run as fast as the Flash, maybe fly like Superman. You probably wanted to beat up criminals and be a hero. Am I hitting on anything here?

(Joel Beasley at 00:40:07) It's talking about everybody. Right?

(Jonathan Ross at 00:40:10) Yeah. And then you grow up, and you're like, okay. Maybe I could buy a car. Maybe I could buy a plane ticket. As much trouble as there is with our police force, it's better than people going around and randomly beating people up as vigilantes. Right? So maybe there's another thing that I could spend my time doing that would be more meaningful to the world. Right? And so the realization that the things that you thought were important aren't that important, and there are more important things, is growth. And we want to hire people who can do that with themselves so that they can also do that with the product that we're building.

(Joel Beasley at 00:40:49) You ever ask people to describe like, the story of their life?

(Jonathan Ross at 00:40:53) I haven't done that one. I like because if you do that, it could throw them off a little. It could get them thinking a little creatively. But, most likely, they're just going to give you what they thought that was, and you have to guide the conversation a little bit. Right?

(Joel Beasley at 00:41:14) Sometimes I ask people what the most difficult thing they've ever experienced in life is.

(Jonathan Ross at 00:41:18) Starting a startup. Yeah. Other people have definitely gone through more difficult things, but that was the most difficult for me.

(Joel Beasley at 00:41:26) Dude, Elon Musk says it's staring into the abyss and eating glass. And when I heard that, I said, yep. That's how it feels, my friend.

(Jonathan Ross at 00:41:33) Well and ghost pepper covered glass. Don't forget that part.

(Joel Beasley at 00:41:36) Ghost pepper covered. Are you surrounded? Are you in a group with like, other founders? How do you keep yourself balanced, or how do you get around people who you respect, but you can also vent or talk with?

(Jonathan Ross at 00:41:53) Okay. That's super, super important. Actually, yes. There are other startup founders, some I knew before they became founders, that we talk to each other and trade advice. There's also a couple of groups, one led by one of our investors where every company they've invested in, it's just a WhatsApp channel, and we can all talk with each other. And we trade advice, you know, what external recruiters are good, you know, what should I do with this situation, that kind of stuff. And we're pretty open with each other, and it helps because that investor sets that tone. They're open about themselves, and that gets other people to be open as well. But yeah. And actually, I would say it's really important because even though the sort of problems that come up when you're running a company feel unique because they're not the sort of things you're trained for in school or other experiences, they're not unique because everyone else who started a company has gone through exactly those same problems.

(Joel Beasley at 00:43:00) That is 100% true. Yes.

(Jonathan Ross at 00:43:03) One of our investors likes to say, starting a startup is like playing those old Nintendo games, which is every level ends with a big boss. And that first big boss that you beat, you're like, wow. That was tough. I don't know how I'm going to beat the next level. And you keep advancing, and you get to bigger and bigger and bigger bosses. And if you ever play that game from start to end again, it's boring at the beginning because you know how to beat those big bosses. But later on, like, as you keep advancing, each one feels equally difficult as the one that you just defeated.

(Joel Beasley at 00:43:35) Right? Yeah. Except I just go look up like, the cheat codes and just advance. Yeah. And it's—

(Jonathan Ross at 00:43:40) That's what it's like talking to other people and asking them for advice. Right? And that's the smart thing. Yeah. Yeah. You're clever.

(Joel Beasley at 00:43:48) I'm so serious. I was, the way when I started this podcast, I was watching all these people that were lesser engineers than myself because I have seventeen years software engineer. I would go look at their work, and I'm like, how are they making millions of dollars? How are they? Relationships. It's all about relationships. Empathy. Yeah. And I said, how do I get relationships? And then I was like, alright. I heard Gary Vaynerchuk talking about, he's like a marketer. Heard him talking about podcasts and building, it's a lot of work and it takes a lot of discipline. I have a lot of discipline. And so I said, alright. I'm going to do that and build relationships. And I know it'll take five, ten years, but I'm going to go do it anyways. And I mean, it was a lot of work, I guess, but it was a lot of fun too. Because I love hanging out and talking. I love the sponge thing. That is such a good exercise because it's not super easy right off the bat. Usually, things I get pretty quickly. That is not immediately easy, which means it's something you have to work at, which the best things in life are the things you have to work hard at.

(Jonathan Ross at 00:44:55) Well, I think as Don Staley says, growth only happens outside your comfort zone. Right? So you want your interviews to grow people. You want to make them uncomfortable a little. Right?

(Joel Beasley at 00:45:07) Where's your growth area then? What's the, what's where, where are you focused right now? What's the thing you're doing that's uncomfortable?

(Jonathan Ross at 00:45:15) So actually, the interesting thing about growth is once you know what the area is that you're working on, then that's half the battle. Right? The other half is then working on it, but the hard part is figuring it out. So I work with a coach. Actually, one of the unique features of Groq is we actually are putting together a coaching program for every single employee. Right? It's not just the top folks. It's everyone because we take growth seriously. Right? You can't have an innovative product if you don't have people innovating on themselves. And so it's just part of it. And so I have a coach. And one of the areas that we're working on, he says I need to be more vulnerable.

(Joel Beasley at 00:45:57) Alright.

(Jonathan Ross at 00:45:58) So I think that's a vulnerable thing to say.

(Joel Beasley at 00:46:02) Yeah. Yeah. You get credit for that for sure. But tell me, did he give you a lot of training data, or did he just say be more vulnerable?

(Jonathan Ross at 00:46:10) Oh, he had me dialed in. Like, as soon as he said it, I'm like, yeah. You're right. Like, he didn't need to— How do you do it? Well, we're working on that. That's the second part. I'll let you know when we rematch.

(Joel Beasley at 00:46:22) Please. Please do. Yeah.

(Jonathan Ross at 00:46:23) You know, my interview is all about whether or not people could think outside the box. So you actually did come up with some good ones.

(Joel Beasley at 00:46:30) I did not do good. I needed to be fluid, and that's the bar. By the way, no matter how well I did, I beat myself up.

(Jonathan Ross at 00:46:37) And that's an important thing. I mean, you have to always want to grow and improve, but like, the other lesson there is have you read Growth Mindset by Carol Dweck?

(Joel Beasley at 00:46:49) I've listened to the audiobook.

(Jonathan Ross at 00:46:50) Yeah. Yeah. Yeah. Then that's reading nowadays. Did you consume the book? Right?

(Joel Beasley at 00:46:55) I consumed it. Yes.

(Jonathan Ross at 00:46:57) Okay. So the thing is you can actually crush people's ability to improve by giving them the wrong kind of compliment. By saying, look. You're awesome. People are then going to be worried about not being awesome the next time they do something. So you have to be really specific in the feedback you give. And so I actually kind of messed up with you right there where I'm like, you know, you did a good job. Like, one of the things I liked was how quickly you were able to take the thing that I said about the brick and the rough edge and apply that to the sponge. Right? Most people don't do that. I give that example. And if I were to ask them about a sponge, they wouldn't bring that up. So you were making connections. Right?

(Joel Beasley at 00:47:35) That was the only thing that was in my head was the three examples. I was like, those probably don't count. And then I had some weird constraints too, which was interesting. One of the things I learned after was that I had constraints that it had to be an invention of some sort, that it had to be somehow more, that it had to be interesting to me.

(Joel Beasley at 00:47:54) Versus, like, with your example data set, you broadened how weird I could get, for lack of a better word. For me, there was these rules in my head that were already there.

(Jonathan Ross at 00:48:06) And that's the problem. Right? It's functional fixation. Right? Everyone gets fixated on the way. Like, so I've done a lot of interviews.

(Jonathan Ross at 00:48:14) I actually interviewed everyone at Groq up to 100 people. So I've done a lot of interviews. And one of the things I learned is when you ask a question that's too close to people's experience, they actually have more trouble being creative because they just draw on what they know. And so you want to ask them things that they have no experience with so they can be more creative. Right?

(Jonathan Ross at 00:48:34) And so this is also a lesson for how you organize a company. You don't want to put the person who's been doing something for twenty or thirty years in charge of that thing. You want them to advise someone who's had some similar, but not the same experience. Right? So they can think a little bit outside the box.

(Joel Beasley at 00:48:52) Yes. Dude, this is great. I want to plug your company a little bit more before we wrap up here. Alright. Who are your customers?

(Joel Beasley at 00:48:59) Why do they buy?

(Jonathan Ross at 00:49:01) So, verticals, we have AV customers. We have finance customers. But the common thing, the vertical part isn't as important as, does someone have a compute problem that cannot be solved with today's existing solutions? Like, is it something where there is nothing that's low enough latency? Right?

(Jonathan Ross at 00:49:24) Because you can buy throughput. You can spend more for it, but you can't buy more latency. Right? Or do you need to scale up beyond what's possible with GPUs or CPUs? Right?

(Jonathan Ross at 00:49:36) And those are the sorts of folks. So AV companies love us because we're super low latency, deterministic, predictable. Right? Like, it's great for when you're trying to build real world systems. And the finance folks love us because we can scale up beyond anything they've ever seen before.

(Joel Beasley at 00:49:52) That's right. Because your value prop is strong. I mean, it's like what everybody wants. Like, we can process everything much faster. More compute.

(Joel Beasley at 00:50:00) Yeah.

(Jonathan Ross at 00:50:01) Like, just one last analogy on that, by the way. So what is the difference between a product that becomes a commodity product and a product that never becomes a commodity product? So let's say that I offered you a car that was identical to your current car in every way, except it could go twice as fast and it cost twice as much. Is that a compelling proposition?

(Joel Beasley at 00:50:26) It depends on how fast the first car can go. Can the first car satisfy all my needs?

(Jonathan Ross at 00:50:31) Well, you could drive at highway speeds, and you're sort of hitting on to the point. You're getting to the punchline. Right? Which is, now imagine I offer you a car that can get you to work in half the time. Would you be willing to pay twice as much for that?

(Joel Beasley at 00:50:46) Is it legal? Street legal?

(Jonathan Ross at 00:50:48) Absolutely.

(Joel Beasley at 00:50:50) I would.

(Jonathan Ross at 00:50:51) Yeah. So cars have sort of become commoditized because there's not much to improve on them. But with compute, no one has ever said, oh, you know, I've got enough compute. I'm good. Why don't you give that to someone else? Right?

(Jonathan Ross at 00:51:07) People always want more compute. And so that's why this is such an interesting field to be in, because you can just keep making your chips faster and faster and faster, and it makes things run quicker, and you can get workloads done more quickly. And it's sort of like there's all these things that people want to do. They're almost like options that are out of the money. And when you give them enough compute, the options or the things they want to do come into the money, and they want to do them.

(Jonathan Ross at 00:51:35) And so this compute could just keep going for decades or years being an interesting field.

(Joel Beasley at 00:51:42) When are we going to reach the max potential of what the physical chips can do? Isn't there some sort of limit?

(Jonathan Ross at 00:51:47) Landauer's limit. We talked about it. That's the limit. And how close are you to the limit? So it actually depends on the temperature at which you're computing.

(Jonathan Ross at 00:51:58) So as weird as that sounds. Right? And I think we're multiple, like, I think it's more like a million X off, but also don't forget you can change the question you ask. Right? So there's this thing called reversible computing.

(Jonathan Ross at 00:52:15) And if you form your algorithm in a way where nothing is erased, you can actually, sort of like taking a shortcut, you can avoid Landauer's limit. So we're going to start getting clever in how we phrase the problems, not just in how we actually run them.

(Joel Beasley at 00:52:30) Okay. Earlier, you were, I didn't go into it too much because I just, you're talking about the little demon in the door, and you're talking about information getting erased. I lost track of, like, information. Give me an example of information being erased.

(Jonathan Ross at 00:52:43) So when you add two numbers together.

(Joel Beasley at 00:52:48) Okay.

(Jonathan Ross at 00:52:49) Can you, and I give you the result. Do you know what those two numbers that were added together are?

(Joel Beasley at 00:52:57) That would be information getting erased.

(Jonathan Ross at 00:52:59) However, if you know one of those two numbers, you actually can figure out what that number was. So has it been erased? So now it gets so weird because it's like, think of it this way. Imagine you encrypt something. Is it erased when you lose the key?

(Joel Beasley at 00:53:20) I like you. Yes. Yes. Yes. Yes.

(Joel Beasley at 00:53:23) Yes. Yes. Man, it must be so much fun working at your company. Are there other people like you there?

(Jonathan Ross at 00:53:27) Oh, yes.

(Joel Beasley at 00:53:28) That ask these questions?

(Jonathan Ross at 00:53:29) Remember, I try and hire people who are smarter than me. Right?

(Joel Beasley at 00:53:32) Oh, wow. Yeah.

(Jonathan Ross at 00:53:33) Yeah. It's fun. Like, we've got some really interesting characters.

(Joel Beasley at 00:53:36) Wow. Alright. So do we have a call to action? I mean, you mentioned earlier if they want faster compute, but what do they do? Like, do they go to the website?

(Joel Beasley at 00:53:46) How do they get in touch with you?

(Jonathan Ross at 00:53:48) So you can go to the website. At the moment, we're working with really large companies. It's more sort of a mutual thing. We're pretty much talking with everyone we want to, but the call to action is we're going to start broadening that up. You know, we've got this, we started with the compiler.

(Jonathan Ross at 00:54:05) It makes it easier to port things to our chip. And so once that starts happening, we're going to make this available to a lot more people. And so I guess the call to action would just be, stay tuned because we're pretty soon going to have something that everyone can use, and it's going to really accelerate whatever people are working on.

(Joel Beasley at 00:54:24) Thank you so much for listening. And if you found this episode useful, please share it with a friend or colleague who you think would get value from it. And if you have topics that you'd like to hear discussed on the podcast, either add me on LinkedIn or send me an email, [email protected]. Every time I get an email or LinkedIn message, it absolutely makes my day and inspires me to keep going.