Inside Google Cloud’s Future of AI Report with Mayada Gonimah, CTO & Co-Founder at Thread AI

(Intro Narrator at 00:00:00) Today, we're talking to Mayada Ghonema, CTO and co-founder at Thread AI, about how you can integrate AI without blowing up your tech stack. You're listening to Joel Beasley, Modern CTO.

(Joel Beasley at 00:00:18) Oh, but I live in Tennessee. Where do you live?

(Mayada Ghonema at 00:00:20) I'm in New York City.

(Joel Beasley at 00:00:21) Okay. You're in New York City. And you like it there?

(Mayada Ghonema at 00:00:23) Yeah. It's a lot of fun. I'm a big fan. I grew up around the Middle East in Cairo, and there's some parallels, but it was a big contrast after graduating. I went to undergrad in the middle of rural Massachusetts in Williamstown. So coming to New York was a big change.

(Joel Beasley at 00:00:40) Oh, yeah. Yeah. I spent some time there. My brother-in-law and some of my family's in Westchester. It's only an hour away from the city. But yeah. Well, I'm excited to talk with you today. So Google Cloud recently launched a report called the Future of AI Perspectives for Startups report. Features 23 AI leaders. They're all discussing the future of AI. But you wrote a section that was really interesting. Now, before we get to your opinions on it, could you just tell me more about the report?

(Mayada Ghonema at 00:01:08) Yeah. Happy to share some of the background. So the context was some of the conversations we had with Google around our experience with operationalizing AI, and a lot of it has been drawn from both previous experiences working at larger enterprises, but more interestingly, my experience building this current company, Thread AI. So we are in the business of helping enterprises and other large companies operationalize safely with AI. So you can think of us as the unsexy glue code that no one wants to write or think about, but that you need to build meaningful workflows with AI. And a lot of what we're seeing stems from some frustrations over the past few years of folks trying to experiment and build workflows and putting them in production and running into all kinds of issues. That's the high-level summary.

(Joel Beasley at 00:02:08) Tell me what type of issues. What type of frustrations are people running into?

(Mayada Ghonema at 00:02:12) So, yeah, I think there's a big explosion of tooling. Almost like there's a lot of many different IKEA pieces that folks are forced to play with. But we're seeing many cases where people have done a few prototypes and have done various different kinds of proofs of concept. But when they try to put a number of these workflows into production, they notice how things start falling apart. And there's many cases where a lot of the tooling doesn't make it easy for folks to embed AI safely into their existing stacks. So companies have already invested in various different portions of the stacks, so in their data layer, in various state machines, and gateways. And a lot of the more recent tooling was pushing for a new paradigm of AI applications, which is pretty challenging. But I think the way we think about it is, and our sense has always been, that AI is never a workflow in and of itself, but is always part of a workflow. So how can you think about intentionally embedding AI into existing systems, which may be legacy or some are more modern, but you don't have to basically blow up your entire stack or redo your entire infrastructure in order to take advantage of some of the advancements in AI?

(Joel Beasley at 00:03:32) I want to blow it up.

(Mayada Ghonema at 00:03:35) If you can afford it, go.

(Joel Beasley at 00:03:37) No, no, no. It sounds expensive. And then are you doing this as consultants? When an enterprise wants to get into this AI, they want to do this...

(Mayada Ghonema at 00:03:47) We're actually not in the consulting space. We're actually a platform. So we do offer our product in the form of SDKs and APIs. And then we have also the UI, the observability component, and also low-code, no-code builder components. And the goal is to always, even in the initial phases, if you are prototyping with a customer and there is any kind of consulting or custom code, our philosophy is you always have to fold it back into product primitives so that you don't end up with these very niche solutions. And if there is some level of data transformation or customization level, we always push them back to the customers who will own that custom code. But, ultimately, everything runs on a platform.

(Joel Beasley at 00:04:35) Okay. And the platform is Thread AI?

(Mayada Ghonema at 00:04:37) So the platform is called Lemma. Yeah. And the company is Thread AI. Yep.

(Joel Beasley at 00:04:41) All right. So the company is called Thread AI, and the platform is called...

(Mayada Ghonema at 00:04:45) Called Lemma.

(Joel Beasley at 00:04:46) Yep. Lemma. Okay. Very cool. Now what do you think the most controversial take that you have is on the future of AI?

(Mayada Ghonema at 00:04:55) There's a few, but, again, a lot of it stems from our unique experience seeing the proliferation of these different tools where there's almost too many tools that don't solve an overarching problem. So there's many different smaller components of the value chain that folks are left trying to stitch together or trying to make sense of that in one cohesive system. But we think a lot of these tools are slowly going to go away. And, obviously, a lot of the main thesis that we had has been playing out. So we knew from the beginning when we set out to build this company that models will continue to become better and get smaller. So when things like the DeepSeek announcement—that was not a thing that obviously would change anything in our road map or in our product. We know that a lot of these models are going to continue to improve, but the question becomes, which part of the tooling system is going to make it easier for folks to take advantage of the improving databases and the improving models? And we think, initially, I guess, compared to last year when there was a lot of hype and buzz around agents, but now folks are taking a step back and saying, okay, no one actually runs recursion that way in production. We need to think about the main fundamentals that you would think in any enterprise software. So the security, the data governance, the provenance, role-based access control, and other unique paradigms around guardrails and human in the loop. A lot of these ideas are resurfacing, and the initial hype around AI-native applications is slowly starting to die down.

(Joel Beasley at 00:06:54) Are you talking about the wrappers? People doing wrappers around it?

(Mayada Ghonema at 00:06:58) So, yeah, there's different flavors of wrappers, but some have rushed to introduce almost new interfaces and new paradigms that folks have used to test various things. But when it comes to production, folks are starting to undo a lot of that work.

(Joel Beasley at 00:07:16) And they're undoing it just because it's not delivering the value that they thought it would?

(Mayada Ghonema at 00:07:20) Or it's not the kind of code that you would run in a mission-critical production system. It's good for proof of concepts or some...

(Joel Beasley at 00:07:29) Any specific examples that you can share?

(Mayada Ghonema at 00:07:32) I guess a lot of the patterns of how some of our customers come to us is they'll do some proof of concept or prototype for a single case. But then once they start putting it through to scale across thousands of users or more in different slightly different flavors of inputs or different security situations, things would fall apart. And they've come to us basically for redoing or rethinking about how do you redefine these workflows using different kinds of primitives and fundamentals. And, also, I guess, initially, there was a lot of push around some of the chatbot primitive, which makes sense. In certain situations, you do need a chatbot. But in many cases, a lot of these problems are best suited as long-running processes or processes that you can pause and unpause durably, and not everything can be solved with a chatbot.

(Joel Beasley at 00:08:35) Nice. And so how did you co-found this company? How did this come up, this opportunity?

(Mayada Ghonema at 00:08:40) So it's been kind of a fun journey. So my background is I've been a distributed systems engineer for around 15 years. I started my career on Wall Street, actually, working at Goldman Sachs. Mostly on the engineering side of things, so building financial services software. I worked within the prime brokerage group, so working closely with hedge funds and futures trading systems. And then I made the transition. After around four and a half years, I went to go work for the New York Times, which was very different. Mostly focused on e-commerce type software, but also building up payment processors. So I got a chance to see different kinds of products, but still within the highly regulated spaces where we were processing a lot of credit card payments internally. So some very similar work to the work done by Stripe or Braintree. But, also, the second half of my time there, I spent working on moving a lot of the infrastructure from on-prem out to cloud-native. So I got a chance when some of the cloud primitives were still earlier on to spend a lot of time with that infrastructure. And it wasn't until, again, also, the joke is every four and a half years. So after four and a half years, that's when I made the transition into the ML and AI space. I basically wanted to see how I can break away from the e-commerce domain, and candidly, anything ML, AI-adjacent. In 2019, it felt like that space was going somewhere. And, basically, I took a step back. I was considering a few positions, I think, at the time, pretty interesting offers from Spotify and Squarespace. But the Palantir sales pitch was very compelling. And that's when I joined Palantir in 2019, where I met my now co-founder. So at Palantir, I was hired to build a real-time inference engine, so very similar work to AWS SageMaker live deployments. And I got to spend a lot of time working with Department of Defense and the CDC and other regulated spaces, mostly within the ML Ops space. So building things like model training, model inference, and model evaluation. And, yeah, me and my co-founder spent a lot of time working together. She was leading product, and I was leading the engineering side of things. And, yeah, together, we grew the team from three people to over 30 people. And we left in spring of 2023 to found this current company, which is very different from what we built at Palantir. And you can think of it as the sequel. So at Palantir, we spent a lot of time building the fundamental infrastructure around the model lifecycle. So how do you build it? How do you train a model? How do you save it? How do you deploy it and containerize it? And after leaving Palantir, we spent a lot of time researching the market and seeing now that we have these different models, and you can assume these models will continue to pop up. What do you do? How do you build meaningful automations that may or may not use models? And how do you safely operationalize AI, generative or non-generative, to help businesses build meaningful applications? And in some cases, redefine how they do business. In other cases, replace certain processes that are not very efficient or augment certain processes.

(Joel Beasley at 00:12:19) And tell me a little bit more about the work that you're doing with Google.

(Mayada Ghonema at 00:12:23) So Google has been a wonderful partner to us. We initially actually were connected much earlier on. Our strategy has always been to be multi-cloud and as an infrastructure company. But earlier on, one could say maybe it was a controversial take. We, before even finding product-market fit, we invested in making sure that we are not just running on one cloud and started redefining some of our stack so that we can build with Google and take advantage of some of the best-in-class Google primitives and SDKs and infrastructure. So we were introducers through the Springboard program, and we started basically building a lot of our primitives there and then offering it on the Google Cloud Marketplace. So for some of our Google Cloud customers who have committed on cloud spend, they can simply purchase our product through the marketplace. What I've uniquely enjoyed working with them is they get it for startups. Obviously, they see all kinds of startups, and our needs as a workflow automation engine is very different from some of their other startups who are foundational model shops. Their acute GPC needs are mostly things like GPU access versus our needs, which are very different. And, also, we will have different usage patterns and enhanced different commit spend.

(Joel Beasley at 00:13:55) And what's your role versus your co-founder's? How do you define those responsibilities?

(Mayada Ghonema at 00:14:00) So we're fortunate in that both of us actually have engineering backgrounds. So my co-founder, Angela—but we both come from an engineering background, but very different where she focused mostly on the data side of things and some of the AI and ML and the math and the theory more than I did from my background. My background has mostly been distributed systems and more systems and infrastructure focused. But the way we've worked, even at previous places like Palantir, she was leading product, and I was leading engineering, which has been a nice split. Obviously, in an infrastructure company, there's a lot of overlap. But the way we like to—obviously, when the company started, we spent a lot of time working together on a lot of different things. But now there's way too many threads where we can't both be in the same thing at the same time. But there is a nice split. Obviously, there's a lot of sales and operational work and legal and outreach and recruiting. And there's also product and engineering infrastructure. So we try to split things evenly, and we both go in and out of operational things. But the goal is, as we scale up, we do less and less operational work and continue to focus on product and infrastructure. So she will drive a lot of the work with design and customers, and I will focus on basically a lot of the engineering and architecture and infrastructure.

(Joel Beasley at 00:15:38) We had a related question from somebody who listens to the show. They listened to Meg's episode, and then they wrote in. And I thought the question aligned well with you because your role as CTO. Is that correct?

(Mayada Ghonema at 00:15:50) Yes. That was good.

(Joel Beasley at 00:15:51) So can I read you the question and we can try—we could try to figure out what the question really is, and then we can try to answer it? Is that okay?

(Mayada Ghonema at 00:15:58) Sounds good.

(Joel Beasley at 00:15:59) All right. So there's a lot of questions there. Let's start with the beginning. I'll break it down a little bit. CTO role definition for early-stage startups. What should CTOs be doing in early-stage startups?

(Mayada Ghonema at 00:16:11) I think the CTO role isn't too different from the traditional T-shaped skills role, but maybe there's a lot more emphasis now on the breadth. But you still have to have a core technical competency, whether it is in deep infrastructure or AI if you're a researcher or pure back-end or front-end, there needs to be at least one technical path that, obviously, you have mastery of. But at the same time, the expectation now is you can step in and out of different levels of implementation detail. So that's something that we pride ourselves in. Obviously, we're only 13 people now. Things are going to change.

(Mayada Ghonema at 00:16:58) But even when we led groups, so like internal teams of 30 or cross-functional projects that span maybe 50 people, the ability to step into every single line of code, but also all the way up to explain the problem space to nontechnical users is critical. Obviously, the path to CTO and the experience of each CTO is very different and unique, especially if there are some folks who haven't come from a purely traditional technical background. But at least for us, and especially for attracting the kind of talent that we want to attract, people want to see that the founders actually know the code or can step into different RFC reviews or architecture design. Obviously, as we grow, there's going to be less time to go to know every code base. But for the current size, I think a bar that I hold for myself is if there's a production issue, I should still be able to diagnose it and go in. If hell broke loose and there's no one left in the company, you can still push the feature out or fix the issue.

(Mayada Ghonema at 00:18:06) And obviously, there are ways to scale yourself and split the responsibility. But being able to diagnose and know where things are or know what kind of questions to ask to unblock any kind of unknown. So basically an ability to know the unknown unknowns within the scope of your company, obviously.

(Joel Beasley at 00:18:27) And for context, what's the current size of your company?

(Mayada Ghonema at 00:18:30) So we're 13.

(Joel Beasley at 00:18:31) Okay. So you guys are like brand new, making it happen.

(Mayada Ghonema at 00:18:35) Yes.

(Joel Beasley at 00:18:36) Cool. Because that changes when you hit like a thousand.

(Mayada Ghonema at 00:18:38) Exactly. Right?

(Joel Beasley at 00:18:39) Yeah.

(Mayada Ghonema at 00:18:39) It's completely different. Yeah. And people's views on work are also very different philosophically. So I know some of my views might not agree with everyone, but a lot of the folks that we hire at their core are very academic. They're very hungry. They're very humble, and they're interested in the content. So when folks are doing work on the weekend, it's not because anyone has asked them to, but because they're excited by the space. And obviously, people's relationship with work is very different. But for me, I find it hard to invest or have the be in a career where if I'm just working for the weekend or the two days in the week, I would rather work for the seven days or for the five days, not so finding something that you don't mind spending the entirety of your life doing because that is your skill, your craft, part of your identity. But obviously, some people might think that's very toxic.

(Joel Beasley at 00:19:47) No. I don't think that's toxic at all. And I have a saying that I share around my family and house, and it's you find something you love and you let it kill you.

(Mayada Ghonema at 00:19:57) Yep. They go there.

(Joel Beasley at 00:19:58) Either way, you're dying. So you might as well find something you love and put your time into that and let that kill you than things you don't love killing you.

(Mayada Ghonema at 00:20:06) Exactly.

(Joel Beasley at 00:20:07) So I get it. And also, it's hard because it's a moving target. Right? But you've been managing people for years. When I was that guy that would just because I loved the code and loved the problem, I'd be working on the weekend.

(Mayada Ghonema at 00:20:20) Mhmm. Yep.

(Joel Beasley at 00:20:21) Until I had a family. Now even though I still love the problem, like I still love it, but family just has to happen on the weekend now. And so it changes. Your dynamic changes. When I was a single guy versus married with three kids, it's just different.

(Mayada Ghonema at 00:20:36) Yeah. And also, yeah, where exactly where in life you are.

(Joel Beasley at 00:20:40) Mhmm.

(Mayada Ghonema at 00:20:40) And what coefficients you're looking to optimize for. Obviously, there were some stages in my life, even financially, where I couldn't just quit and build a startup, which obviously is a privilege, and not everyone is fortunate to have those opportunities. So it's not something that I take lightly. But yeah, to your point, there's point in other stages, not even just financial, where you have other, where life happens, basically, and starting a company might not be the right time.

(Joel Beasley at 00:21:14) Yeah. Alright. Well, no. Thank you for answering that for me and getting into that. I want to talk about, as we start to wrap up here, what's one tip that you can give to CTOs and technology leaders out there listening today?

(Mayada Ghonema at 00:21:29) This is something we always internally try to live by is don't reinvent the wheel. There is so much great open source and literature that is out there. And just because it is not AI first, it doesn't mean it's not suited to solve some of these problems. So I think spending some time before any product initiative, doing a fair bit of research on the ease of the existing tooling is really fundamental. I've seen cases with other startups where folks will just rush to build, build, build, but a lot of without necessarily scoping out the market or going deeper in some of the existing libraries. But obviously, we can't spend our entire time researching, but I think there's a lot of great stuff there, if not for repurposing for inspiration at least.

(Joel Beasley at 00:22:27) And what's one mistake that you've made early on at your startup?

(Mayada Ghonema at 00:22:31) I think the people components will continue to be challenging. And obviously, the demands of a venture-backed startup is very different from bootstrapping a slow growing business. I think it's hard sometimes to divorce emotionally from some of the decision making where as a company scales and grows, you have to be so good at compartmentalizing, so good at, and where one great meeting followed by one bad meeting followed by another so and so. Basically, doing that emotional reset every time is something that continuing to kind of push myself to do.

(Joel Beasley at 00:23:19) It's hard. But you're making it happen. You're out there making the infrastructure and building it so that teams don't have to and they can move faster with these AI models. Is that right?

(Mayada Ghonema at 00:23:26) Yep.

(Joel Beasley at 00:23:27) Nice. Now Google Cloud, Future of AI Report coming out, Perspectives for Startups. We're going to post a link to that in the show notes, and it will be active February 25. This has been great. Anything else you want to get out there to the world today before we wrap up?

(Mayada Ghonema at 00:23:49) I think, yeah, it's, I know the ecosystem is pretty overwhelming, but it's never too late to start. We've been fortunate in working with all kinds of companies. Some obviously who have a leg up and have invested in their data and folks who are still starting are much earlier on in their AI journey. And I think both have been equally rewarding experiences. For us, our mission is to democratize this kind of infrastructure and help level out the playing field because, I guess, yeah, otherwise, things will continue to result in pretty big gaps, and AI will make that faster. So we're always thinking about how can we embed and help folks who are maybe stragglers in the AI journey, and how can we give them that leg up so they can be on the same playing field as folks who have a lot of the AI infrastructure.

(Joel Beasley at 00:24:51) Thank you so much for listening. And if you found this episode useful, please share it with a friend or colleague who you think would get value from it. And if you have topics that you would like to hear discussed on the podcast, either add me on LinkedIn or send me an email [email protected]. Every time I get an email or LinkedIn message, it absolutely makes my day and inspires me to keep going.