The Quantum Leap in Digital Analytics with Mario Ciabarra, CEO & Founder at Quantum Metric

(Intro Narrator at 00:00:01) Today, we're talking to Mario Ciabarra, CEO and founder at Quantum Metric, about how they're leading the revolution of digital analytics with AI. You're listening to Joel Beasley, Modern CTO.

(Mario Ciabarra at 00:00:18) I play games. I play a little Fortnite with my son and stuff like that. But, you know, Super Mario, it's a little bit outdated, I guess. But I'll play if it comes up. Or Mario Kart.

(Mario Ciabarra at 00:00:26) Let's go.

(Joel Beasley at 00:00:27) My kids are fans, and I do play Mario Kart on the Wii with them.

(Mario Ciabarra at 00:00:33) Yeah. Nice. How old are your kids, Joel? Good to meet you.

(Joel Beasley at 00:00:37) Good to meet you. I have three kids under seven. So seven, five, and two.

(Mario Ciabarra at 00:00:40) All right. Well, I've got three kids over seven. They're 13, 15, and 17.

(Joel Beasley at 00:00:47) Is that a tough time?

(Mario Ciabarra at 00:00:49) I think they're all tough times, and they're all enjoyable times is what I would share. But there's some good and some bad and some crazy, but, you know, whatever. It all works.

(Joel Beasley at 00:00:59) Yes. Yes. That is absolutely the truth. Well, what I had as our main goal today was to help the audience understand what a digital analyst does and what an AI-powered digital analyst can do. And so I want to start there. What is a digital analyst?

(Mario Ciabarra at 00:01:21) When we started the company, we made these T-shirts just saying, "We make it simple." And part of the rationale behind those is analytics in general is not very friendly or inviting or easy to both understand and operate. Think about analytics. So I think if you have one person in a store, you don't really need analytics. You can watch someone's experience. You can talk to them. You can understand what's working and what's not. Analytics starts to come into play when we use the word scale. And so if you think about a digital analyst, it's someone that's trying to analyze some part of their business to understand something to improve. So an analyst might be on a retailer, like, how do I improve my conversion? How do I improve people going through my checkout funnel? It could be things focused on top of the funnel. It could be bottom of the funnel. Somewhere in a funnel, how do I get people to complete a specific task? And that's the job of an analyst. And so when we think about what the challenges are for a digital analyst, it's how do I crunch all these numbers? How do I interpret all this data? How do I solve for getting people through that funnel, but I can't spend the next six months? And if I think about it, sometimes they're micro-level issues and sometimes macro. So micro would be, you know, what did Joel do and why did Joel fill out a survey or call my call center or email or chat and say I have a problem? So how do I understand Joel quickly? Because it could be hours or days to understand Joel. And that scale problem comes into play. What if I have a million Joels today? I just can't solve for that. So that's how do I make this efficient? And then there's the macro. I don't have specifically one person saying this or that. I'm just looking at a funnel and seeing people not complete the task of buying something, for example. How do I make it better?

(Joel Beasley at 00:03:12) And so that's what an analog analyst will sit there and comb through the numbers, and that's what they're looking for. But when I say analog, I mean a human. So a human person that's doing the digital analyst stuff is looking through the analytics, trying to find meaningful things and ways to improve. Is that correct?

(Mario Ciabarra at 00:03:30) Well, no. I would say I would say you have analog and digital, which I don't think would be the words I would use. I think you have AI and non-AI. So digital versus non-digital is like a digital analyst is an analyst looking at a website or app. An AI empowered versus just a human would be how do we take and crunch this data using more automation to do the analysis? So think about a process of why are people falling out of this funnel? There's a bunch of manual work that we're going to use systems like Quantum to analyze, look at graphs and charts and reports, and try to figure it all out. How do we make that more automated? How do we make that easier for the human to understand that data? Because there's a lot of data being sent to these humans across these digital properties.

(Joel Beasley at 00:04:17) Do you have any specific examples of how the AI-powered analyst has uncovered insights that maybe would be missed by traditional methods?

(Mario Ciabarra at 00:04:25) Yeah. I think not just the word missed. I think the speed. And let me give you an example of what I see changing. Imagine that you go to an airline's website, and you're trying to change your flight, and you get an error. And you keep trying and you try and you try and it's just not working. What's Joel going to do? More likely than not, Joel's going to call in, and they're going to ask Joel to press one through nine on the IVR. And Joel presses four, and he gets an agent and, "Hi, Joel. How are you?" "Well, trying to change my flight." "Oh, Joel, you're trying to change your flight? Hold on. Let me transfer you." "Hi, Joel. How are you?" Now imagine instead that call looks like you don't press one through nine. You get routed to the expert that can handle your call, and they say, "Hi, Joel. It looks like you're trying to change your flight. Looks like you got an error in the app. Sorry about that. Could I help you with this or something else?" And the difference between those two experiences, if you think about it, every airline collects data about what's Joel doing on the website. The difference is how do they analyze that data in a short period of time, in one to two seconds, and be able to connect that into the agent and say, "Hey. This is Joel's problem. Here's a script that you can read to make Joel understand that we know this issue," and they can jump right to it versus the back and forth and the transfer and the pressing the one through nine. So it's really about how can I activate this as a new use case when I can understand it in seconds?

(Joel Beasley at 00:05:52) Is this theory, or do you have this out in the wild?

(Mario Ciabarra at 00:05:54) This is what's driving our success today. So I hope, Joel, both you and the listeners today, every time you press one through nine on your phone, you're like, "Where's Mario? Where's Quantum Metric? Why are they wasting my time?"

(Joel Beasley at 00:06:07) That happened this week. I had to call an attorney, and he had the best Google reviews. And, you know, business attorney, and I was like, "Hey. All right. Let's talk to this guy. He's the best one." And I call him. And it was the most 1989 archaic system in the world. Are you guys doing anything for small businesses or mostly enterprises?

(Mario Ciabarra at 00:06:26) It's mostly enterprises just because our motion today has been, "Hey. Look. I mean, it's the same effort to do this at a small company, a large company is the same, and there's a better outcome for our business. It makes more sense." We're constantly looking at, "Hey. We crack this." Because I think one of the things that makes it more viable for the large enterprise is the deployment. How much effort is this to get standing up and how much work is it to continue, on a day-to-day basis, to make sure that they're successful? And there's a good amount of work involved, which is why it makes more sense at the enterprise. There's also another problem I mentioned earlier around, you know, why we have analytics is the scale problem. If it's like if there's three people calling in, they should just take the phone call. Right? They shouldn't even have the IVR and make it go through. But if there's three million, that's when it makes sense. So there's a scale issue, and then there's an implementation and management issue. What we're working on today is how do we automate the deployment of what we do? How do we automate the understanding of what we do, not just in that call center use case, but, of course, we service product owners or engineering owners or operations owners or design owners or executives. How do we get them to see the value of what we do in seconds and all their different use cases? So that's what we're working on and how do we get our product deployed to give them those insights very quickly. So we're using generative AI to actually solve those exact problems, which is really exciting. This whole space has been, you know, I mentioned earlier, we made these T-shirts say we make it easy. We did make it easy in the beginning. And then we started solving more and more use cases, and I just look you in the eye and say, it became more complicated. And so we stopped making these T-shirts that we make it easy. And we've been joking about, "Wow. We should probably put more T-shirts on this. Hey. We make it easy again," because using generative AI, we're solving for the things that made it complicated and hard. But so we, I guess I'd say, we solved one use case, and it was easy. We made it easy to do that. But as we solved more use cases, it became hard. And I think with generative AI, we're going to collapse that back to, "Look. We're going to solve these large amount of use cases, but we have to do it in a way that's easy to our customer." So that's an exciting advancement. I think if we look ahead at 2025, how do we use generative AI to make analytics easy?

(Joel Beasley at 00:08:39) I want you to make it magical, man. I want the technology to be so advanced it's indistinguishable from magic.

(Mario Ciabarra at 00:08:45) I joke around with, obviously our company name is Quantum Metric. Sometimes you see these kind of visions of the future, and you're like, "Wow. This is like Quantum Magic."

(Joel Beasley at 00:08:54) It is. There you go. You guys could make some memes in your Slack channels or whatever.

(Mario Ciabarra at 00:08:58) I'm all in. I'm all in.

(Joel Beasley at 00:09:01) What makes a good generative AI strategy? I've got an email the other day that said one out of every eight transformation AI-type projects actually sees the light of day and succeeds. And I was like, "Wow. It's like a 90% failure rate." Is that true? Do you see a lot of people failing to achieve these way higher projects?

(Mario Ciabarra at 00:09:20) I think that in itself is a myth. I think it's way higher because I think, you know, any new technology where, "Wow, this could solve everything. And it's a panacea." And then we start to get, you know, I think Gartner calls it this trough of disillusionment, et cetera. There's this hype cycle. And what is it that we see working? You know, I think Sam Altman of OpenAI, as we all know, he knew early on. I was in a conference where he shared, I think the early successes, '24, '25, will be in call center technology. He knew it. And I think part of it is because it's the paradigm shift. If you're a technologist listening to this podcast, you probably can relate with this concept of, you know, deterministic outcomes. You know, if I give a program the same inputs, I'll end up with the same outputs. And that's not generative AI. We know this in our own personal consumption of generative AI. We ask the same question seven times. We get seven different answers. Sometimes they're pretty similar. Sometimes they could be on different ends of the spectrum. Sometimes a yes can be a no when we ask it a very direct question, and that doesn't make sense in our programming past. And the fundamentals of what we learned about technology doesn't make quite sense. And so I think there was this, "Wow, it can do anything" perspective, and so we can apply it anywhere. And I think we've all come to realize that's not true. Where can we apply it? And we saw this first wave of, I'll just use the words co-piloting because we realized the machine can't do it perfectly. So we're going to need a human in the loop, and a human plus the machine can do better. And so we're trying to figure out what are the applications where this drives real economic benefit in organizations? And so I've seen a lot of people say, you know, "Gen AI. Hey. We have Gen AI X or Y or we rebranded or we pivoted to generative AI." And most of those, I'd agree with a strong taste of skepticism because there's a lot of people saying they have something that does something with generative AI that doesn't really have economic benefit, and it works great in a single demo. And so I look at the bar to pass of, is this one of the successes? Does it work at a high consistency rate for the objective it was trying to do? Not did it work in the demo? Because people always show me good demos. You know, how do you go off and create this? I'll tell you, I have seen so much hype, you know, you as well. Blockchain, big data, even the word AI without the generative side of it. You know, there was real economic benefit that could come from any of these technologies. It was just hard to find. And I think generative AI is slightly different, but I'll call all those the hype of the year kind of projects that didn't really conflate to economic value for most businesses that looked at it. You know, we have Web3. We have tokens. We've got so many different things that people have put out that, you know, got hype and deflated quickly. I think generative AI is going to be different, and I didn't believe it upfront. One of our board members called me up and said, "What's your generative AI strategy?" And I said, you know what? It was John Chambers, the CEO of Cisco for 20-plus years. And I'm like, "John, I'm really good with product. This is my forte engineering. I don't really subscribe to the latest hype or the shiny objects. I don't have a Gen AI strategy." And it's if he had asked me three, four years ago, "What's your blockchain strategy?" I would probably just roll my eyes, you know, and say, "John, probably nothing." So, you know, we moved on in that conversation, but he ended up calling my CMO. He called my head of product, my head of engineering. And I think what really struck me is he called my CFO, all independent calls. And he said the same question. "What is Quantum Metric's generative AI strategy?" And they all said, "We don't really have one. Mario really doesn't want one." And when he called my CFO and asked it, I think something stopped. Why would you call the head of finance to ask him about generative AI? And I started to realize, what I lean on John for, he's a great human leader. He has incredible business acumen. But what I love about John the most, I would say, is that he's really good at understanding how to lead, how to create the right culture for an organization to succeed. So I lean on him for that more than anything else. So when he comes and tells you about technology ideas, I'm kind of listening, but kind of not. And the other part I realized that John has been very successful in his career. You might not know it, but he is affected with dyslexia. And it was something that he suffered, as strongly as it, you know, during his childhood more than, I would say, today because he's figured out a way to work around it. He figured out, most people go from one point to another in a straight line because that's the fastest path. He found a way to beat them on a curved line, which doesn't make any sense in physics, but he found a way to see things different than others. And I would start to say the words like seeing around corners, seeing around market transitions that other people aren't seeing because of this challenge or you might say gift. And I started realizing, why is he pushing me? Sorry. He never called me about blockchain. He never called me about these other hype cycles that happened.

(Mario Ciabarra at 00:14:50) Why is he calling and pushing so much on generative AI? And it was at that moment I realized, I think he sees a market transition here. I think he's seeing around the corner that I'm not seeing. And I said, maybe we should really explore this. So the path to creating the right Gen AI strategy for me was we came up with like 10 ideas.

(Mario Ciabarra at 00:15:07) And which 10 ideas was going to have our audience react? So I was at a conference, and I love conferences because it's a great testing bed because you get to meet so many people still in such a short period span of time that you can test a lot of ideas and see their reactions. So here I was at this conference, a number of senior level executives I got to interact with over a course of three days, and I was pitching them all a couple different ideas right around Gen AI. One of them, people were like, their eyebrows raised, and they said, wow. We could do that?

(Mario Ciabarra at 00:15:38) I was like, okay. No. Not yet. We're going to go back, and we're going to go attempt to do this. So we spent six months attempting to do it.

(Mario Ciabarra at 00:15:45) And there was all these early hints of success where we were like, wow, it's working. But back to that discussion I was saying earlier, it works once or it works twice. And my bar of the team is, does it work 19 out of 20 times? Does it work 95% plus? That's what we have to get to before I start feeling really good about this Gen AI solution.

(Mario Ciabarra at 00:16:06) And we kept iterating and iterating. I think one of the false veils of Gen AI, it's so easy. It's like putting two LEGO blocks together. You can get it to work so quickly, but you can't see, does it work consistently without you testing over and over in different datasets? And I think that's one of the things that, you know, if people walk away from this podcast thinking, it's not about getting the two Lego blocks together and you can get it to work.

(Mario Ciabarra at 00:16:31) Because linking an API into Gemini is super simple. We could be done by the end of the hour and you could say, wow, Mario works. But it's really about that patience for how do you test and determine its consistency? Does it really work the majority of the time? And I think that's the big leap on bringing a generative AI project to market.

(Mario Ciabarra at 00:16:53) Can you get it to work in all different sorts of circumstances for whatever project you're trying to solve for? So we had this idea. We tested the story of it working, and it resonated strongly with executive leaders. And then we iterated, iterated. And, you know, I think some of the takeaways that I would have had were finding the right dataset to get to the right answer in the most consistent way was probably the most secretive piece of our success.

(Mario Ciabarra at 00:17:20) It's really hard to figure out what goes in really changes what comes out. And I can go into more detail about it if you like, Joel. But I think how do you come up with a great Gen AI strategy is list 10 ideas of things that you probably could create and then go test the market. Do any of those 10 ideas resonate with your audience?

(Mario Ciabarra at 00:17:43) If it's none, come up with another 10. If one of those ideas resonates very strongly, and I like the words gets people to get out of their seat, that's the project I'd go work on.

(Joel Beasley at 00:17:56) Yeah. So you stole the scientific method.

(Mario Ciabarra at 00:17:58) Yeah. It's yeah. I mean, this none of this is actually super secretive. Like, oh my gosh. Mario has the key to success that no one else has.

(Mario Ciabarra at 00:18:08) I think, you know, I'll tell you in that same vein, you know, Quantum has become a multibillion dollar success. And probably the meanest thing I say to people is it's not hard to create a multibillion dollar success. I don't really think it is. I think it's just execution day in and day out. And I remain passionate, as passionate as I was 10 years ago.

(Mario Ciabarra at 00:18:27) And if you can stay passionate about what you do, every day is a joy. And it's not like there's no secret shortcut to get there. It's a hustle, and it's staying passionate. And you can create a great, you know, economic success, a successful business. But the same thing around generative AI.

(Mario Ciabarra at 00:18:46) It's just you gotta do the hustle. You gotta do the work. There's no he's not gonna get on this podcast and have this super secretive, like, oh, that's how you do it really with ease and then in two seconds. So it's just hard work, and you better love the hard work or else it's probably not gonna work out. You're gonna end up like the kids at the soccer game that wanna quit after the first game.

(Joel Beasley at 00:19:06) So I wanna make sure that we cover exactly what Quantum does, and then how brands can reach out to you to use your product or service.

(Mario Ciabarra at 00:19:17) Yeah. Quantum Metric is a digital experience platform, and those words start to confuse folks on what can analytics do for my organization. So I'd love to kind of just switch it into just a conversation of ways that the understanding of what a customer is doing on digital impacts our day-to-day lives. So one of the projects we've released in March is called Felix AI. It's a generative AI solution that can summarize what someone is doing on digital for different audiences.

(Mario Ciabarra at 00:19:48) What's really exciting me, Joel, is Felix AI. It's the world's first Gen AI digital analyst. And what that really translates into in easy understanding is imagine you're trying to understand is a customer happy or not or where's the, you know, what's causing friction on your website or app or kiosk or anywhere there's a digital touch point. Imagine instead of having to spend hours and days trying to track down the why, imagine generative AI just gives you the answer. It summarizes why is this person or this segment of people frustrated?

(Mario Ciabarra at 00:20:22) Why are they dropping off at the last step? What is the root cause of the issue? Without you having to dig through the data, it can do the analysis automatically. So I'm just super honored, super excited that we're able to find a generative AI use case that our customers have fallen in love with and to be the world's first Gen AI digital analyst with Felix AI.

(Joel Beasley at 00:20:40) That sounds amazing.

(Mario Ciabarra at 00:20:42) It's fun every day when your customers say how impactful you are in the organization. Selling software is I don't think that's that fun, you know, but selling impactful change in an organization. I was talking to the chief digital officer of another telco in the US, and he said to me, do you know what Quantum Metric is doing at our organization? And I said, I'd love to hear in your words. And he said, it's changing the culture of the organization.

(Mario Ciabarra at 00:21:07) This is a 150-year-old telco. And could you imagine the words changing culture? How can software change culture? And what he described to me was he has two direct reports. He's got business and product, and he's got technology.

(Mario Ciabarra at 00:21:22) And they come into the room in the past, and they would fight like, hey. Here's our marketing priority. Here's our product priority. Here's our design priority, and here's our tech priority, here's our DevOps priority. And he had to be the parent in the room to kind of negotiate which parties were going to win.

(Mario Ciabarra at 00:21:37) Now they come into the room with, here's our customer priority. And that was a massive change for that culture because, you know, it's easy to get siloed. It's easy to say, here's our party and our part of the organization. I like to ask executives, look, if I call every one of your direct reports into the room and I ask them, what's the company's top priority? Would they all say the same thing?

(Mario Ciabarra at 00:21:56) And they often look at me and laugh. Like, no. They're all centered in their own kind of universe, in their own persona. Like, I'm in product. I'm in DevOps.

(Mario Ciabarra at 00:22:05) I'm in voice of customer. But if you can make that shift to, let's put the customer at the heart of what we do, and we'll start to automatically get alignment on where we can have the biggest impact for our customer, which translates into the biggest impact for our business. That's the most rewarding part about the journey, everyone. It's a weird thing to say our mission statement is we help companies build cultures maniacally focused on winning the hearts of their customer. Now people, when you say you're a digital analytics platform, you say that mission, they'll look at you a little bit weird, Joel, because it's hard to connect how is analytics or data going to change my culture?

(Mario Ciabarra at 00:22:42) But we see it over and over again. We didn't come up with that mission statement because it sounded good. We came up with it because we kept asking customers, what do we do for you? And they kept saying, we're seeing cultural changes in our organization to align around the customer. It's been a fun journey.

(Joel Beasley at 00:22:59) Well, and you're able to track it, and you're able to see and share success stories. And when you put a fix in place, you can quantify it.

(Mario Ciabarra at 00:23:08) That sentence you just said, Joel, it seems so obvious. So many organizations will spend a week, a month, six months doing a release. They release it. Some don't even look at what the impact was. Some look at the impact and say, wow.

(Mario Ciabarra at 00:23:21) We didn't move the needle at all. Because when they go back to those planning stages, oh, well, we have a guess that this is going to be better. If we make this change but imagine instead of guessing you use data, you understood the why people were failing to complete some task you were hoping they would do with higher percentages and use data and you use the ability to see what they were doing and say, wow. They're getting stuck right there. Let's remove that friction point.

(Mario Ciabarra at 00:23:45) Let's optimize here. And it's amazing what data can do to align on we can have an impact.

(Joel Beasley at 00:23:53) So as far as people reaching out to you, what is the best way to go about getting more information from Quantum?

(Mario Ciabarra at 00:24:00) It's easy to go to our website, quantummetric.com, to read more about how Felix AI is having an impact in the enterprise. And it's really about aligning the entire organization on what does it look like from the customer. And instead of spending hours or these companies sometimes spend days with multiple people trying to understand what one person is doing, what Joel is doing, and why he's mad, why he left something on X or on Reddit or on TikTok or just filled out a survey and said, I'm upset. It sometimes takes hours a day. So do you imagine if you're a large organization, how many Joels there are?

(Mario Ciabarra at 00:24:37) How many people filling out surveys or getting feedback? And how can I understand the why? Now organizations do all of that today, but they spend so much time doing it. Imagine simplifying that using generative AI to take all of this data that we have about Joel and his experience and summarizing it. Joel is upset because he tried to apply a promo code, and it gave him an error.

(Mario Ciabarra at 00:25:01) And a fun example of that actually happening in the real world is one of the companies that we work with, they sent out promo codes. Huge hit to the website. People were really excited about getting 20% off. It was something like fall holidays was the name of the promo code, and it was working. People were applying it on the desktop.

(Mario Ciabarra at 00:25:19) They came to find out using Quantum. On mobile, you know what happens? You type fall holidays in a mobile text field. Do you know what happens? No.

(Mario Ciabarra at 00:25:27) It puts a space between fall and holidays. So they type fall holidays, but it ends up fall space holidays for autocorrect. They hit apply. It doesn't work. So this company, this brand started using Quantum.

(Mario Ciabarra at 00:25:40) They're like, oh my gosh. We sent out this marketing campaign. We're getting the traffic. It's not converting. And it gets, you know, a couple of layers deep to understand the why to use Felix AI and saying they're typing this promo code and it's not working.

(Mario Ciabarra at 00:25:52) And immediately, like, wow. It's got a space in it. So they went back and they added fall space holidays as a promo code, and they made millions of dollars of more revenue from that campaign. So it's understanding what's causing that friction.

(Joel Beasley at 00:26:05) I love it. Well, thank you so much for doing this, Mario. I really appreciate it. We made a podcast. How do you feel?

(Mario Ciabarra at 00:26:12) Well, you know, when people ask me, you know, how's my day going? How I feel? I only have one answer, Joel. Every day is a great day. So I feel phenomenal, literally just every day.

(Mario Ciabarra at 00:26:23) And as we talked about that point of optimism, it's not like you can't force yourself. You just have to believe and just intrinsic in who I am. And I just think have a positive outlook. It's you will manifest success by having a positive outlook. So how do I feel?

(Mario Ciabarra at 00:26:39) I feel phenomenal. Thank you, Joel. I appreciate it.

(Joel Beasley at 00:26:41) Thank you so much for listening. And if you found this episode useful, please share it with a friend or colleague who you think would get value from it. And if you have topics that you would like to hear discussed on the podcast, either add me on LinkedIn or send me an email [email protected]. Every time I get an email or LinkedIn message, it absolutely makes my day and inspires me to keep going.