Fact from Fiction on Mainframes with Vince Re, Founder & CTO at VirtualZ Computing

(Intro Narrator at 00:00:00) Today, we're talking to Vince, the founder and CTO at VirtualZ Computing, all about demystifying mainframes. If you're a fan of today's conversation, go check out Vince and his team's new podcast, Skyward Data. You're listening to Joel Beasley, Modern CTO.

(Joel Beasley at 00:00:23) Yeah, so I had a really fun, energetic morning. Then I was like, yeah, let's go do a podcast now. Yeah, keep the fun times going.

(Vince at 00:00:30) Well, you're having more fun than me this morning, so good for you.

(Joel Beasley at 00:00:33) Yeah. Now, you said you're a pilot. Are you currently flying?

(Vince at 00:00:38) Yeah. I'm a pilot and really small aircraft owner. It's one of my big loves in life. I've been flying since the 1980s. So yeah, I get a thrill out of that. My oldest son and our grandson live in the New York area. So for us to go visit him is, I don't know, 10 hours driving. It's close to that to take commercial airlines because everything's connections and all of that. But for me to jump in the airplane and get myself up there is generally three hours, maybe a little less sometimes.

(Joel Beasley at 00:01:20) It's a lot cooler too.

(Vince at 00:01:22) Yeah. Well, you get to come and go on your schedule. You get to, you know, you don't have to deal with LaGuardia or JFK. You pick whatever small airport you want to use. My airport here, I actually live almost walking distance to it, so it's only about a mile away. So it's literally climb in the car, drag all the stuff to the airport, load the plane, put the car in the hangar, get in the plane, and yeah, you're in the air 20 minutes later.

(Joel Beasley at 00:01:54) See the kids.

(Vince at 00:01:55) Yeah. Well, one grandson so far. But yeah, he just turned three. And yeah, so it's a wonderful age to at least feel like you can be engaged and all of that with them.

(Joel Beasley at 00:02:10) 100%. Yeah. I have a—my daughter just turned six this past week, and I had one of my first conversations with her, you know, in the sense that, I guess, more mature conversations with her, because we were talking about how she was reacting to certain situations and what she could work on—like a personal development conversation. She's like, "Yeah, Dad, I understand. I really can improve on that." And I was like, that is the coolest thing because, you know, all the conversations from age three to five or whatnot, they're just general everyday living conversations. But this was like a personal development conversation I had, and I told my wife after, I was like, this is the best ever.

(Vince at 00:02:52) Yep. Give it another five, ten years.

(Joel Beasley at 00:02:55) Oh, I don't want to. I can see it coming. Yeah. She's six going on 16. I can see it happening. I'm thoroughly scared. So it'll be an adventure of a lifetime. And my son's more predictable. He'll just punch me all the time. It's like living with a ninja. So, but we're going to talk about mainframes. Curious about what's the difference between a mainframe and a server. How are they different?

(Vince at 00:03:23) Yeah. Well, in today's day and age, I'm not sure there is a huge difference. I mean, the mainframe is a really, really big server. You know, and you make an interesting point too. I mean, mainframe computing is, you know, I think of it as one of the crown jewels in IT. A high percentage of the world's largest commercial sites, government sites, and so forth, they do a lot of processing on mainframes. And yet, you know, not a lot of folks know these systems in any kind of depth. I used to do guest lecturer stuff with one of the universities. And I remember walking into a roomful of grad students—and these are master's and PhD candidates in computer science—and right, so you would think experts. And one of the questions I would ask to start the class is, how many of you have ever seen or heard of mainframes? And, you know, I taught this class probably four or five times, and I never saw a hand go up when I asked who knew anything about mainframes. And yet, I don't know, 80% of the world's commercial data probably traces its way back to a mainframe one way or another.

So yeah, I mean, and the other thing there is it's a really interesting history lesson. I mean, today's mainframes trace their roots all the way back to the 1960s and work that IBM did back then. And, you know, it's almost hard for people to kind of wrap their heads around—is applications that were written, even 30, 40, sometimes 50 years ago, continue to work unchanged at the binary level on today's most modern mainframes. Now, certainly today's systems are vastly different in terms of technology. And, you know, mainframe has pretty much all the same componentry that you see in any other platform. You know, it can run Java, it can run web applications, it can have complex networks—you know, all the same things you find in any server are there on the mainframe. It's just that they tend to be kind of out of sight and out of mind that way.

And, you know, when you look back historically, you see that a lot of the things that have kind of come along over the years as breakthroughs in computing were done, like, decades earlier in the mainframe environment. I remember a funny story about that. I remember when Microsoft released the very first version of Windows NT, they were really bragging about its multitasking capabilities and the fact that you could get more than one processor core on it. Well, you know, those features existed in mainframes in the 1970s. So, you know, it's definitely a different approach. Definitely different operating systems. You know, most mainframes, the customers we speak to are running IBM z/OS operating system, which is kind of its own little ecosystem, very different than Linux or Windows or anything else out there. But yeah, really capable platform. And, you know, in terms of capacity, a large mainframe is easily hundreds of times more capacity than even a large server would be in the Windows or Linux environment. So, you know, some fundamental differences, but at the end of the day, they run a lot of very similar workloads.

(Joel Beasley at 00:07:25) Okay. Now, I'm going to ask more pointed questions because you've given me something to go off of here. I'm going to be relentless about my personal understanding of a mainframe.

(Vince at 00:07:36) Well, it's good. We can win you over.

(Joel Beasley at 00:07:37) You can win me over. I don't even know what they are, so let's hope I can figure it out first. Yeah. All right. So IBM—so if I have a—if I go buy boards and a processor and a drive, and I basically build my own computer, and I put IBM z/OS on there, is that now a mainframe?

(Vince at 00:07:58) No. Well, so first of all, you can't quite do that because, way down at the processor architecture level, the mainframe is a different beast than, you know—and it's not running on an Intel architecture or anything that's kind of off the shelf that way. It runs—yeah, it's really its own architecture, and so its own hardware instructions and so forth. And, you know, as far as I know, the only one who actually builds that hardware at the chip level is IBM. And the only place these things go is into, you know, the mainframes that they sell. So, you know, over the years, there have been competitors. There have been Hitachi. There have been, you know, companies like Amdahl who have built their own hardware that implemented the mainframe architecture. But generally, you know, you can't do what you described and just kind of cobble it together. You buy kind of a completed system, you know, typically from IBM these days. So, yeah. But other than that, you know, I think if there was a market and you could go buy a mainframe processor chip to plug into a motherboard someplace, then what you described is pretty close to the truth. The internal architecture of parts is different. There isn't today.

(Joel Beasley at 00:09:24) I mean—

(Vince at 00:09:25) No. No. There are a few, I'll call them emulators. Yeah. Basically, PC programs that can pretend to be a mainframe, and they create that kind of a virtualization of the mainframe environment, so you can run that in software. One of the popular ones is something called Hercules. So, you know, that's what a hobbyist would typically do. He would cobble together what you described—a server with whatever componentry you wanted. You would run something like Hercules, and now you have a mainframe that you can kind of put your hands on. It'd be very small in terms of capacity. You wouldn't have all of the mainframe features. And probably the biggest problem you would have with that is you can't just go to IBM and say, "I'd like to license a copy of z/OS and install that operating system on it." You know, they tie that software license to a piece of hardware. And certainly, they wouldn't sell you that to run on something like a Hercules emulated mainframe. But, you know, conceptually, you're in the right spot. It's just different internal architecture for all of this stuff.

(Joel Beasley at 00:10:40) Got it. So mainframe is not a brand, correct?

(Vince at 00:10:45) That is correct. I mean, it's—

(Joel Beasley at 00:10:46) So it's not a brand.

(Vince at 00:10:47) If you look in the movies, you know, often you'll hear people talk about such and such going on on, quote, "the mainframe." And then they'll show a picture of it, and you'll see all sorts of things, you know, big Unix systems, big mini computers, we used to call them. I mean, they're just kind of big and impressive because they have a lot of lights. They probably make it interesting to film. But that's not what we're talking here. Usually, when people talk about mainframes, they're talking about, you know, IBM brands it as System z. And there are dozens of different models that have been produced under that name over the past X years. So, you know, that's generally what we're talking about. And you would—you know, if we kind of wheeled one in for the podcast, and by the way, that's a great idea if you can get some artwork woven in—you know, people would probably love to see one of these things. They are not impressive. I mean, they look all the world like a commercial refrigerator or something like that. It's not like you would, you might think, you know, this panel of dials and lights and knobs. And I mean, that existed in the '70s. That hasn't existed probably in, yeah, at least 30 years now, though.

(Joel Beasley at 00:12:09) Yeah. I get most of my knowledge from movies. So those are the—I'm like, it's just the servers. I was like, they're just servers. It's just a normal rack. And I'm like, what is this mainframe? And so that's really helpful. Now, let me continue to make some assumptions and have you correct me here. So mainframe is not a brand. Mainframe would be a specific type of architecture?

(Vince at 00:12:38) Yeah. I would think of it as—you know, on a similar level to—I mean, you used the word server before. You know, when I say server, people tend to think of kind of rack mount things that are populated with Intel processors or AMD processors, but they, you know, run a certain kind of workload. Mainframe is kind of that where we're talking about IBM System z hardware and z/OS operating system and all of the things that kind of go with that.

(Joel Beasley at 00:13:10) Are there mainframes that are not IBM?

(Vince at 00:13:14) Well, so, you know, putting aside the software emulated mainframes that we talked about briefly, there were three or four other hardware vendors in this space. You know, Hitachi, Amdahl, you know, a few others. Fujitsu has a nearly compatible version that they sell mostly outside the U.S. So, you know, there are those kind of things that generally get lumped into that category as mainframe. There—yeah. In the 1990s, these, I'll call them alternate brands, were very—I won't say very popular, but they certainly had 20, 30, 40% of the overall mainframe market. Yeah. In the years since then, they've kind of dwindled. Yeah. IBM made a major leap in technology when they brought 64-bit processing to their mainframes. And at that point, it just got too expensive for a lot of the competitors to kind of stay in that space. So, you know, little by little, they've been fading away. I'm sure there's still some of those systems scattered around the world. We run into mainframe customers, especially outside the U.S., that have been running whatever they're running since the 1990s without really changing anything. So, you know, I'm sure there's still some of those out there. But, you know, the kind of mainstream mainframe would be, you know, what we describe—would be IBM's processors, operating systems, and, you know, all of that stuff.

(Joel Beasley at 00:15:07) And would workloads be able to shift between manufacturers? Like, if I had a Fujitsu one and then I—is that architecture identical to the IBM architecture? Is that, like, open architecture that they're aware of and they can recreate? And can the workloads be moved between them seamlessly or no?

(Vince at 00:15:27) Yeah. I mean, it—and that's basically what kind of spawned this competitive thing back in the 1980s. You used to see that, you know, IBM published very detailed specifications about the architecture. In fact, their operating system was one of the first examples of, you know, what today we call open source. You know, when I was kind of a youngster coming up in this space, I'd have a really deep question about how the operating system worked. I kind of walked down the hall, and we had this cabinet of microfiche. And on that microfiche—and, you know, people probably don't even know what microfiche is anymore—but, you know, it was basically the source code to all of the operating system. So, you know, it was there. And that meant if you were a competitor, you had kind of a pretty clear path to understanding how all of this stuff worked and, you know, all the fine print of the architecture.

Yeah. At some point, I think IBM felt that, you know—and I would feel the same way if I were a shareholder of IBM—you know, that that's a lot of intellectual property that needs to be protected. So little by little, it became, you know, trademarked and patented and so forth. And now, probably all that same information exists. It's just that first you have to go license it from IBM. And of course, if you're looking to build a competing mainframe, that's going to be a really, really expensive thing to license from them, or they might just, you know, refuse to do it outright. So, you know, the foundations are there. Yeah. You can understand that mainframe architecture in an incredible amount of detail—much more. Yeah. I'm one of the rare people. I've developed commercial software on mainframes, on Windows, on Linux, on, you know, lots of different platforms. And I have to say, it's not even close. If I want to know some detail about how the mainframe works, it's much better documented. There are better tools to help you understand it. You know, all of that. I can understand kind of the internal workings of the mainframe way better than I can understand the internal workings of, yeah, let's say a Windows server. So it's—yeah, it created an environment where it was easy to compete because of all of this stuff. And that's what we, that's what we, you know, saw, you know, back in the '80s. And, you know, the value prop for a lot of these companies was that, you know, they would give you 20% better performance for 20% lower cost.

(Vince at 00:18:18) So they were able to lure a lot of business away from IBM that way. And the only way they could do it, to kind of circle back to your original question, the only way they could really do it was to show that they were absolutely 100% compatible. You could run a particular program on the IBM hardware, take that exact program, run it on the Brand X hardware, and it would process exactly the same way. Maybe a little bit faster, maybe the cost would be a little lower, but the results would be the same. So yeah, that's an important part.

(Joel Beasley at 00:18:59) Do you believe that applications within the ecosystem were written with more stability and better performance because of the access to the source code of the operating system?

(Vince at 00:19:11) I mean, in some cases, yes. You know, it's a lot of things, though. I mean, in the earlier days of mainframe computing, it wasn't uncommon for folks to write even large applications in assembler language, you know, the low level language that the hardware understands, where the programmer is writing one for one. Here's a hardware instruction, one after the next after the next. You know, in the big software vendors, that's still how the world is.

(Vince at 00:19:47) And I compare the efficiency of that to, you know, let's say writing a Java application that runs on the mainframe. It's night and day. I mean, a well-implemented assembler language routine running on modern mainframe hardware is going to be just light years faster than trying to do that same algorithm in Java or C or C++ or whatever other language you have. So I think in those days, I mean, it's important to understand, a lot of those earlier mainframes, we measured disk space in megabytes, not terabytes. And memory was, let's see, not terabytes.

(Vince at 00:20:33) So when the business required, I need this particular function, and it's got to fit on a 4K page or whatever it was, your options were pretty limited. You didn't really see a lot of high level language use in the earliest days of the mainframe. That came with time. And, you know, like I said, today all of that exists pretty much the same on mainframe as other platforms. It's just much more of that legacy code is still the more efficient stuff versus the new applications that are showing up there.

(Vince at 00:21:15) You also had a really interesting thing there too. You know, one of the ways I kind of started out was I worked on what became one of the first commercially viable cybersecurity products for the mainframe. And we needed to do very low level things inside the operating system. So, you know, as an example, when you opened a file, we needed to insert some logic there that said, are you allowed to open this file? And all of this.

(Vince at 00:21:50) Yeah, exactly. So being able to understand kind of the little gory details of how the mainframe operating system was working was a huge enabler to that. It would have taken a lot of trial and error work to figure that out without at least a little bit of insight into the structure you could get from the source code and so forth. And, you know, honestly, in my mind, that's where the software industry comes from. You know, before the period of time that I'm talking about, all of the software came with the computer you bought. You know, so you went to IBM, you bought one of their mainframes, and it came with all the compilers. It came with pretty much everything you needed to run your environment.

(Vince at 00:22:44) Yeah, there were a few small upstart companies after that who had the idea that, well, if I did a better, you know, fill in the blank cybersecurity product, people would pay me money just for that. And that created the software industry. You know, this was well before the PC came along, and it continues today. It's an important development in how the industry thought about what's the role of hardware and software, and how does it get packaged together, and what vendors can do what, and all of that stuff.

(Vince at 00:23:22) So yeah, interesting archaeology exercise.

(Joel Beasley at 00:23:28) Yeah. So the architecture of the chips, that is one key differentiator between just building my own compute and storage and then buying or owning an actual, quote unquote, mainframe. Capacity, you've mentioned a couple times too. So that's one big difference. The architecture is a huge difference, but you've mentioned capacity is a big difference.

(Joel Beasley at 00:23:53) Can you explain to me how the capacity of the mainframes are different than the capacity of, let's say, something I could flip on at AWS?

(Vince at 00:24:01) Yeah, I mean, so mainframes are a little bit different in how they're sized, and what their capacity limits are, and so forth. One of the really key differences is that, you know, for that architecture we're talking about, I might have one physical cabinet that's got a bunch of processing and memory installed into it. And, you know, unlike most Intel systems, a lot of that is dynamically reconfigurable. You know, sometimes based on a command that the IT operation folks enter into the system.

(Vince at 00:24:44) So it's not a fixed capacity, in other words. It's a varying capacity based on what your workload requires. And, you know, the big underlying reason for that is all about cost. You know, it's not that there is—today we look at Intel processors and, you know, whatever the hot CPU is today, it's got a certain price and a certain set of capabilities. And, you know, we recognize that, everybody understands that.

(Vince at 00:25:16) Mainframe isn't like that. You know, there might be three or four different models. But within that, there might be 100 different capacity settings, depending on how big your workload is. So if you're, I don't know, a big retailer, you might spend most of the year with that capacity dialed back, so your costs are pretty low. And then maybe in that period from Thanksgiving to the end of the year, you dial it up, and, you know, without any physical change in the box, now you might have double, triple, 10 times more processing capability to handle the business transactions that you get then.

(Vince at 00:25:56) So, you know, and of course, if you're a large company like IBM, you create a whole marketing structure around that. So it's sold that way to the customer. It's very dynamic to kind of meet his needs. And yeah, that's one of the real key differences is that you can very easily size the hardware to the workload that you've got. And even if your workload varies all over the place, you can keep up with that pretty easily without continuously doing hardware upgrades.

(Vince at 00:26:33) The other thing too is just the way we think about mainframes. Things like fault tolerance are really important. You know, I want to be able to have, you know, almost—you would think of it as clustering on other platforms. Yeah, I want to have multiple mainframes that connect together, work together as though they're one system. So that's kind of another way I can scale these systems. I can have one standalone mainframe.

(Vince at 00:27:08) That's probably fine for a lot of smaller clients. I could have 32 mainframes in a single cluster, and to external users connecting into these systems, they see that as one system image. You know, they see it as one cooperative thing that shares resources. So yeah, you have that kind of scaling as well.

(Vince at 00:27:33) And then, yeah, of course, virtualization, you know, just like you have in the AWS cloud is possible on the mainframe. So that one mainframe device, you know, that one System Z server, the physical thing, can be carved up into lots of virtual servers. You know, so you might run, I don't know, a production workload and a test workload and a development workload and, yeah, whatever your needs are that way. And it all adds up to just a different way to think about the virtualization that way.

(Joel Beasley at 00:28:11) So there's not a large difference in capacity, how we look at capacity between the two systems.

(Vince at 00:28:17) Yeah, I mean—

(Joel Beasley at 00:28:17) It's mostly architecture.

(Vince at 00:28:19) Look, when you're dealing with small numbers of systems, of course, you have to have massive capacity. So, you know, an example of that is I can have 96 100-gigabyte network adapters on a single mainframe. You don't see that on Intel-based servers. You know, but on the Intel server, you might have, I don't know, a thousand rack-mount servers.

(Vince at 00:28:48) And, you know, in aggregate, it's—you know, you can certainly build the capacity that way. Much harder to do off of the mainframe, right? Because you have to design your software to really take advantage of that type of scale. Mainframe tends to be way easier to do that stuff. Yeah.

(Vince at 00:29:09) Just, you know, we used to struggle with this quite a bit on the software vendor side. Yeah, how do you do even things like pricing that are fair to everybody? You know, some users might scale it down really small, you know, equal to just a couple of servers. Some might scale it up to the equivalent of a million servers.

(Vince at 00:29:31) You can't charge the same price for that. So, you know, it creates complexity. Is it the same amount of hardware, though? I mean—no, it would not be the same amount of hardware. It would be one box or, you know, a handful of boxes, physical things.

(Vince at 00:29:48) Versus racks and racks and racks and racks of Intel servers.

(Joel Beasley at 00:29:53) But so just charging for dynamic usage was what you guys were trying to figure out then.

(Vince at 00:29:58) Yeah, I mean, a lot of software in the server world is sold per device. And so every server I have, I charge a thousand dollars. When your mainframe can go from, you know, one X to 800 X inside the same physical footprint, it's hard to do it that way. You have to have some other metric to either not give something away or not have to overcharge so much that the small customers won't accept that.

(Vince at 00:30:33) So yes.

(Joel Beasley at 00:30:34) Were these small customers connecting into mainframe clusters then, and it was like multi-tenant? Is that what you're describing?

(Vince at 00:30:40) I mean, it certainly could be then. I mean, there were all sorts of things. I mean, we saw, you know, on the small side, you would see things like universities or smaller companies that had mainframe applications, maybe for legacy reasons that they just needed to keep running. You know, they might just need a small amount of mainframe capacity. That's really different than, I don't know, a Citigroup or, you know, someone like that who's doing millions of transactions a second on their mainframes.

(Vince at 00:31:12) So yeah, it really just depends what your workload is all about.

(Joel Beasley at 00:31:16) So how you design your strategy for capacity is a key differentiator between mainframe and not mainframe?

(Vince at 00:31:23) Yeah. No, absolutely. And, you know, in this day and age, a lot of that really just comes down to what's the best way to achieve low cost per transaction. You know?

(Vince at 00:31:34) And in fairness, mainframe is such that if you don't have a big problem to solve, it's probably going to be too expensive for you. You know, if you have a handful of users and a handful of transactions, mainframe is going to be really, really expensive. On the other hand, if you have massive workloads, you know, maybe you're an airline doing a reservation system or a big banking system or something like that, where you're processing a gazillion transactions, the cost per transaction on the mainframe is going to be much less than most other platforms would be.

(Joel Beasley at 00:32:08) Because of its efficiencies.

(Vince at 00:32:10) I mean, that's a core part of it, but it's also things like it's always easier to manage one big thing than lots and lots of small things. You know, the one big thing, you put some policies in place and it kind of takes care of itself. The large collection of small things is complicated. You know, you have a lot of interdependencies, things that can go wrong there that just can't go wrong on the mainframe because you only have one of them or a small number of them. So, you know, that has the effect of driving up your management costs.

(Vince at 00:32:46) And then you think about, well, how am I going to do fault tolerance? How am I going to react to spikes in workloads and, you know, all of that kind of stuff? And, you know, generally, those kind of things are just kind of built in to the mainframe architecture. There are absolutely ways to do it on other platforms. It just—they tend to work out a little bit more costly on a per transaction basis.

(Joel Beasley at 00:33:13) Okay. So on the mainframe, if I exceed my—like, let's say I'm an airline, and I have to purchase for the greatest capacity I'll ever experience at the highest point in the season, correct, as far as number of mainframes?

(Vince at 00:33:26) Yeah, I mean, I guess the best practice there would be that, you know, you would sit down and do—yeah, I mean, this isn't something you're doing every day. So you want to make sure you're correct. And, yeah, there are designated experts in a lot of sites that are doing this kind of work.

(Vince at 00:33:47) And what they'll do is they'll project out over, you know, whatever their buying period might be. Often it's a three-year kind of a time horizon. And they'll say, okay, over that, we think here's where our peak workload is going to be, you know, based on business conditions and all of that. And then they'll look at what's the cheapest way to do that. Is it, you know, multiple data centers in one big data center?

(Vince at 00:34:14) How many mainframes? You know, there'll always be a sweet spot in terms of the costs to handling that workload. And that's what they'll put in place. They'll go contract with IBM and all other software vendors and all of that to put a price in place. And then they kind of just execute against that plan for, you know, however long it's in place.

(Vince at 00:34:40) Now, certainly surprises happen. You know, if you're that big airline and there's a sudden spike in travel activity or, you know, whatever it is, regulatory change, whatever it is, and you need—you realize you're going to be 20% short on capacity. Yeah, I mean, you deal with that kind of as you go. But, you know, the important thing is you tend to see that coming.

(Vince at 00:35:04) And there are really good tools in this space on the mainframe that help the capacity planner understand what the trends are, you know, and if there is something unexpected. Is it just because you implemented some inefficient software that could be done better? Or are you truly handling more transactions than you expected to handle? So, you know, again, it's an iterative thing, and you have many options. You might sit down and say, you know, maybe we shouldn't have rewritten that older COBOL program in Java because now it's taking too much processing power.

(Vince at 00:35:43) But, yeah, if you go through all of those options, the most direct thing to do is add capacity. It's generally a matter of just paying for that and activating it. And, again, mainframes are so dynamic. There's no changes generally for that. You know, you have the good folks at IBM activate that extra capacity inside your box.

(Vince at 00:36:11) You kind of wander over to the machine, push a few keystrokes, and now your system got bigger. So yeah, totally non-disruptive. It's not like you even have to reboot the systems in any way in a lot of cases like that.

(Joel Beasley at 00:36:27) It's very cloud-like then.

(Vince at 00:36:29) Yes.

(Joel Beasley at 00:36:29) So then me being on the outside of this and I've written business logic, Ruby on Rails, PHP type code for the majority of my software career. And so that's, you know, my experience. So when I'm looking at it and having a conversation with you as an expert who has been eating, sleeping, and breathing this stuff for thirty years, right? And I'm trying to understand the differences between the environment I grew up in and what people are calling this mainframe.

(Joel Beasley at 00:36:56) The key difference that I'm gonna walk away with, and so you're gonna have to correct me before I do walk away, but the key difference I'm gonna walk away with is that because of the architecture that is implemented on mainframes, people write applications that are more efficient, which at scale will create a better cost per transaction. So these large companies use these mainframes because it's just the most cost efficient way to do things.

(Vince at 00:37:26) Yeah. I mean, that's largely true. I mean, I would not discount the fact that a lot of it is just facts on the ground. You know, if you're, you know, a big insurance company, you have a portfolio of thousands of mainframe applications. And the cost of redoing them to run that workload somewhere else is, you know, there's just no business value in that.

(Vince at 00:37:48) So you just kind of keep going with what you have that way. So, um, I mean, if you put yourself in the shoes of the CIO at a lot of these sites, he's got something that, you know, it's running, it's been in place forever, it's, you know, meeting their business needs. I mean, he's not gonna dismantle that just to say, you know, it's in the AWS cloud or something unless he has, you know, some important reason for that. And, you know, I'm not saying that doesn't exist.

(Vince at 00:38:18) He might have a good reason for doing that. He might, you know, he might look out over his mainframe staff and they're all kind of my age, you know, early sixties. And, you know, he might look at that and say, you know, in five years, who's gonna do this work? And, you know, that might be the motivation to take him to, you know, some other platform. But, you know, just doing it for—

(Joel Beasley at 00:38:42) the sake—

(Vince at 00:38:43) of doing it? No.

(Joel Beasley at 00:38:44) As the people retire, there'll be a smaller number of people that know this technology, and then they'll become higher in demand, and that cost will raise to a point where it might make sense just out of bandwidth of being able to make changes to your system to move the systems.

(Vince at 00:38:59) Yeah. I mean, you would think that, and, you know, it's funny you make that observation. I actually wrote a big research paper on this many years ago, because it was one of the things we were very concerned about, you know, where would the skills come from in the future. You know, it's not like most American universities have programs in mainframe computing anymore. And in fact, even the professors, if they wanted that, they couldn't teach it today.

(Vince at 00:39:28) You know, there just aren't folks who, uh, who know this space in that, you know, academic world anymore. So, um, and, you know, I don't mean that as a wholesale statement. I mean, I don't wanna get comments from some of the—

(Joel Beasley at 00:39:44) We're gonna bill it as a wholesale statement. That is the official view.

(Vince at 00:39:48) Yeah. But but, you know, the point is it's, um, you know, it's uncommon. It's a source of concern. But on the other hand, if you look at it, what you said is very, uh, is a very good observation. You know, you would expect that the laws of supply and demand would step in in that case.

(Vince at 00:40:08) And as folks kind of hit that retirement age and the pool of folks with mainframe talent shrinks, you would expect wages to be going through the roof. You would expect, you know, all sorts of things to happen. And you know, just to go past that, it would eventually be self-correcting. You know, people would look at it as, you know, I've got my computer science degree. I can earn X as, you know, a web developer somewhere, or I can earn two X by going and doing mainframe things.

(Vince at 00:40:37) And so eventually that would tend to balance out. I can tell you—

(Joel Beasley at 00:40:42) Proceed.

(Vince at 00:40:42) That's not happening. You know, if I go back and I track inflation-adjusted wages for mainframe developers from the year 2000 to now, it's less growth than you see in most other areas. A PHP developer, for instance, their wages have inflated more in the last twenty years than a mainframe developer has. Yeah. So, yeah, I think what that really comes down to is this is a trend everywhere.

(Vince at 00:41:15) And there actually is more demand for, you know, all of the other skills we're talking about versus mainframe skills. You know, another thing, um, that you see in the mainframe community is, you know, for all the reasons we're talking about, you tend to be able to run a large mainframe site with less staff than an equivalent amount of capacity in a, you know, non-mainframe site. So, you know, there are, you know, an example, a customer that I used to be pretty actively engaged with. You know, when I looked at his mainframe team, the whole thing was about, you know, twenty, twenty-five people. The non-mainframe side of his business was closer to 500 people.

(Vince at 00:42:04) So the truth seems to be it's, you know, people are finding ways to bring new skills in because it's not like you need hundreds of them. You only need, you know, a relative handful of them by comparison. And I think that's what's helping keep the kind of skill balance, uh, alive and really across the industry. Uh, you know, we saw it, um, you know, I spent many years at CA, and we had this fear, um, you know, quite a bit, you know, where would our future stars come from? You know, the folks with really deep mainframe technical skills.

(Vince at 00:42:43) And, uh, yeah, we did what any business would do. You know, we invested, we created, um, a couple of new programs, new development centers, we staffed them with, you know, younger folks. And we made the conscious business decision that we were gonna spend maybe three to five years training them and grow the skill that we needed organically. And, you know, that would serve our needs for, you know, the foreseeable future. I think a lot of the larger mainframe sites are, uh, kind of going that way.

(Vince at 00:43:18) Now, if you're a medium-sized or small site, you know, maybe you don't have the option to do that. And maybe you are really concerned about the skill issues here. So, you know, it's not really a one-size-fits-all answer. But it does seem that, uh, what you would expect as kind of the economics of supply and demand for mainframe developers isn't really behaving the way you would think it does in, uh, in the, you know, at least in the first cut.

(Joel Beasley at 00:43:47) Yeah. I'd say I, look, I don't have a dog in this race, but I would think that there, I agree with all the points you made. I don't think any of them are wrong. I'm interested, here's how I can say this. I'm interested to see what happens in the next twenty years with that wage growth, largely because there is a component to a lot of these people starting their career and then living their career out for thirty years.

(Joel Beasley at 00:44:14) And the dynamics of just what's happened in our country with the finances, uh, as far as, you know, buying homes at a lower cost per total salary, living in those homes, a lot of, there's just a different mindset of people in their fifties and sixties than there are today of this generation. And so I think that if we haven't, from your research, if we haven't seen a large growth change that we're maybe a decade or two away from seeing it because then you get more people retiring and then you're just gonna have that need. Uh, I think, you know, I've had one conversation with, uh, a while back ago. I'm pretty sure we didn't air it because of some reason or another, but it was a major public utility company in a state which will go unnamed. And they had a problem with their systems, and they looked around and they realized that they knew nobody in the industry that understood how these programs were written.

(Joel Beasley at 00:45:18) And that they were down and they just had this huge, huge issue. So they started calling all these retired guys and pulling them in as consultants trying to get them to help. And, uh, and so I thought that that was fascinating that that can actually, that can actually happen. But, um, yeah. The worst case we can do is, you know, I see a dude, they can hire a bunch of people. We could also go to the retirement homes. You, we can start a little business together, my friend, Vince. So we can go to the retirement homes, and we can train ChatGPT. We can Neuralink up the older people, have it train ChatGPT, and then now we can have one person run the entire mainframe.

(Vince at 00:45:57) Yeah. Well, you know, the other thing that's, um, important here is the skill level is different today than in the past. Uh, you know, when I was kinda coming up in this space, yeah, no matter what your role was, you generally had kinda, you know, today, we would think of them as very deep skills. You know, you knew how to write, uh, you know, assembler language, you know, the language of the machine. You knew, you know, not just how to administer things, but how things actually worked underneath and how to take advantage of that.

(Vince at 00:46:34) You know, over the years, and it's, you know, it's not really just IBM. I'll give them a lot of the credit, but a lot of the software vendors too have just automated so much of that out of these systems that the need is different today. You know, today, it's more about how do I administer it rather than how does it work? And, uh, you know, because of that, you know, I think that also has been a big help in the skill issue that we're talking about. You know, plus the other thing too is that, you know, one of the things that helps on other platforms is, you know, the whole offshoring idea.

(Vince at 00:47:12) You know, the fact that I could have, you know, 200 developers tomorrow sitting in, you know, in some country like India, uh, all I have to do is pay for it. I can't do that very easily with mainframe developers. You know, and, you know, for whatever reason, those skills just never, you know, blossomed in a lot of the places where you find low-cost offshore talent. So, you know, that just creates a different dynamic than what we've seen in, uh, other places. You know, I want a website built, you know, with the technologies you mentioned, PHP, and, you know, MySQL, and, you know, all of that.

(Vince at 00:47:52) I mean, that, you know, I could do that anywhere. I can even crowdsource that. I don't have to do it, uh, you know, in a formal kind of a way and have employees for that. Can't really have most of those options with, uh, mainframe applications, even though a lot of those same technologies exist in the z/OS operating system, and they sort of work the same way. It's just not, uh, the common environment that, you know, people have access to or would feel comfortable with.

(Vince at 00:48:25) Um, yeah, funny, uh, example of that. One of the projects we worked on involved a little bit of open source. And, you know, we looked at the open source and it didn't support the mainframe operating systems. So we decided we would do that. You know, we took a snapshot of the source code.

(Vince at 00:48:42) We enhanced it so it would run fine on z/OS. And, you know, we were pretty happy with it. And, you know, at the end of the project, we figured we would just turn what we had done back over to the, uh, original developer, and he could, you know, carry it forward as new releases came along and so forth. He refused to take it from us. And his rationale was, I don't have a mainframe.

(Vince at 00:49:07) I don't have any way to test that in the future. I don't have any way to know what works and what doesn't work. So, yeah, that that kind of work sometimes gets orphaned that way because you just don't find, you know, a broad skill base there the way you do on, uh, some other platforms.

(Joel Beasley at 00:49:26) Well, I'm watching time. I wanna make sure that we get a little call to action in for what you guys are doing over at your company, VirtualZ. Can you just give me the thirty-second why VirtualZ exists, what problem you solve for the mainframe?

(Vince at 00:49:39) Yeah. I mean, uh, VirtualZ, um, is all about making it easy to, you know, share that data that's on your mainframe with other platforms. Uh, no matter what you're doing with the mainframe, the reality is, you know, today, every business has mainframe and other things. And, you know, one of the critical needs there is how do I get data out of my mainframe and into, you know, wherever I need it? You know, it could be an AI tool.

(Vince at 00:50:08) It could be, you know, God knows what. And historically, that's been a hard thing to do. You know, it's been, you know, file transfers and FTP and all of that kind of stuff. Well, what we do in our products is, uh, and really there are two key products here. The first product is about leave the data on the mainframe, but make it easy to make it accessible from anywhere you want.

(Vince at 00:50:33) So, you know, we make all of that mainframe data look like a big network drive in the sky, and you can consume it from just about any application. Number two is kind of the opposite of that. It's, you know, I got the application on the mainframe, and it needs some data that I'm producing in a cloud application, or in, you know, whatever it is. It's not on my mainframe. So, yeah, I want my existing mainframe application that was maybe written thirty years ago to be able to read and write data in, you know, let's say the Amazon cloud or Azure or Google or, you know, wherever it happens to be.

(Vince at 00:51:08) So that's kind of the problem that we solve for customers. And, you know, and it's a funny thing. We know how to do this for, you know, a decade now on other platforms. You know, if I'm running a Windows Server and I need a couple of terabytes of storage, yeah, I don't even think twice about that. I go to Amazon, I go to Dropbox, I go, you know, wherever I want to get that storage, and I use it.

(Vince at 00:51:31) You know, and that's not a particularly novel thing to do anymore. Mainframes have never really participated in that. And that's what we enable today. You know, we enable, um, any mainframe customer to leverage low-cost cloud storage, um, you know, share information across, you know, whether the application is a mainframe application using cloud data or a cloud application that needs mainframe data. You know, we make it just drop-dead simple to do all of that.

(Vince at 00:52:00) Saves the customer a huge amount of time and money having to figure out how to get data from A to B. Uh, you know, we guarantee that it's all done in a secure way and with a lot of integrity so that, you know, you don't break other applications that might be referencing that same data. And, um, yeah. And it feels like, um, it feels like an important requirement that mainframe customers really struggle with today.

(Joel Beasley at 00:52:29) Nice. And that's VirtualZ. We can put a link in the show notes. And, uh, man, that's it. We did it.

(Vince at 00:52:36) Well, thanks guys. It's been, uh, been a pleasure. Not nearly as painful as it could have been despite the AV issues.

(Joel Beasley at 00:52:44) Thank you so much for listening. And if you found this episode useful, please share it with a friend or colleague who you think would get value from it. And if you have topics that you'd like to hear discussed on the podcast, either add me on LinkedIn or send me an email, [email protected]. Every time I get an email or LinkedIn message, it absolutely makes my day and inspires me to keep going.