00:00
I'm gonna start us off on time everybody. Thank you so much for coming and joining us today. I'm Calvin from pure storage and we're here to talk to you about how Domino's serves competitive advantage and customer delight with A I and analytics. And joining me today is Bill Hallman Junior. It's me from Domino's uh Bill.
00:22
Why don't you tell us a little bit about yourself? I work at Domino's Pizza. I manage the storage, virtualization and compute teams. We manage all file block and object storage along with bare metal server VM Ware and data protection uh for Domino's. And, and what makes like what does Domino's stand for?
00:40
What, what, what makes you tick when you, you come in and you're working for them? Well, every day is a different, every day is different, right? There's always new challenges. I mean, you dove tail that into the talk that we're about to have and there's always new expectations, new demands, new requirements. And I think that, um you know,
00:58
what keeps me and my team going is that it's never dull. I'm sure that a lot of people in the audience also have that those experiences as well. Um, but the good news is, is that if you're at this conference, then pure can be an excellent tool for you to, uh, combat some of those, uh, less than specific requirements for a high end ML workloads.
01:19
And I think everybody is well aware of the brand Domino's Pizza, but they might not know. Let me see if I have this right. 20,000 stores, 90 different countries, 3 million pizzas delivered a day in that realm. Give give or take. I mean, I think it's a little bit more now,
01:39
but ultimately that is, you know, tied to two store truck deliveries every day to each of those stores for fresh ingredients and all of the necessities that you need to run a store and all of those, all of those pieces needing to be kind of crunched down, analyzed and understood by people to make decisions, uh, to keep the business going forward and continuing to acquire more market share.
02:03
And, and the thing that you guys are well known for in, in our industry is the use of technology and how you run your business, how your franchisees or franchisers, um, make decisions about everything that they do and we're gonna get into a bunch of that today. Yeah. Awesome. Well, A I is in the news every single day,
02:26
right? It's when we first talked about this session. These were some of the headlines from back then, but just if you even pick up your phone and just look at today's news. It's, it's there and you're obviously here interested in learning more about that as last year. Chat GP T and generative A I kind of came out just really bringing it to the consciousness of
02:47
everybody from your kids to the CEO S of, of your companies. Um, and Mackenzie did a study where from 2017 to 2022 the adoption of A I technology has grown by 2.5 times and it continues to grow um very, very aggressively. The other thing about that as well is that companies that are very mature in their use of
03:13
analytics and A I like Domino's, they're pulling away from their competitors. They're using it as a tool to accelerate operational excellence, more revenue per employee, uh better their customer satisfaction by 2.5 times and be more innovative deliver product 46% faster. These are uh data points that came out of an ESG study on this.
03:39
But you guys have used A I for years now, tell us more about kind of that, that secret sauce that you're using um to run. I can't give you all the secrets, but I think what it was, it was recognized early on that like business intelligence was a great first start. Um And we were getting a lot of insights out of just the general uh batched reporting that we
04:01
were getting from end of day sales and that sort of stuff. And so that propelled the business to what it was. Um prior to uh the 20 tens, we'll say right after that, um these new fangled ideas of having machines kind of comb through this data faster than we could put the people looking at the Excel spreadsheets and tying all of the data together
04:23
to make decisions um would work. So, I mean, we've uh in 2015, we started putting together this uh analytics and insights group which started with basic tools, um Python scripts and coding, building pipelines through containerization of micro services to be able to analyze this stuff quicker, faster and more flexibly.
04:49
Um Some of the some of the reasons for that is because you could have rooms of people to kind of work through all of these problems. But if you understand kind of what you're looking for, you can have the computers come up with those decisions quicker and then you can then ask it more complex questions in terms of, well, what's next? And I, I think that that's really what Domino's was stretching for is,
05:12
well, what's next? We know how many pizzas we sell, we know when we sell them, we know how much we're selling them for. We understand what the food cost is. We get, we have a complete understanding of the product. But as it turns to how do you, once you become the world, the world's number one pizza company.
05:28
Like, how do you, how do you continue to gain more market share? And that's really kind of what propelled this team from in 2015 of 15 data scientists, um to today over 60/60 right? And so if you go to Domino's dot com as I did and you look at kind of the job section, you guys actually have an analytics and A I like it's front and center is part of kind of what you guys do,
05:55
but you're, you're not part of that team. No, I'm not, I'm not smart enough to be part of that team. They are very uh yeah, but I don't have a phd and I don't, I didn't go to MIT I mean, we were interviewing people from Harvard like uh uh to try and figure out how do we come up with the, the quantum mechanics of this whole thing to put
06:13
all of the math together to get these uh things figured out. Um I'm part of the infrastructure group. So we're, we're like the plumbers of, of the A N team, right? We are the people that are putting the technology underpinnings together for them to leverage just like you would a faucet or, or drinking fountain,
06:30
right? Um Or now it's a bottle refilling station. Those are, those are more popular. But I think that um you know, our job is to make sure that the data is where it needs to be on time every time. So that the models that they're training or the models that they're using or the deep learning cycles that they're using, um, to analyze all of this stuff is there consistently without
06:50
failure and as fast as it possibly can be. So, if you got a team of 60 Harvard mit, like that's pretty pricey. So you, you have to make sure that what they're running on, they're not waiting on. Absolutely. That's kind of your, your deal and your, your team's size is 44 people.
07:11
Wow, that's amazing. Um And so some of the challenges in kind of quick restaurant services that you've talked about to us include managing obviously your inventory, your food costs, supply chain management, getting, getting the, the trucks to all of your franchise uh locations and of course,
07:33
all of your labor. So tell us a little bit about how these challenges kind of are dealt with and how you use A I and analytics to deal with that. So a lot of the inventory pieces and the deep learning that they're using with, I say, deep learning and machine learning. I think that the same thing, but also that's why I'm not on the A and I team, but using machine learning,
07:56
they're able to go through all of the stuff that we are buying, keeping um looking, looking at trends in the market for things like uh at the beginning of the, of the food cycle, right? So in order to get milk from a cow, you have to feed it, right? And so you have to keep an eye on things the cows are consuming in order to make the milk, in order to make the cheese,
08:15
right. And so they're taking a look at things very early on before it gets to the finished product that we're getting. In order to keep tabs on, you know what we're using, um how we're using it and how much we need that translates into the supply chain center management because we're ordering the food and all of the domestic goods and all the vegetables and all of the,
08:33
um, you know, vegetables from uh mushrooms to onions to peppers, right? We have to, those are my favorite olives, right? That's another one. all of that comes through our uh supply chain centers domestically and that is what keeping food in the stores and the proper quantity and, um, correct freshness window is really
08:51
important to the, to your experience at Domino's and making sure that pizza tastes the way that the same pizza that you order in. Brooklyn should taste the same as it does in Ontario, Canada. So, like all of that stuff goes into, um, the system and comes out into, uh into the finished uh analysis. The labor component is important as an independent business owner for,
09:13
uh, for Domino's Pizza because we're giving you that real time store telemetry on how healthy is your business. Um And so there are streaming analytics that take place in order to give you a dashboard of the, the pizza store going back five years might be more that I don't have to look up. But um uh compared to today and the year, you know, what's most important to you is the year
09:36
prior. So Super Bowl Sunday is a big day for us and we want to make sure that the store owners know how many drivers based on last year's demand there was. And so we're giving them help in terms of making sure that they understand what they need, what they need in the refrigerator, how many people they need making pizzas versus delivering pizzas versus taking waters?
09:57
And all of that is done through our A I and Business intelligence teams. Yeah, I, I, you know, in preparation for this, I made sure to get to, to get my order and I, I downloaded the application, you know, ordered and selected everything got notification of when it would be ready. Uh Let us know that you're in and, and then when they,
10:19
when I got there, they knew exactly who I was. It's pretty cool and well, thank you for enabling Bluetooth for the app. But, um, you know, that's, that's a real part of, you know, again, Domino's is very concentrated on customer experience. And so all of that telemetry that you're getting in the app is actually coming from real
10:35
time telemetry in the store. And as you're walking in, um, they know through, uh, in store wifi and that beaconing that, hey, Calvin's here to pick up his pizza. So, you know, you're greeted you, they know where they don't have to ask, you know, who you are, unless there's a line of 20.
10:48
But generally they're pretty quick. And, uh, hopefully the experience was something that you like enough to repeat. It was great and I, I did the survey and, you know, like you have information on me now. We know now. Yeah, all I need is your email address and I can figure out, you know what your favorite pizza topping is.
11:05
Um, so what we have next are a couple of more specific use cases around how you're using analytics and A I. So maybe we could talk about some of the, uh, the first one is on GPS Autocomplete. And I think this is one of your new electric vehicle, uh, electric vehicles. That's another thing. Right. Yeah. Well, this is actually the dxp,
11:27
the Domino's, uh, expert, uh, delivery platform. And this is something that we worked with GM on in order to kind of, uh, give franchisees the ability to, instead of getting, you know, your minivan and driving the pizza there, we have a place for the pizza. We've got a place for like all of the ancillary things like plates and forks and napkins and the two liter,
11:45
the those big cup holders that you wish you had on a road trip. There's four of those things in there for them to put the different size bottles of soda. Um, and all that all happened around the 2014 era, uh, and was really popular with a franchise. You could order one and then I think there's maybe 22 of those and still a waiting list of people who want to buy them.
12:07
Um, but GPS Auto Complete is important. Um, in terms anybody ordered Domino's pizza recently and followed your driver on the GPS, uh, driver tracking app. So a handful of people, all of you should download the app and order Domino's when you get home, just one pie, try it out. It's worth hungry every time we talk about it. But so the, the drivers have the, have a driver app on their phone and that
12:31
the route that goes to your house gets painted in a, in a GEO fence which, um, the A and I team figures out, uh, on your order. So if you are, what's, what is important about autocomplete is, is as it's coming through your geo fence, it knows when to send you an alert to tell you, hey, your pizza is almost here.
12:49
So put your pants on, get the, get the stuff out of the doorway. Domino's is about to arrive. Um, what's important about GPS Autocomplete is that it will, if that, if that driver is going on a run for two. So I've got two customers pizzas because I live close to Calvin and then I'm going to deliver
13:04
to Calvin first and then I'm going to deliver a bill. We don't want to fire off that alert to a, the wrong person or b the wrong GEO fence. And so GPS autocomplete allows us on an order basis to make sure that the driver in the order of delivery is going to make sure that Calvin and I get the correct notifications so that I don't have to get out of my pajamas before Calvin does,
13:25
but he's of that. Um And you use it for store siding too. So actually, when I, when I tried to order earlier, it told me that I, I wasn't within range for delivery, but I could do a pick up. And I think that that plays into all of the data that you're collecting of where your
13:43
stores are so that you can make sure you have a quality product that's delivered. Yep. So all of our stores have what we call a delivery radius. And so the franchisees very rarely will extend themselves outside of that radius only because the, the, the tests and all of the formation of the system say that if I go from the store to
14:05
Calvin's house, he's gonna get the experience that we expect and the quality of the food that we want, it's not gonna be waiting, you know, half an hour in a grub hub delivery to get to you and all soggy and horrible. Right. So store citing is very important in a number of ways. I think the stat I gave you was that 74% of all Americans live within a delivery radius of a, of a Domino's store.
14:25
And it may have been that, that Domino's Store was only doing carry out because they didn't have enough drivers. So our CEO CFO have been doing a lot of announcements around the availability of drivers in the store, which kind of lends itself to that electric driving fleet because Domino said, if the system has uh people with driver's licenses in the store,
14:44
if we provide them with an electric car to go drive, not only is it great for ESG, but it also gives us more drivers in the store who don't own their own car to deliver pizza. So that was part of that store. We take a lot of geographic data, we take a lot of data um about the population and we, what we do is we make sure that we have enough stores um within enough of the population um to
15:08
drive market share value and the um the continuous customer experience. So we're not happy if a franchisee only has one store, we want you to have a handful, um which is good for us, but it's also good for you and that we're doing all of this research tying in all of these data points in order to give you the franchisee the best possible um platform for success.
15:30
Um A couple of years ago, our CEO was talking about this Castling initiative where we would take a store, look at its radius, look at all of the surrounding things and figure out how to split that radius up and multiply the stores so that you would have not just one but maybe a couple that you could choose from in a standard zone to be able to deliver to more customers. But again, ensuring that same product experience and quality of pizza and that
15:58
when it comes to forecasting, and actually, that might be, we might be jumping ahead. Vehicle routing is that kind of related to that. So vehicle routing, vehicle routing is a new thing. Uh And I was just learning about this as we were talking to you about setting the session up.
16:15
Um, but it was a partnership with uh NVIDIA and uh um, is a platform that they use, um, in order to take information about the store, um, the uh, order and the available labor. So, um, when we're talking about, it's basically a way to optimize your delivery force. And right now, it's not completely rolled out to all stores,
16:39
they're still testing it in a handful of them. But what we hope this will do is in the event that you live in a more populated area and you have a couple of dominoes to do it. We can route your, your thing to where it needs to be um where it can come to you faster and get to you within a better expected time frame as well as optimizing existing store delivery labor in order to help get the cars where they need to be as efficient as possible to overall
17:07
lower the cost of delivery for not only you the customer but the franchisee as well. Awesome. Um And then the last kind of business use case we're gonna talk about here is forecasting clearly very important when you have days as big as the Super Bowl, as well as you know, more community oriented events that might be happening in one city or neighborhood or the other.
17:29
How does that? So the forecasting that it, it transcends like all of the silos of Domino's business. So from supply chain, from food ordering from uh sales and marketing campaigns, um all of this data um is all over the place. And so uh the, the, the team, the A and I team, what they do is they plug in a bunch of uh their deep learning models and a
17:54
lot of their um algorithms to basically figure out was this successful? Was this not successful compared to other things that we considered? How predictively, how can we think, do we think that doing making this tweak to this model? What does that look like in terms of outcome for the business. So if that's, if that's running a campaign a week earlier,
18:13
based on, you know, what's happening in the world to um making uh to helping with um uh where are we going to build our next supply chain center to keep food in the stores? Which is an investment that Domino's continues to make is all very critical to the business decision processes that happen at much higher levels than my pay grade. The keeping food in the store and reliably safely, um And keeping the supply chain intact
18:43
is of paramount importance to everybody. But as you keep building stores, you put stress on an existing system, there's only so many semis that you can run. There's only so many places that you can make the dough and all of that kind of stuff. So the forecasting models take into account a lot of those things in terms of where is the business planning to expand next? How many stores are we separating up?
19:03
Where is the largest growth for our franchisees to be able to make sure that we're putting the supplies as close as economically close as possible to those expanding markets to make sure that we're making the right decisions, we're making the right decisions based on the best data as possible. So these franchisees, they might, they might have one store,
19:22
they might have a couple, but really, they've got the power of all of this kind of back end uh information and data to arm them to, to be as successful as they can personally for their store and for their employees. Yep. Um let's get into a little bit of the uh the infrastructure and a little more on the technology side of things.
19:48
So what are some of the key A I applications, analytics applications A I frameworks? What are you using to kind of help run the business? So I think like torch um tensor flow. Um Python is a large portion of what we do. They do a lot of what we do is developed in Python and then using a lot of open source tools for,
20:12
for those uh algorithm building practices. Um Deep learning is a big one for, for us um specifically in terms of like that picture that you showed at the beginning where an international franchisee was using imaging in order to come up with quality. I mean, you have to teach a computer what a pizza is and what it and what a a um a properly dressed medium pepperoni pizza which only has 30 pepperonis on it looks like,
20:39
right? So in terms of those activities, there's those deep learning, the deep learning frameworks that we use. And then the um coop one which was the in uh a recent announcement from NVIDIA that we're using for the uh GPS stuff. Got it. And you've used pure a long time now. So when,
21:00
when you started off with A I, I think you started off with DGX systems, the, the NVIDIA specialized, you know, compute for A I, what point was it decided that you needed to kind of inject pure into the mix? And what did it bring, bring to you? What, what were some of the challenges that you saw? Some of the challenges were as I was describing
21:21
before, before we um started consolidating uh database workloads, whether it be non sequel sequel proper um or um key value store pair type databases. We they were placed on different storage platforms across the enterprise and they were also uh uh kind of cordoned off from one another. So getting information out of one place,
21:44
putting it into another place and kind of all of those ETL type ish processes, not only added a lot of time and overhead, but it also gave us a headache because if one, if one window was missed, then the rest of all this stuff can't happen. And then you start having a chain of events that keeps them from processing what needs to be handled by introducing flash blade. We were able to reduce all of that ETL and data
22:09
movement by 70%. And so what we ended up doing those, those little pockets of information data silos. All. Exactly. Thank you. The data silos were, were kind of squished down so that we didn't have to move it so often. Um And now what we do is we funnel it all through flash blade,
22:28
which is quick, we load it into the databases which run on flash array, which is also quick. And then we are able to cop, we are able to take entire copies of our data warehouse and attach them to what we've done data marts. So that if you're in the finance or if you're in the supply chain department, you have an entire copy of the data warehouse of Domino's
22:48
proper and then you can tweak and fiddle and mess around with it as much as you want in order to get, you know, the insight that you're looking for. Um which is a lot of what um the A and I team uses is a regular refresh copies on flash array um for that exact purpose. And so give me a storage platform that in the course of a single day can move 78 petabytes worth of data on and off the array.
23:13
It's a tough task. Yeah. So uh but we're able to do it uh through all of the native API S uh that flash blade and flash a rate come with and um are really happy with that. So we, we talk about that a lot, right? As, as companies are using more applications for analytics or for A I, right? You, you've got like,
23:35
oh, I've got this application here, I've got this storage dedicated to it, got another application here and this storage date and, and that introduces latency, it makes it complex to manage all of that. You don't get any of the efficiencies of shared storage. So you're able to consolidate that with the combination of flash.
23:53
I mean, when we introduced, um when we introduced flash blade, I mean, we were taking, we were using traditional um NAS file share for all of that ETL. So all day, every day from every store at the end of close, we're getting those exports. And so there's hundreds of thousands of files that have to be processed and munged in a way
24:12
to get it loaded into the data warehouse or into the areas that they have to go. If you're loading that onto a NAS system, it has to be performant, it has to be built for that task. It has to be resilient and it has to be easily managed. We weren't doing any of that. So bring on flash blade and I think we were one of the very earlier customers of Flash blade.
24:33
We brought that in and like I referenced that 70% number. It's totally real once we started. But once we started with the uh taking all of the data in on flash blade, running all the ETL on flash blade and then loading it either in directly into things that were running on flash blade proper or into uh databases that were running on flash array. All of that is flash to flash to flash.
24:56
And so then we started to think, well, wait a minute if we can load it and record it this fast. What about backups and restorations? And so we had a horrible day. It was somewhere in 2017, I think, where a developer got a little, um, got a little happy and he truncated a billion row, 14 billion row table.
25:17
And so, you know, should you have a table that large? I don't know, I'm not a database expert but the um but he's a guy but, but the rest, but the restoration of that off of our existing uh backup platform um was uh took us two weeks to do and all through that time as you're watching that counter go up and up and up here, like is this gonna finish? Right?
25:40
Is this gonna, is it and when it finishes, is it gonna be what we expect it to be or we're gonna have to start the whole thing over again? Um We ran that same type of test um after we switched all of the sequel backups to Flash Blade and we took that test from two weeks down to uh two days. That's amazing. And then once we got them to optimize their
26:02
tables, we were able to do the entire database um going from that two day period for that one table to 4.5 hours for the entire database. And so ultimately, you know, you're not sacrificing by consolidating on the right technology, you're not necessarily sacrificing uh performance um or uh availability. Well, it's funny, right, because people think of backups and they,
26:25
they're trying to get the lowest price for that. But, but when you can use your existing infrastructure to do both the high performance stuff as well as kind of get that, you know, really important backup capability as well as restore capability when that needs to happen. Hopefully not. Um, but that's really critical too. It, it really is.
26:45
And the nice thing about it is that we don't have to think about it being a pure customer. Um I think the to take that up one level, right? We have, um, I just got the, I just got the note from my team but, um, we have, he's looking it up real time. I'm looking it up real time only because I asked him, I didn't wanna, I didn't want to mess this up.
27:08
But, um, the, uh, let's see here, it says 19.5 petabytes of provision storage. And so the comes up to you one day and says malware is a problem. Uh, how are we gonna fix it? What do our backups look like? And I said, well, you know, backups are our, you know,
27:33
our VM backups are good or sequel backups are good. We replicate them, you know, we have protections against this, that and the other thing, they air gap to a certain degree um He's just like, no, no, no. He said, um I need to be able to guarantee that we're protecting against, you know, your people going rogue and we need to make sure that we're
27:50
able to actually recover in a short amount of time. How do I tell the board of directors that, you know, we are covered from, from ransomware. Um You guys had been working on uh the ability for safe mode to happen. Um You introduce safe mode, we decided on a topology that would take all of our data from
28:10
the flash array and the flash blades replicate that to a bastion array that is disconnected from all storage. It took us and then enabled the snapshot schedules, replication and safe mode implementation. We did all of that on petabytes of storage in a month's time span. And so when you're, when you go from ideation to uh architecture to implementation,
28:35
I mean, we were covered only because you guys had given us an extra piece of software on inside of assets that we already owned. And then just recently you took safe mode to the next level to where I don't have to do the whole array if I don't want to, but I can do it specified on groups which is even better because I don't have to sacrifice the capacity that I have for, for that safe for the safe mode snapshot if I don't need it.
28:56
So I mean, you guys just continue to layer that stuff on to where we're very, where it's easy for us to implement. Um And through our partners, uh get it going really quickly. It was great to have such a good answer for your, for your management there. I mean, it made me look really good. So, I mean,
29:13
that, that was good. And then we, we did it, we went right through uh the importance of data protection. Um So we're in, you know, the last part of our, our talk here, um we've got 15 minutes left. We, we'll have some time for Q and A but I wanted to divide up some of the tips and best practices to the different audiences that might
29:37
be in the room for back within your companies, right? A lot of times the pure storage customer is it focused, storage focused, they're on the infrastructure side of things. Um You know, you got the VM ware folks, you've got the database folks um and the data scientists and, and, and data engineers are kind of elsewhere in the company.
29:59
So let's start first with the it side of the house. What tips would you have for them when it comes to trying to support A I and analytics initiatives within their companies. So for us, we were, we were lucky because the um the group that managed the A I teams was a part of our organization.
30:20
Uh But once that got separated, um we kind of lost touch, you know, we were only like pen pals and they were only calling us when they needed another, you know, 20 terabytes worth of storage. Uh But ultimately, you know, if you are able to sit down and talk with these folks, not so much at the nitty gritty level of what they're using and how they're using it to
30:39
create these things that we barely understand, but to just take it just like a genuine interest of what the outcomes are that are expected of the thing that they're doing. Because I, what I find is that if I'm able to talk to a data scientist who is thinking about things from a much different lens than I am, um I can apply um the infrastructure acumen to how I think we can help them get to the outcome that they're looking for aside from,
31:04
you know, where the data is, how silo it is or the movement of all of that stuff. It's really just an understanding of your own infrastructure environment, its capabilities and its applicability to the outcome that they're talking about. Um I wouldn't um you know, when we, we were trying to attract developers and we weren't, we didn't have a lot in the cloud at the time back in 2015,
31:25
um we were attracting people to the A and I team because we could give them three on prem like they could develop as if they were in the cloud on premise and still be close closest to all of that data. They didn't know that um that that was even possible. And at the time, it was kind of convoluted in terms of how you would do it. But out of the box Flash Blade gave it to us.
31:47
I mean, understanding that that was something that they were used to leveraging and just let them grumble that we'll never get this done on prem we need to be in the cloud because of Xy and Z. Um you know, you can kind of, I just, I just tried to play um devil's advocate and say, well, we can give you this level of performance, we can give you these tools and we can plug into it.
32:08
Um This way when we were uh a cute story about that was that we were using our old NAS device as I was describing for a lot of the ETL work, but they were also storing all of the maps and all of the geospatial pieces on that array and then tying it to the NVIDIA trying to work out all of these complex GP U type things. Does anybody have ad GX in their data center? Have they ever heard it whine and scream.
32:32
Um Looking at plugging it in once we, once we moved that data from this less performant tier which they were unaware of and moved it on to flash blade. Like that thing went up three octaves. You could hear it from the glass enclosed room where we have more room meetings for because it was working just so much harder based on the fact that it had so much faster access to the storage and the data that it was looking for.
32:55
That's amazing. But how would they have known? They just, they, they don't just have to be open to kind of getting a peek into what they're doing just a little bit of interest and then they'll really open your eyes in terms of saying, not only this but take a look at, you know, five years from now, we have this talk another, you know, five years from now.
33:13
I can't, it's gonna look amazing and, and I think that's just the thing, right? So on, on your first point, these DGX systems are very expensive, very expensive. You've got a team of 60 A and I folks, they're also very expensive. You, you've invested a lot in that and if that system is not running as fast as it possibly can to make them as productive as possible,
33:39
you're throwing money away. And so Flash Blade helped you really kind of optimize those results. Yep. Absolutely. I can see a marketing video about, you know, like that GP U screaming faster and faster as we uh as we put in the flash blade. Um and you guys married or kind of like coupled in vidia and flash played together
34:03
before we even came out with an A I Ready Infrastructure. I mean, you guys were super close like we were, we were, we were doing it and then I think at that year's accelerate, you guys announced a right? And so like it was, it, it was super, super close. And so we just kind of like looked into it
34:18
saying like, well, what if we plug these two things together? You know, and, and so it was, it was uh it was a good trial and it worked out fantastically and you're a I ready infrastructure was better than ours. Only because you guys could operationalize it that much faster where we were struggling a little bit with the opera operationalization of all of the tooling through the DGXS and make
34:39
them worker nodes for the GP US when they were needed and compute, you know, elsewhere when it was not needing those GP US. Um You know, you guys really put it all into one package. And so had we had, we not been making the investments, we were, had we not had the demands that we had then we would probably be in the re camp,
34:56
but because we kind enough, but we, we came close. So honorary can't make sure. Um And for those of you who aren't aware, uh A or A is our A I ready infrastructure. We announced that and launched in 2018, uh we're on our second generation. So last year with flash blade s we upgraded that system to the uh ES and so that system now,
35:23
you know, supports all the latest GP US from uh our friends from NVIDIA. It's got the Flash blade s in there. It's got nvidia networking and it's all pre validated. Have you had, has your team had the chance to use our rapid file tool kit by chance? So that is something that I was not um very aware of prior to meeting with you, but you had mentioned it uh a few calls ago.
35:47
And so they're, they're, they're uh they're looking at, they're digesting it now. Wonderful, wonderful. So the rapid file tool kit comes with or you can download it from our works with the flash blade systems and where you have 2030 40 year old Linux UNIX operating like um file utilities, right? As you're cleansing as you're filtering,
36:11
sorting all all through the data for your A IML activities, that stuff takes time, right? And so the rapid file tool kit currently, I think 2.0 something um enables you to do all of those activities much, much faster in the magnitude of 20 to 80 times. So we're looking forward to the optimization because those are problems that we were trying
36:35
to solve before. So if only we had met like six months prior, then what it would have had that for him. But, well, next year, next year, um great. So let's pivot a little bit now. So we just went through a bunch of tips for the it side of the house on the data scientist side of the house, right? Like we've talked to customers who know they
36:56
exist. But that relationship that you said you lucked into didn't, didn't exist. Like the data scientists are doing their own thing. They're doing things on their workstations on their laptops, maybe in the cloud and they've got like different groups that are all kind of doing that. So what would you advise that side of the house?
37:15
So, I mean, I think that you can't be, you can't be shy, right? I think that's a lot of the a lot of the walls that are traditionally there, even between infrastructure and database administration teams, like there's those, those walls um throwing things over the fence only gets you so far. Um But those, those bite size conversations in
37:35
terms of um what am I doing doing and how does, how can you help me get to what I'm looking for if you can articulate what the business outcome is without kind of filling it in with all of the nitty gritty technical details of how you plan to get there. Um Other people can kind of get, get on your train by saying, well, I know a little bit about this, I know this person over here and these folks um do is
37:58
a small enough company where I mean, we are mid to large company now. But I mean, we're small enough where our culture is such that um if I were, if I wanted to go talk to Doctor Craig Stevenson, I could go walk down to his cube and say, hey, Craig, you know, have you, have you heard of this, that or the other thing? And he would likely tell me,
38:16
oh yeah, we were on that two months ago and this, that and, and, and we're, we're applying it here and here. Um I think really, it's just a, it's just a matter of being able to um articulate the outcome that you're after with people that have either been there a little bit longer than you or just as long in order to kind of coalesce around, like how are we gonna get there?
38:37
Um Because a lot of the um as a, as an A I developer, I don't necessarily care about the plumbing, but the plumbing affects me. And so if I, if I, if I'm, if I'm keen to that idea at the, at the onset of whatever project I'm working on, um then I can create relationships um and kind of get that map optimized so that I can get to what I'm looking for faster.
39:03
So if they don't have to worry about managing or deploying a system, if you can lower the cost of their infrastructure overall, and maybe best of all, if you can accelerate a bunch of their results and have them not wait around. That's how you kind of share your value with them. Generally that, I mean, that's, that's the
39:28
brass tacks of it. All right, because we are all after the same thing. Um, whether they, you know, we all want Domino's to succeed. We all want to do it the, the best way possible. Sometimes that's not always easy to accomplish. Um, Domino's as an organization very early on. Um hosts these learning days where we uh sit
39:46
down and learn about what other departments are actually doing. Um And then a lot of times my teams will take that back and say, well, this is going to all of these future plans are going to affect this, this and this. And so this is where we have to kind of concentrate where we're looking at moving one thing or consolidating this other thing.
40:04
Um But it also gives the, the every everybody from the Java developers all the way through the A and I organization to say, you know, this is, this is what we want to use the cloud for. This is what we have on prem today. This is where we're getting all of the, the things from and how it impacts you and your different vertical. Um And it really helps the entire Domino's organization really understand how collectively
40:28
we're going to get to these goals that, you know, seem far, you know, so hard to get at the beginning of the year, we normally nail at least 80 to 85% of them. Um Given that we're all encouraged to go and have these conversations and to, and to take the time to learn about these other verticals.
40:48
I think that like people, culture element of Domino's is kind of key and that communication is kind of like the linchpin that pulls everything together. It really is. Um And I feel like I feel comfortable, um, talking to anybody in the organization. I mean, I've only been there for eight years now.
41:08
Um, but, and a lot of people have come, a lot of people have gone, but I mean, the mission is always the same and then, you know, at the end of the day, we just want to sell more pizza and have more fun and can A I and M get us there faster? We believe that it can, which is why we continue to make those investments.
41:24
That's awesome. Well, thank you so much for joining us on stage today to be here a few more minutes. Uh, in case there are any questions in the audience.
