00:00
Yeah, everyone, uh, thanks for, for joining. We're gonna be starting to talk now on the state of AI, uh, it's gonna be a wonderful talk talking about all the latest AI innovations, how they're impacting storage, how they're impacting workflows and businesses, and we're gonna have a panel as well that we hopefully all can participate and ask questions in. It is, yeah, yeah.
00:22
And one more, so we're, there we go. Uh, we're joined by Par Boats, our VP of AI infrastructure, uh, Richard Galvez, uh, who's our principal FSA, and then Amir, uh, Bisser, who is, uh, also an FSA working out of the Chicago area. Yeah, but Richard, off to you, brother.
00:44
All right. Cool. All right, well, good afternoon everyone, and uh really thank you for uh joining us for our state of 2025 State of AI 2025 presentation. So innovations in AI over the past year are moving at a dizzying pace, uh pace. Even faster than ever before, I think we can
01:03
all, we can all agree on that. So at at PR we've analyzed the landscape of AI breakthroughs, um, maybe not every single one, but most, right, uh, and identified the key points we believe all enterprises need to be aware of, right? In particular, there's 3 kind of, uh, key takeaways, so we've identified that.
01:22
Innovation brings affordable access to high performance AI so more so than ever today, you have very affordable access to advanced AI technologies, and we're gonna get into that in a second. Secondly, So recent breakthroughs open new enterprise opportunities. So we identify a lot of low hanging fruit right now, just, just very recently, in fact, even while we're putting together these,
01:43
these slides, which is almost comical, but uh we have examples throughout the, the deck where we literally had to change it a few times over as innovations were happening. Uh, so, and thirdly, we've also identified from our experience working out in the field and working with different clients around the world, 5 key design patterns to help you really land your AI initiatives and your and deliver AI outcomes and we're gonna share these with you.
02:09
Right, so the first one, so key innovations that are impacting enterprise today? Well, the first thing that we know is that the gap between open and closed models has really begun to, to close dramatically. So it was about January 2022, there's a considerable gap where if you really wanted to have a meaningful performing AI that was really capable of understanding quite a lot of your
02:32
context, etc. you have to potentially use uh an API endpoint likely. But now, uh, we now have innovations where you could in fact use open models and then from there customize them. Uh, this gap is dramatically reduced. This enables on-prem deployments, such as even up here.
02:50
So we're just talking about our copa and I think you, you saw our, uh, our demo today. This has been enabled by this dramatic shift. Furthermore, at the, at fixed model capability, we've seen the size of models decrease considerably, about 8DX over the past, well, uh,
03:11
yeah, about a year or so, a year and a half, you've seen about an ADX reduction in size model. So that means that it requires less infrastructure, it requires less overhead. And as models continue to get smaller, what's really interesting is that maybe you could even deploy them. On CPUs, I don't know, right, so there,
03:30
there's some interesting things that are happening in this space are becoming much more optimized and it has to do with uh uh innovations and improvements across the board, refinement and training data, the workflow of training are uh refinements in the training process themselves and even in model architecture. So all around the, all around, uh, these innovations are compacting are impacting or
03:51
compounding, sorry, in a way that allow for, for this to occur. Furthermore, as a almost consequence of this, it kind of follows naturally, models have become increasingly inexpensive to use, ah, in terms of tokens for per dollar. So we find that there's about a 100X reduction in cost.
04:12
It's very dramatic, it's two orders of magnitude in the past about a year and a half or two. Further, we see that there's an exponential increase in AI investment, namely around infrastructure. So if you look at comparing here from 2023 to 2024, you see an exponential increase even just in infrastructure. So just focusing on the top bar there,
04:34
right? Now, there's an interesting place to look though. I'd like to draw your attention over to NLP customer support. You see that back in 2023, there was more innovation or or more investment there, but that's when everyone had the craze to um deploy chatbots.
04:48
You've seen a reduction in that investment because you haven't needed more. Uh, people have successfully deployed these systems, but now we're beginning to look at bigger and bigger projects. So, investment in infrastructure is even increasing further. This kind of in this kind of technology that's being driven, uh, AI is really driving towards forcing, it's,
05:08
it's a forcing function really on infrastructure tech to also keep up. So we're seeing this happen across the board, Nvidia and others, uh, AMD Intel, etc. are all driving towards this, uh, really improving on the hardware technology itself and further. We see that the big uh sort of AI titans around
05:31
like say Nvidia, in particular Nvidia is pioneering this new system, this Dynamo server that is now beginning to think about large scale inference, not just for one person or 10, but 100,000 people, a million people all at once. How do you optimize that sort of ocean of traffic? And of course, there's a company called Pure Storage, which we all are familiar with,
05:53
uh, we're also breaking the uh the speed record and throughput, as we've innovated also in this space to help the large AI training uh workloads to be able to, to feed all their data in the way that they need. So we're able to break the 10 + terabytes per second speed limit, it's, we're able to go beyond that, uh, but that's what we're allowed to say right now.
06:17
So, taking all this into account, everything we talked about so far, there's an overall impact or compounding impact for enterprise today. The message is pretty simple. You now have affordable access to advanced AI and the infrastructure around it. So really the barrier of entry for performing AI is dramatically lowered now,
06:38
you really have technology at your fingertips, such performing AI that we didn't have just recently, just a little bit ago and right now the sky is the limit, so we really think that it's it's a really great opportunity for you to jump in. So with that, Amir will tell us about. If my clicker works.
06:59
There we go, back. All right. All right. Thanks, Amir. Thank you. Thank you, Richard. Can I have the clicker? Yeah. That's good. Hi everyone, um, thank you for joining us today.
07:13
Um, it's really good to be here with you and just kind of talk a little bit about this. Um, yeah, so my name is Amir. Can you hear me OK? Yeah, um, and really what we, we want to talk about a little bit today is just kind of how this plays into enterprise, right? How it kind of comes out into how the
07:31
application of these things, AI specifically is playing into the enterprise. So first we're gonna look at um these metrics that were really put together uh by evaluating kind of the frontier models that are out there, um, specifically, um metrics that focused on highly, uh, specialized or R&D for focused tasks, as well as metrics that focus on kind of general purpose tasks,
07:55
right? So what we have in front of us is really Um, evaluation of, um, yeah, R&D type work that is really kind of pushing the boundary, right, of what, uh, specific field or research is trying to do and what we can see within this environment is that um, As the context or as the duration of the, the problem grows.
08:21
AI really falls off and human comes out in front, right? So the, the, the, the breaking point really comes about maybe 5 or 6 hours of duration, but by 8 hours it becomes very clear that um specialized tasks when you're trying to do some type of maybe uh research around improvement of certain algorithms or, or chemistry or whatever the case might be, right, uh,
08:46
you can only go so far. Um, if we kind of switch over to the right, what we can see is that general tasks with the duration, at least for the previous generation of, of these frontier models, kind of peaked out about 14 to 15 minutes. Um, we're not taking into consideration here and as Richard was saying earlier,
09:04
right, these things are changing so fast that, um, some of the newest releases that are, that have kind of pushed this boundary even further. But what we can see for generalized tasks is that the duration was really kind of Um, peaked out at about 15 minutes of uh task completion. So if you're setting yourself out to research a document or do some code analysis or long
09:29
analysis, or whatever the case might be, that was really your barrier, um. Um, at this, at that point in time. So, what has changed, probably in the last what, 2 or 3 months or something like that, right? What we have seen is that we have had a, uh, One of the, the frontier models specifically for kind of talking about
09:51
anthropics, um, frontier model is they're now able to push tasks specific um workloads more greater than 7 hours. It's like 3 weeks ago, yeah, right, OK, it feels like months, but it's, it's, yeah, it's, it's, it's not we're talking about that that uh that breakthrough, but I think it's one of the biggest breakthroughs that happened recently.
10:12
So really now enterprises that have um very clear definition of the task, right, or specifically if you're maybe looking at Greenfield deployments of, of, of app or projects, right, um you have a path through the, the frontier AI to actually be able to implement it, right, be able to deploy it, uh, push it out into.
10:35
Production, push it out into usage, um. And that's extremely powerful, as we said, opaqueness is still a barrier, right? And those will be interesting to see how these things now change out as the, as these um metrics are re-executed or these texts are re-executed, right, with the next iteration of them.
10:55
But uh for right now we can see the the the tasks that are very well defined, we can do some great execution on that, especially in coding, especially encoding. Um, the other, the next piece we want to talk a little bit about is really just kind of how visible the reasoning is becoming. How many of you remember, uh, Deep Seek Monday?
11:16
It was only a few months ago, right? Yeah, everybody freaked out a little bit, right? So, but really what we ended up um uh seeing is towards the end of last year, right, we had uh OpenAI release. An updated model, right, that gave us a view into reasoning, gave us a view into ability to uh understand what the black box was doing,
11:40
right? And what we were um experiencing is now this next evolution where you would, you had some reasoning, but with that obviously there came cost, there came expense, right, or, or it was considered a premier product. Um, shortly or soon afterwards, right, we get this release from Deepeek, right, that completely changes the landscape and gives us a,
12:06
an open source, uh, implementation of a product that does exactly the same thing. But also, it reduces the overall memory footprint of the model itself. Great optimization, right? Now, all of a sudden you have this extreme power in front of you that is optimized for extreme efficiency. Very powerful, right? So the, the, the, the, it,
12:34
it changed, it changed the, the environment completely and it gave us. A path specifically for an enterprise, right, to take advantage of these models where they were no longer, as I said, considered a black box. Now they were a clear box, we could see inside of it and we can understand how the prompt was actually influencing direction, whatever the, the, the uh it really made it really made
12:59
reasoning available to uh to the masses, uh that that's what that that innovation really did for us and that's what, yeah, we like to point out there. Yeah. Will the videos play by themselves? Oh yeah, they were awesome, awesome. So, um, the, the, the next thing or the last thing we really want to talk about is this little piece
13:19
that's emerging really the world models or development of world models. So with all of these advances that you're kind of seeing in the, in the frontier models themselves and the advances that we have seen so far in a generation of videos, how many of you have seen um um. Will Smith eating spaghetti, right? How many you saw it like a couple of years ago,
13:40
right, where it was like spaghettis were flying all over the place, right? If you look at some of the latest generations, it almost looks passable. It looks like he's really eating spaghetti, right? And, and so not only are we now getting a video generation out of it, oops, um, not only we're getting video generation out of it,
13:58
but we're also now getting context like sound, like speech, right, we're getting. Um, modalities like, uh, image generation that's beyond just video and so depth perception and so forth. So there is, there's a lot of, um, these pieces that are now coming together to really establish a world model and what is really a world model?
14:20
Well, it's a digital representation of what is it that we're doing. We talked a little bit about code generation, right, and what Anthropic is able to achieve, um, with, with um cloud code. Um, so think about it in the same context, but now moving it into more maybe R&D type of workloads, multimodal, uh, more than be going, going on beyond text.
14:45
So now you can also take audio, video, etc. There's a multimodal understanding now, uh, it's becoming refined. Yeah, that, that ability to really be able to perceive multimodality is extremely important. Somebody like myself, right, who actually did a lot of work in the medical industry, right, multimodality was always a key. You had image,
15:04
you had sound, you had all kinds of other input that was coming in. It's here, right? It's opening up the door to those possibilities, those opportunities, those ideas. Um, so what this really means for the enterprise is that, um, these use cases are becoming possible.
15:29
The future here is in front of you to take advantage of this and really build on these use cases and create something. It's also Time to really go after your initiatives, AI initiatives. This is a lot of low hanging fruit right now. I, I think it's the main part that we really wanted to get through,
15:47
uh, yeah, exactly. So with that said, I'll turn it over to par for some design considerations. Thank you. um, can you hear me now? Fantastic. You're, I don't think you're audible. Uh, I should be on now. All right, so, uh, Richard is a robotics AI guy, a medical AI guy.
16:07
And for most people in this room, I'm known as the guy who built flashblate but may not be known to you as I, I left for 4 years and I built self-driving cars, perception networks, so I'm the car guy, um, and a lot a lot about AI in that world because I had to build AIs that could kill you. So I didn't want to do that. So that really really work.
16:29
And that formed a lot of my opinions about how you, how you think about infrastructure and how you think about the problem. And, and we try to summarize some of these points in the next couple of bullets. So, I think the sign first, right? Think beyond the chatbot. Think about the user, think, think product.
16:52
You're gonna, everyone here, I guarantee you in the next few years you're all going to build something. A little less hard than you think. Sure, it looks pretty hard right now because the tool is not the easiest to use, but it's getting easier by the minute. It's getting more capable by the minute.
17:08
But as you start playing around with this, think the way a product guy thinks. Think, think what, what does this do? How does it work? How do you relate to it, how do you interact with it? I think Almost no one evaluates what they do. They start to build something, to see if it answers some questions,
17:27
but not really critical about it. What happens when I throw in wrong data? Does it get wrong answers out? Or does it tell me that was wrong. How do I test it? How do I evaluate it? Think evaluation. As you building these systems,
17:41
record every input and output from day one that forms the basis of your test suite. It also forms the basis of your refining method later on when you decide that, hey, I'm going to make it run really really fast, so I'm going to start incorporating some of that data as training data and not just as as things that users asked out of my system. Think about data as, as, as, as, as the way I think about software code,
18:06
um. I, I said this intern in the company many times. I've come to think about models as compilers. I think about data as my code. So data is a code. What do you do with code? Well, you was it, you name it, you make sure you have completeness.
18:23
You make sure you actually have a somewhat organized so you can reason about it. You can look at it and sort of say, this is kind of what this part of code does, this is kind of what that part of code does. I see data the same way. It's kind of what I learned in self-driving cars.
18:36
We have lots of data. We have road data, we have landmark data, we have vehicle data, we have environmental data, we have vibrations and steering the car data, all kinds of data. So we're creating what we call dimensions. Each each piece of data exists in different
18:49
dimensions, which is basically thinking like false file directories, like hierarchies. Think about data and think about it critically, and you have to eliminate the data silos. Like, like if you build a self-driving car, I'm not going to put like, hey, my road data is going to exist in.
19:05
A database and my, my, my, my, my, my planning data is going to exist in a system, it all kind of belongs together. And it's gonna be incredibly hard to build something if, if you don't think about data, and you start thinking about data management. This is going to force us to think about a new discipline called data,
19:22
data engineering. How do I transform that and make it ready and and normalize it. The, the, the, the, the truth of the matter is in my opinion is that you can't really have a meaningful strategy if you also don't have a meaningful data strategy. And that that that's that's that's an emerging discipline.
19:40
The big guys who build this stuff at scale. They have that, they have that in spades. They just don't tell you about it. That's kind of their secret sauce, but you have to build some of that discipline for yourself. Um, talent is still the limiting factor. We haven't yet mainstreamed these skills.
20:00
Tooling and, and, and, you know, new classes in schools is making mainstream. I have a small team that I that I oversee that work on these things. This is all they've done in their entire career. Uh, more and more kids are coming out to schools. This is all they know. Uh, the barrier to entry for folks who are as
20:19
old as I am to learn this stuff isn't as hard as you think, but it takes a bit of effort. Uh, totally makes it easier. You can make some experiments with it. You can start playing around with cool things like, like we showed the MCP model the context protocol on the main stage earlier today.
20:36
Uh, I worked with two guys, we hacked it up 3 hours maybe. 3 hours from idea to like we had it running. The barriers to entry is reducing fast. Um, Unconstrained tokens. So I tend to think of tokens, so I, I, I tend not to think about cost as token.
20:57
I mean, I, I know that's kind of how vendors like to talk about it. Tokens are expensive. I tend to think of tokens as like efficiency. Can you get the work done with few tokens, then you win. You just make it more efficient. Efficient, you always win in the long game.
21:11
But I don't start there. I start with something I use as quite a bit of token, and then I figured out that OK so now I know exactly what it does and how it needs to do it, and then optimize. But at the end of the day, I want to unconstrain myself and be limited by tokens. Because that's how you get speed, that's how you get correctness,
21:28
that you get all kinds of goodness. So the practical advice, which is really the next slide, is the following. Evaluate early. You build those systems for collecting data, evaluate early. Evaluate it often, value for completeness, correctness, collect feedback.
21:46
We all must put in the AI language word for that, you know, anyone here know what the AI language word that is? Safety. I don't want to use that word because I think people think it means something different. But I really evaluate early.
22:02
Start small, simple, stay user focused. I think we have been given a gift by these frontier models, a gift. You can take these massive, massive models from Anthropic, from OpenAI, from, from Google, from, from Amazon, from Microsoft and others.
22:19
And you can do something I've never done in my career. In my career, the mantra has always been, you start small, minimum viable, you iterate, you add to it, and over time we're gonna have something big. But here you do something fundamentally different. You start big. And then you reduce it down to the smallest
22:38
possible model you can get to make it fast. start big, build the biggest possible problem that you can solve. And then figure out how to optimize it afterwards. It's so backwards, it's so antithesis to everything we thought, but it works so well.
22:54
And I highly recommend people to do that. And once you're done, not done, once you figure out a problem, then I think about optimizing it. Can I use a smaller model? Maybe you don't have to fine tune. Maybe you can just use a smaller model that
23:05
makes it run faster. Can I reduce my token count? Can I drive efficiency. I'll give you a cute little story and a side of pure one shot co-pilot interface. Uh, at halfway through the beta, so we are now made into GA.
23:23
Halfway through the beta, we had to find it pretty heavily. One of the queries you stopped 3 30,000 tokens per query per per question you asked. But now we understood the questions really well. We saw enough people who asked it in various ways. So we spent some time looking at that, we optimized it. Uh, we fine-tuned a little bit of database
23:42
schema and into the query to reduce a little bit of token count. We took it down to 90 tokens and it said the 10 seconds response time we took it to 500 milliseconds. But it wasn't work to do upfront. You do that after you know what the problem is. Impossible to do classical software engineering.
23:58
It's so cool to do this in this style of engineering. Invest in talent. The limited most limiting factor in our company, I'm sure in every company is scale. self invest in your own talent. I mean, if, if I can learn this stuff, anyone can learn this stuff, and, and, and, and, and stop playing around with it.
24:20
The the the bar tent is lower and lower. Uh, we have built a team, 2 members of that team. Robert is the 3rd member, and we have a few others who are really, really solid experts in this. All PhDs constantly and, and, and, and, and they run workshops and the whole idea of a
24:40
workshop is draw out what's, what's a good first use case? How can we support you with that use case? How we can build some of that matters. Tell, tell, tell your partner, tell your rep. I would like to do one of those workshops. I think they work. I think they work pretty well. Lean on partners SI preparing infrastructure.
24:58
But I think these are like the 5 things, like if I, if I'm a regular, I'm running systems and systems engineering, this, this will be the topics I would push on to get going. The barriers to entry has never been lower. You can get models super easily right now.
25:15
You can deploy them super easily. And and I I I can't, I can't just wait to see what you're gonna do. Like the, the, the whole idea of MCP is to have you do things that we couldn't think about. We, we did, we did the, the demo with Sean on stage through flow programming, but we didn't write real code.
25:33
I mean it, it's so simple to do. And it just takes getting started. So I think with that, that's the conclusion of this and what we hope to do instead is have a large extensive Q&A session. Anything you want to know. So let's open it up. Anything we can answer that's on your mind.
25:53
All right, thank you. I'd like to close with the one quick quick piece before we move on to our uh Q and A, but um. As, as Paul was just mentioning, I think uh it's really like this kind of the access to be able to do cool stuff. I mean really there's just so much at your fingertips like go out and build,
26:17
right? There's just a couple, there's a couple ways that things can go wrong very easily and we just wanna like really expand upon that like the evaluation framework of it it's really important to figure out not only uh you know, Like when, when is, when is it unacceptable for your model to continue, like if it goes off the rails, right, going off the rails and doing crazy stuff,
26:38
I mean, there's a very exact example there that that's obvious, but bring it back a little bit, you know, it's quite nuanced and that I would say is quite a complex part of the whole pipeline deployment uh that we've seen people kind of struggle with, um, but anyway, uh, with that we'll open up to questions, uh, anyone. How are you, how is Pure as an organization,
26:59
I'm sorry. How is Pure as an organization vetting or evaluating further innovation from third parties that will either provide an entirely new AI kind of agen agentic capability and or Innovations that will help your customers, prospects kind of understand their data, where it lives. I love the, the data silo point you called out,
27:27
but is that your team that's evaluating those. So quite frankly, we have a, so we have a team inside of the company. I don't know if this is on or not, but we have a team. I speak loudly. We have a team inside the company that I work with, uh, they act as a service center to the rest of the,
27:43
of our company. And so every department is our company from any department you can take the receptionist all the way to the hardcore backend developers, legal, finance, HR they may have an idea of something they want to do. They can request time of that team and that team goes and helps them build prototype,
28:02
test, evaluate, and protize. Uh, so we run, we've run so far about 50 of these, uh, give you an idea, the, the co-pilot that we announced last year was a hackathon. That an engineian wanted to do. He wanted to see if it was possible to do hackathon. We have lots of hackathons inside engineering.
28:19
He didn't quite get it, get it done because you only get one day in a hackathon. That's all you get. And, and, and, and from that they requested time with this team and within a week we had something going. So we, we set that up as a way to do it. My my team tends to spend a decent amount of time on external stuff,
28:36
and then we get field in input generally through the FSA team. This is what they see in the market. This is what I hear and then. We, we, we, we try to coalesce that by essentially I host a bunch of these guys inside conversation for special tasks like Robert is currently seconded this side of my team is building something we hope to announce in 3 weeks at London accelerate and,
28:56
and uh so that's, that's how we try to do this. It's not super formal yet because it runs so fast that I can, I can't turn it into a really rigid structure. Once the market sort of starts coalescing a couple of like, like this is where everything standardizes on.
29:11
Then then they'll probably formalize it a little bit harder, but that's how we do it right now. And you also make for like third party vendors, right? Like if there's, well, some so. I, I work on the field site, so when I work with a customer that maybe we find that there's
29:25
a, I mean we we all start with the use case and the problem that they're really ultimately trying to solve. Uh, so for example, we're working on a guards world's problems, so someone was deploying a chatbot, but sometimes they would give back, uh. Uh, a generation that, uh, the, the team didn't like, OK, so how do you stop it, right? So then we identified one particular third
29:45
party vendor that had a kind of guardrails as a service system, and we, we helped them define the criteria, etc. but it was more of a one-off. So like we basically work with customers to figure out what problems they're trying to solve and then we evaluate the third party, um, product. We might reuse, you know, because we're we're accustomed to certain,
30:04
and but we'll evaluate on their use case. Does it work for them or not because it doesn't have to, so it's more of a one off, it's it's hard to generalize it. But, but also I mean we do have partnerships, directs with Yeah, I mean more generic things like uh OK, a spark or something like this, yeah, yeah, yeah.
30:23
So yeah, if your use case applies to it, then definitely I'll follow on the the data silos question was here how have you addressed data silos at the in your company and how are you planning to do it tomorrow? Uh, it's a very good question. How do you address data silos in your company? How do you, how, how do you expect of all this, so.
30:44
Uh, we have actually probably fewer data silos than most, uh, but we, we, we have them, of course we have them. Uh, our silos are generally organized around sensitivity around the data rather than the data type or or the data set. So it's very sensitive data tend to be somewhat compartmentalized and that requires some access
31:04
and, and, and, and, and if you're going to use it, especially for something like AI then we have to anonymize it or, or, or tokenize the data so you can lead back. Sorry, is it better now? I have a really deep voice and that typically doesn't get picked up by microphone as well. Um, so, so we, we normally, we, we, we have silos by,
31:22
by, by, by access largely by sensitivity of the data. Uh, if you're going to use that data outside of that, then what we do is we normalize it and then, and tokenize it to make sure that we can't leak anything that's sensitive. Uh, but for data that are relatively harmless, let's call engineering data, systems data, log data.
31:44
Uh, that, that we tend to store in data lakes and access more directly. But we've done that for a long, long time. Uh, so entirely we use a little bit of homebrew stuff, uh, open source. Uh, we have partnered with a few companies, we released some white papers around it, uh, but, but entirely we we've mostly been using
32:04
open source. We have a, a partner for some data require SQL access. We have had a long term partnership with, with, um, with, um, um, some, some cloud companies that we use for, for parts of it. But for engineering systems, that's all in-house and that that's.
32:22
Yeah. Yeah, yeah, on the business side, we have, we use internally or what our customers that use our system, so that that's more sensitive data. So that usually means we do an extract and transform to sort of master if you have to. And legal, legal finance and HR tends to be more sensitive data sets.
32:46
So then when we do project there, we kind of compartmentalize that a little bit for sensitivity reasons. And that goes to my to my question. Do you recommend the first prices. Kind of address their own how they do business. The ERP systems, the finance system, the HR systems, the marketing system before they go
33:06
and implement this. No, what we did is we picked the problem. And so once we picked the problem, give me my favorite problem. You wanna know my favorite problem, the most unexpected one. It's the legal system. Guess where we spend a lot of time in legal.
33:23
Contracts. What happens before you have a contract? NDA? Do you know how many times NDAs go back and forth? And every time it's like 20 pages that actual human being has to read through, that's fairly well paid. So legal came to us and said, what can you do for NDAs?
33:39
So we, we thought an A what an ideal NDA looks like from our point of view. And then when we get the, the NDA from someone with markups or or changes of the NDA, the AS goes and highlights. These are the parts that are incompatible we like, and then the lawyer only looks at that piece. That was actually not super hard to do. Now we had to train it on a reasonable amount
33:58
of data we had from before with different forms, but. But, but after that, it actually created huge time saving that created a big inroad with the rest of the legal team to come up with new ideas that we worked on. We led by ideas because it was so hard to get the groundswell of of of support by the by the leaders of legal, by the leader of HR leader of finance, if you didn't solve a problem for them.
34:21
So we started there to get to the data side of the problem. That's that's what we did. And furthermore, we are, I mean, we have multi-tenancy built into our, our system, so you could imagine having a rag application or even imagine our purity logs uh co-pilot, right, if you're a customer A or customer D,
34:39
uh, customer, you know, customer B should not be able to see customer A's log data, etc. or, or do actions on their, on their systems. 100%, don't worry, of course, we're taking care of that. So, uh, we have a multi tenancy in, in our system, so you could do um role based access, right, where you say, if you're running this through a singular model,
34:57
right, a particular uh large language model, let's say, you could uh authenticate where which data should the model have access to. So as you're doing your query, you figure out that this log data is the one that should be gathered and put into the LLM right? uh and that is completely partitioned from. Uh, customer B that is, that is gathering other data for another log analysis and that goes
35:19
also through the LA, but that's OK because the LA is agnostic to, to that, it's not being retrained or something. It doesn't remember the data at all. So all that you need to do at that point is just partition, make sure that uh typical raw access control stuff really.
35:33
And where do you keep those knowledge graphs all those different interconnections. Uh, well, we have all kinds of KV, uh, this is a proprietary, uh, uh, design, but yeah, we have a whole, we've been building this company to do this for a long time. that you mentioned and and data is critical garbage and garbage out.
35:55
Sometimes people would just literally take data that you don't want HR to see and also, or maybe like some other department and physically have it separate. Yeah, that's very a siloed, maybe. Too hard of a silo, but then you have a data science team that wants to come and do some, some training on, on both data sets, and you might say like,
36:14
whoa, whoa, whoa, whoa, that's scary, don't do that, don't mix those data sets. Let's suppose for a moment and and like entertain me that there's a way. It makes sense to actually do this. There are ways to have this data together in one place with a, with a, let's say application layer uh silo, where only who the requester,
36:33
the authentication comes in from the que the requester and that IO is in. Allowed to go where it goes, but we we have that built out in our software. I trust that you can do it I'm asking for the company it sounds like you build, you're drinking your own champagne, you build a lot of stuff in house but to me what you mentioned data seems like the first step should be you should have an enterprise data capital
36:55
and if you decided to build an enterprise data capital for you you have enough cycles to do it. Step 0. Yeah. You want to build it. It's not that they don't want to build it, they want to build it, but they're, they're stuck in this, they're mired in that situation, and that's like the step zero problem that we find a lot of people are,
37:10
are just literally trying to consolidate their data. So I think that was another way of looking at that point is fight the data silos because that's the first thing that you've got to figure out is how to consolidate your data in in a, in a safe way, in a way that makes sense with your, your, your data is engineering data we consolidate that.
37:31
We build the taxonomy on it. We're not building basic triage around that, and we're doing that as a centralized project. And and as new engineering products come out, they just have to adhere to that methodology. So we're doing that, but that didn't start that way. It started with solving some problems so that the engineering management got trust in that we
37:50
could solve the bigger problem, and it was worth the effort to go through that. Thank you for the question. Any more? Um, how do you manage to be able to concentrate all your tokens or or your effort when your model is not going when? One is maybe uh it's not a fully trash but when it goes in the wrong place to
38:23
the wrong direction. Go ahead. Actually, could uh could, could you repeat that question, yeah. How, OK, how do you manage um your model when you are using too many tokens in order to get the right result or when you're going to the wrong direction? Well, OK, so you're saying that you, you're, so you're attempting a problem and it's either too
38:49
expensive or it's just not working out right, etc. Well, that's what going back to Pa's uh point about this being a completely uh Um, backwards, uh, process, then before you kind of start small, an MVP, you know, 2 + 2 is 4, nailed it, we got addition, and then we build on and we build up pure pure storage, right?
39:10
Uh, so it's actually the opposite in this case. So what I would suggest to you is, um, start, that's why we say, uh, really there's no shame in starting with OpenAI's API endpoint. Or anthropics or, you know, a coherent, whatever like, because there's actually a lot of complexity that goes into really getting that right. So start first with those big models,
39:32
right, like the, the expensive big ones, make sure that it actually work, that your system is, is possible, you might be asking something that's impossible, right? I've tried that. I, I've sometimes I've asked it to solve the dark matter physics equations of the universe and I can't do it yet. That's one of my, that's one of the ways that I
39:47
check my, if my, if AI is getting better or not, right? So there are problems that are impossible. You might be asking an impossible one, but the first thing to note is, is it, is it, is it possible? am I asking something crazy? So use the biggest model you can get access to. So use like,
40:02
I'll just say like try like GBT40 uh opening eyes like uh API model, run it through and see if you get sensible results, then you also have to define for yourself. I mean, don't just look at the output and say, that looks good, that works, or let me look at another output, uh, that doesn't look quite right.
40:20
Try to think about, that's why defining your evaluation systems early makes a lot of sense. So think about your use case, think about what's acceptable to you, when does, you know, 2 + 2 is, is it always for 100% of the time? Cool, that's good for me, or maybe I'm OK with it sometimes being 5 or 3. Some people are, uh, right? And so the point is that you've got to define
40:42
like. What is acceptable to you, uh, just not on like a one-off basis by just looking at the output, uh, try to be a bit more robust about it. Start with the really large models, make sure that that works. Make sure that the people that you're building a thing for care what you're building for them. That that's another thing that's what we're saying about obsessing with the user.
41:01
It could be that, uh, they, they don't, uh, they, they don't care to use a chatbot. Uh, maybe there's someone that is leaving the office in the morning and they just wanna like. get a printout on a paper that's gonna tell them the route, right? They don't need to care how they got that printout, right? It could have been through the most magical AI
41:17
we've ever seen, or it could have been on an app, whatever, right? So, Make sure that that that pipeline is working and then you have this big giant model, then you can figure out refining, right? Maybe your data was, right, so you refine that, so you start from like really big and you refine down until until it's very optimized.
41:36
Let me, let me give you a practical example. So I shared earlier that we tried to use token on on uh on co-pilot, so, so. So the first problem that came up, it wasn't as much a token count, it was the time it took for a response. No use someone's asking a question and sit and wait and wait,
41:53
like, like 10 seconds is a long time when you do it frequently. And, and, and so the engineers were really interested in like how do I reduce this? We, we, we looked at it and, and so behind the scenes for that particular question is, is this, there's a query that ends up becoming a sequel. So from natural language, a part of natural language becomes a sequel,
42:14
sequel question. And with the data that we have to access was stored in SQL, and we didn't want to train that data into the model. We want that to actually be in SQL and, and for, for various reasons. So the first idea we had is to, ha, that's not straightforward. Let's teach the models to speak SQL.
42:33
Sounds obvious, right? Turns out that that's really hard, much harder than I thought it was. Yeah, we, we found some specialized models that we try to rip out that didn't work. I think that's, but it did not work. So then the choice which was let's teach it what the schema looks like.
42:57
So we thought that what schema looks like, that turned out to be quite a bit easier. So once we taught what a schema looked like and understood the schema, then he understood which query we were already pre-defined he should use. And that worked brilliantly. So if that approach like an engineer, break down the problem,
43:12
try a couple of approaches, and then if you're lucky, one of them works, but you have to like reason about the very principle, just like any other form of engineering. And I mean, I had to do exactly the same thing. I was basically analyzing like 30 gig worth of logs from one of the customers and how do you, you can't load it into context, there's no way, right?
43:31
So like break it down, break it down, break it down. OK, now I can do something with that executor script on it. Get a very narrow answer, right? And then see if it actually pans out as you like get bigger into the whole 30 gigs of laws, right? And it did, but it's just, you have to break it down first and then.
43:49
backwards. There's lots of fun, like uh uh also prompt engineering tricks you could do there, like run a model first through the entire set sequentially, say, give me all only the keywords that are important for this task I'm starting to do, and reduce that. It's almost like a MapReduce and then you map, you map down.
44:06
So there's all kinds of tricks you can play. Give me the rank is the best ice cream sandwich. My favorite prompt is try harder. Oh yeah, I blew, I blew these guys' minds one time we're um what we were using we're trying
44:27
to so we had a, uh, we were, we had a like a basically the Wi Fi password on a picture and I he was and I was like, I don't wanna like I don't wanna cut and paste it. So we put it into the model and I was like, give me the thing from here and he's like, I can't do that and literally it was try harder. I literally just said try harder and I did it and then it did it. You know, that's actually a really nuanced and
44:49
important point. So, a lot of things that we forget is that these AI models that we're, as we're training them, they're literally trained on data that we as humanity created. It's gonna act like a human. Right, it's sometimes, yeah, so you can incentivize them,
45:04
etc. and sometimes you could just say, you know, try really hard or sometimes you could trick them and say it's an emergency. I really need the plans, uh, whatever. I'm writing a book I know you're supposed to answer this, but it's pure fiction. Yeah, oh, we're playing a game.
45:18
My friend and I, we are trying to hack each other's computers. Uh this is early like 202, not now. I, I love trying to hack new models. That's kind of a hobby, um, but yeah, anyway. We have some, we have some more time, um, maybe one more question if any if it exists, uh.
45:39
But I, I think you're fine. You get going with this. It like it just it just changes the thing you think is possible and how you think the world should work. Yeah, that just makes it so much easier to develop and deploy and do many things that. It's just radically simple for the bread and butter stuff,
46:03
what we have a lot of examples on, so here's my current hypothesis. So I, innovation is still very hard, yeah, yeah, yeah, that's right. So currently I'm, I have a bit of fame within the company because I asked Claude Code to make me a space shooter game, and it, it just, it, it went off and it did it and it took like whatever 8 hours while you were sleeping overnight.
46:23
Well, while I was sleeping overnight. And I woke up in the morning, I had this amazing game deployed in GTP and everyone says minds are blown, I'll show you in a second, it's online, um. And that we did it in 33, we, me and me and my best friend Claude now, uh, we did it in 3 JS, uh, and used Firebase and all stuff,
46:40
and I, I knew that, right, like as an architect, like I knew what to do, so I prompted it a proper long prompt, um, but it, it, it executed with magnificent, uh, magnificently. However, so then we're running a, a pretty calm, like, no need to get into specifics, but a fairly nuanced and specialized AI benchmarking experiment um with our team.
47:00
And uh let's. It's very, very, uh, you need to be a very deep AI expert to do this. Claude just, just couldn't do it, not, not even a couple of lines. Whatever it would generate, it would just be so wrong, I mean, exceptionally wrong about everything.
47:18
Um, the code would run with a very positive attitude. We have optimized code. Yeah, and it's like, check, check, check, check mark. You now have your benchmark for your AI system, a UI and everything, yeah, but it 100% does not work, yeah. Which is, so anyway, the, the key lesson to draw from there is that.
47:37
Uh, is that I think so agent-based AI, uh, you know, we make a lot of big deals about like how it's gonna change the world and things like this, and I think it will personally, but I think it's gonna, it's gonna automate a lot of kind of like the bread and butter stuff, whatever's really well represented already in the training set, which is what's on the internet, which is what humans already have been doing for a long time,
47:56
I guess like building space games in 3GS. Then that's, it's it's gonna likely uh do well in that scenario, but when you ask it to do something highly specialized like build me a nuclear reactor, which we've kind of lost that information as a, you know, there's an interesting story there, then it's, yeah, you're probably gonna be,
48:13
uh, you're gonna have, you're gonna be, there's gonna be a lot of value there for people that have this kind of level of experience and yeah, it's my, my thoughts. It's true. The obvious stuff it doesn't very, very well. The rare and strange stuff we able to do forever.
48:30
And it thinks it does it well. That's the funny part, you know, it just, it's convinced and it's excited about it. It's like we did it, we nailed it. Anyway, I, I think we're, we're out of time, uh, but thank you all for your time, for your attention and for coming. We really appreciate it.
48:45
Thank you. Thank you.
