Skip to Content
16:41 Video

Data Driven High Performance Containers

This track will demonstrate how we can successfully implement high performance, data-driven containerized workloads that provide frictionless access to datasets while considering both scale, performance and access.
Click to View Transcript
00:01
Mhm Welcome. My name is Robert Test Thomas, and today I'd like to talk about how we can successfully implement high performance data deliver and containerised workloads that provide frictionless access to datasets while maintaining speed and simplicity at scale. High performance computing, traditional HPC, ai or analytics.
00:31
Often the first thing that comes to mind is a data centre room filled with rack upon rack of interconnected systems, all dedicated and tuned to a specific workload. These platforms are used to solve complex problems the next generation vaccine, Discovery Autonomous driving all the interim security on the trick. But whatever the use case, these environments have generally been managed by their own team
00:58
with their own budget, an HPC cider with identifies it. This is because these environments are linked to special projects that are high on the must be success on list of the C suite. But with the advent of new C suite directives such as S G and maturing, new technologies decided approach is slowly coming to an end.
01:24
Containers and kubernetes have become de facto tools for building, deploying, running and managing modern enterprise applications at scale often referred to as cloud native applications, they enable faster innovation and reliable delivery, identify software while using more resources more efficiently and reducing costs. Getting on demand Access to a shared compute resource pool has never been easier,
01:53
as demonstrated by child service providers. Packaging software in container images and running containers has become standard, and Kubernetes has become the most widely used container and resource orchestrator in the Fortune 500 over the past few years. High Performance Computing Bayh Aye and analytics have shifted from the specialist nature approach to the adoption of the more
02:20
mature of humanities and container technology. Simply checked in NGC catalogue to find containers for AI data science software, HPC Applications and Data Analytics, this clearly shows the wide reduction of these technologies on modern cloud scale like architectures. And with most organisations running large communities clusters for their application
02:43
microservices, they could add a few specialist GP notes, AI and scheduled performance drugs to soak up available resources when possible. Because kubernetes and containers are flexible and general purpose, the same infrastructure should in theory, supports all these different workers. This change in the underlying orchestration technology shifts requirements from fixed
03:10
compute capacity to elastic allocation and the allocation of resources capacity bit for enterprise or high performance workloads can no longer be planned. Years in advance storage requirements keep increasing. In a world where yesterday's data may be tomorrow's gold enterprises are faced with the conundrum on how to provide access to as many datasets as possible,
03:34
all while maintaining performance and costs. Hamlet would spin this as to delete or not to delete. That is the question. It is well known that HBC ai and analytics, a data driven workers delivering modern interference capabilities to enable operation automation predictions and recommendations that
03:55
augment human stuff. These modern insights gained go hand in hand with the quantity and quality of the data sets available, and this relation defines that agility provided to users to consume. Compute resources brings value with their ability to access to acquire data sets. So it is not simply a problem of story space, but one of accessibility and performance that
04:20
does not hinder the user experience as the one scale. Keep in mind that these high performance workloads require a storage platform that delivers predictable local agency as concurrency increases. By this, I mean, if a data processing job takes one minutes, then running the same job 10 times in parallel should show little to no deviation from that
04:41
one minute time to result for each of the 10 jobs. Whereas if each of the parallel jobs once one increases, then the storage platform is not or less consistent at scale, with a constant data growth and ever evolving methods to distribute to compute workloads, the shift from the traditional scale up compute legacy storage to a distributed,
05:05
compute and desegregated scalable fast object storage model is required within the industry. This shift can be seen in technologies such as spokesman store to get your mood, elastic, suitable snapshots and object data warehouses. They all clearly show this trend, and we have great sessions covering those technologies to check out. Yeah, I think Yang, presenting open analytics
05:30
as a service field, sparked Data warehouse and Yugoslav kuriansky presenting Log Analytics as a service with Elasticsearch. But you may be wondering, How can this really be considered? High performance architecture isn't as free for archive, and it is true that, as he is often linked to a upload and forget mentality, especially when we consider the
05:51
fight the scale of Donald per gigabyte Price for schooling data on the street. So the assumption that SVS archive could seem correct but the pure We foresaw the adoption of object protocol within the distributed computing ecosystem and hence created a new technology. With this in mind, the industry's first unified fast fall and object platform.
06:14
That's great. The object protocol from a high performance computing angle has the benefit of being directly accessible from within the code of each job. This allows more freedom to define storage actions such as parallelism, handling of metadata or batch processing groups of objects, all enabling the end user to reduce financial results.
06:38
He didn't use a space data path provided by the SP. Protocol has performance in the following ways. Firstly, more connections in parallel can benefit from distributed resources, whereas using a mounted filesystem limits to a single connection. Paramount. Secondly, potentially lower agency.
06:56
Since the data set, requests don't need to cross the user colonel boundary four times and lastly, simpler code with fewer dependencies and unused complexity that kernel filesystem clients convinced the sandals. If one of your use cases requires the familiarity of the fold, infrastructure with the shift from farm to object is too cumbersome.
07:17
Flash Made also provides the same predictable, low latent heat scale over farm protocols, such as in efforts as mentioned, it is the industry's first unified fast file and object platform. So how do we go about adopting a distributed compute and this aggregated, scalable object storage approach within our data? Different performance container is architecture.
07:45
It was first taken up and what can be considered the legacy high performance architecture. It is often one of the multiple interconnected nodes that provide both the distributed compute and storage layer or multiple interconnected notes split into two layers, one for distributed compute and one for school age. This is often the approach taken with most legacy global distributed file systems,
08:10
these architectures attuned to a given iron profile, and then the Io profile changes the platform's fare. Worse to the terms of an acceptable level of performance. We need to reach you in the entire platform, all while providing a less than adequate service level for demanding users. To counter this, we could use multiple silos of storage with collections of notes,
08:30
implementing dense and slow, small and fast and so on. But that has obvious problems. The main one being complexity and cost. Then there is the question of stability. These architectures, due to their complexity, have more pieces that can fail and routine maintenance with us and invest. Client catching is often,
08:49
of course, so the complexity involved the constant care required to maintain consistent performance and stability. In addition to capacity planning and workers' revolution, Problematic have tested the limits of such architectures and are strong headwinds when it comes to implementing as simple as a service for French. I'm not saying the architecture does not have
09:11
its own shoes case, but I would think twice about dedicating a significant cost of ownership to a specific task that may become obsolete over time, part of implementing such legacy architectures. So I hear you saying, Just have communities and by doing so, we enter a more as a service computer model. It's true,
09:33
but we also need to accommodate storage, persistence and shared access to the data sets. This can be achieved in part by a legacy. Distributed storage, too, with all its inherent problems as previously described, or we can implement shared access to a legacy storage system. We now have a high performance container. Most compute platform with shared data access.
09:55
But when using legacy systems to fulfil the requirements of data different workloads, we encounter issues relating to scale and performance they get. The storage systems were not created to handle millions or billions of files and objects, all being accessed simultaneously across diverse bio fulfils. These systems fall short when handling the scale and variation of io profiles BDs
10:18
sequential or random, large or small read, write or metadata intensive. In addition to this, the accumulated more bandwidth and I O distribution required when running multiple workloads across thousands of nodes simply disqualifies anyone scared out legacy storage system from the money. Then we have the use of legacy protesters, and that is,
10:40
I mean old long object physicals. These legacy protocols creates added complexity, from requiring in guest clients or drivers to maintaining access rules to each dataset. What an apps don't need the machinery of file systems. They often run their own microservices for cataloguing metadata management,
10:58
defence occasion and permissions. All the folder hierarchies usually just get in the way, and the simple key value interface is more of what the application wants anyway. Taken offence, for example, to provide access to a date set. I need to know the share name the date set in question the client system to create the
11:17
correct export and miracles, and on top of that, user permissions will need to be validated. None of this is very as a service. It is also tonight they're using current drivers. In this case, for NFS, it complicates the container model, Since we now need to manage host Mount Points and bind mountain into the container,
11:36
this additional plumbing slows down, start and restart operations. It's easier and faster to access storage in user space from within the container. Adopting an object trust mindset will simplify the equation and enable a data driven by performance container architecture. Within such an architecture, the main data sets are presented by s free from flash wait to
12:03
address the requirement of fast shared access at scale shared access by knowing the bucket name the s freakin secret directly from within the code of the distributed workloads. No added clients or driver overheads. No cumbersome protocol management are Users can work on any day set available to in regards scale, flash floods, ability to handle banking committee, massive scale, all while maintaining predictable low
12:30
latency enables this architecture and did I mention effortless management? And the single chassis was seven blades all the way Up to 10 chassis is 150 blades. The storage presented will go hand in hand with your workouts. This seamless implementation of performance at scale will shared access is critical to enabling data. Government microphones,
12:50
containers. We often forget that a fully operational application is more than just distributed workloads. We may want to start drug submitted from a Web front end and push the output to a database that is used to populate dashboards and Giovanna. This is where our port works. Software defined storage layer.
13:08
The kubernetes fits providing finance. If I speeches, such as high availability of personal failure are your profile matching replication and encryption, the poor were clear simplifies the day to day provisioning of storage within communities. Our data, driven by performance container architecture, leverages, community storage, the service orchestrated by poor works and
13:31
consistent performance of scale to object data sets by a flashlight in order to deliver on the modern data requirements. Flash rate has the following characteristics multidimensional performance. The ability to live a very high throughput and performance to support multiple workers simultaneously with any faster size. It's small,
13:54
large sequential than my own batch or real time jobs and, of course, large number of funds. Intelligent architecture means that the story system is built for flash from the ground up to truly leverage the performance and efficiencies of first. It is also simple to deploy and manage an update without requiring constant tuning.
14:15
Modern storage solution not speak simple enough. The day to day operations don't overwhelm the storage teams, and maintenance operations, software upgrades and capacity expansions are completed without disruption. Cloud ready refers to cloud like agility, flexibility and consumption choices within from control, plus the option to operate at the edge and work directly with cloud deployments.
14:39
Always available is the capability of going beyond traditional platform resilience. It is foundational software design that makes it possible for solution to deliver high availability over multiple years and upgrade scenarios. Dynamic scalability refers to the ability to seamlessly scale not only capacity but also performance and multi protocol. Support means that a single platform should
15:02
provide native file and native object protocol support without compromising performance or any functionality. The bulwarks the main benefits are increased application density. Her kubernetes working out a consistent data plane that enables you to achieve zero rto and the sub one minute after the disaster. Recovery of Mission. Critical Data Services The ability to modernise
15:31
more applications, enterprise grade storage features. Remove the barriers against migrating certain workloads to a community's platform and the freedom to deploy on any platform. You are no longer a lot into your kubernetes distribution for your cloud provider, and here are some chilled storage customer statements you can see they cover a wide
15:56
range of benefits from reduced energy consumption to simplify management with non disruptive upgrades all the way to reduce time to results for the actual work trucks. Thousands of customers have experienced meaningful, impacted their business outcomes by moving to a pure storage solution. The sessions. Key takeaways.
16:16
I'll embrace an object class mindset. Flash Made is proven to provide speed and some positive scale as well by by high performance by close and port works enabled storage orchestration First, seamless kubernetes experience have a great time to accelerate and thank you
  • Artificial Intelligence
  • Portworx
  • Video
  • Modern Analytics
  • Pure//Accelerate
  • FlashBlade//S

High performance workloads (AI, Analytics, Data science and HPC) are slowly migrating from bespoke silos of IT infrastructure to a more global enterprise as a service architecture requiring seamless access to datasets. The session will focus on the move from siloed architectures to the adoption of containers and K8s in the high performance workload community and then link to Portworx and FlashBlade as the underlying foundation for an enterprise ready deployment at scale.

Continue Watching
We hope you found this preview valuable. To continue watching this video please provide your information below.
This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
CONTACT US
Meet with an Expert

Let’s talk. Book a 1:1 meeting with one of our experts to discuss your specific needs.

Questions, Comments?

Have a question or comment about Pure products or certifications?  We’re here to help.

Schedule a Demo

Schedule a live demo and see for yourself how Pure can help transform your data into powerful outcomes. 

Call Sales: 800-976-6494

Mediapr@purestorage.com

 

Pure Storage, Inc.

2555 Augustine Dr.

Santa Clara, CA 95054

800-379-7873 (general info)

info@purestorage.com

CLOSE
Your Browser Is No Longer Supported!

Older browsers often represent security risks. In order to deliver the best possible experience when using our site, please update to any of these latest browsers.