How to Tame Time Series Data with ElasticSearch and Pure Storage

When designing an ILM strategy, organizations need to consider factors such as data retention requirements, search performance, and efficiency for different data tiers. Time series data like security event logs, financial data and more, is an additive log data style that is rarely updated, and can quickly become unmanageable without a process to manage growth. With ElasticSearch Data Streams, the cluster can manage time series indices automatically and route searches to all backing indices, improving search efficiency. When optimizing index storage in the cluster, you will likely want to expand the optimization to the entire cluster to support your ILM policy. Elastic suggests combining the use of hot tier data with searchable snapshots on S3 with attention to performance and simplicity at scale. Pure Storage FlashBlade//S was built on the idea that modern data solutions like ElasticSearch require a highly parallelized and concurrent platform that can easily scale. In this talk, we will share how by leveraging ElasticSearch Data Streams, ILM and FlashBlade S3, you can search time series data efficiently and scale easily with the growth of your ElasticSearch cluster.