How Seagate runs Real-time Analytics at the Edge
Challenge
With global data creation predicted to hit 180 zettabytes by 2025, data storage provider Seagate needed to introduce greater automation at immense scale to its operations, to ensure it could keep pace with growing demand.
Solution
Following the objectives of Industry 4.0, Seagate engineered edgeRX from the ground up — powered by multiple cloud native projects, including Kubernetes and Prometheus, it established real-time analytics at the edge via inference and machine learning, and leveraged digitalization to enable Seagate to become much more agile and efficient across its global factories.
Impact
Operational changes driven by edgeRX have fundamentally advanced Seagate operations, improving design quality while speeding time-to-value, which has had a positive impact on customer satisfaction. Seagate has seen a major return on investment, far beyond the initial costs of developing edgeRX. In addition, leveraging cloud native projects has also driven a deeper cultural transformation within Seagate, which has moved to agile ways of working and lean product development.
Projects used
By the numbers
3 Million
High-res images analyzed each day for quality control
Major ROI
Far beyond initial development costs
Customer Satisfaction
Improved quality with faster time-to-value
Seagate, a global technology leader for more than 40 years, has shipped over three billion terabytes of data capacity to-date.
From the early 1980s when it began manufacturing five-and-a-quarter inch drives, with around five megabytes of storage, Seagate has evolved to become a fully-rounded data organization, providing end-to-end solutions to store, secure and transfer data at scale.
According to Statista, by 2025, global data creation is projected to grow to more than 180 zettabytes, more than double the 79 zettabytes created in 2021. With the global proliferation of data created each day, and demand growing for secure storage, Seagate investigated further automation of its factory processes.
To manage the complexity of manufacturing storage for such colossal amounts of data, Seagate turned to cloud native projects Kubernetes and Prometheus to transform and modernize its legacy mission-critical systems across eight factories producing all of Seagate’s data storage products. Following the objectives of Industry 4.0, which aims to make manufacturing faster, more efficient and more customer-centric, Seagate deployed greater industrial automation and imaging endpoints, leveraging digitalization to become much more agile and efficient, with the goal of making continuous incremental improvements in the manufacturing process.
“The complexity of storage design now is profound. We’ve had to develop the technology that includes the ability to manufacture storage at high quantity and high quality. And that has been our challenge ever since, to support efforts to improve quality, lower costs and increase throughput.”
Roger Hall, Principal Engineer, Seagate Technology
Continual Improvement
Continual improvement in development and operations is a constant at Seagate. But to see where improvements can be made, Seagate needed to bolster its analytics and real-time data. With this in mind, it embarked on an effort across all Seagate factories to establish real-time analytics at the edge via inference and machine learning, through the Seagate edgeRX project.
Seagate deploys two systems in its factories, whose teams worked hand-in-hand to develop edgeRX from the ground up:
Seagate’s Factory Information System (FIS) is responsible for running and automating mission-critical operations across Seagate factories globally, through transformation and modernization of the legacy systems, adoption of microservices, and incremental and agile delivery to the FIS’ customers — internal stakeholders in the business.
Alongside, the Operations and Technology Advanced Analytics Group (OTAAG) assists any organization within Seagate to get more value from their data. Included in this is the ability to embed many types of analytics into processes — from descriptive visualizations to prescriptive and cognitive machine learning algorithms — enabling teams to make better sense of their data and see where incremental improvements can be made.
“One of the characteristics of Industry 4.0 is that we aim to drive more Artificial Intelligence into our day-to-day factory operations across the globe. We optimize the performance of existing processes through analytics and AI. This can only be realized through growing capability in data processing in volume, velocity, variety and veracity.”
Hamid Hadavi, Enterprise Architect, Seagate Technology
Scalability at the Edge
Scalability is a huge challenge for a company as large as Seagate. As it moved to edge computing, scalability became harder still.
“Scalability is one of the big challenges, especially when we incorporate edge and endpoints into account. Our computation paradigm, historically, has always been on core data centers, but the more we move to computation on-edge, the services we build do not necessarily enjoy all the resources that are available on the core,” said Hadavi.
Moving from legacy systems running on the core, Seagate took a phased approach to container orchestration and microservices adoption. It first introduced Docker Swarm as a container orchestration engine into two factories, slowly building throughput to a point where the more powerful Kubernetes was required to run container orchestration.
Alongside, Seagate began to redevelop its architecture, to make it more friendly to container orchestration. This necessitated a cultural transformation, from building software in the traditional sense to building software in microservices.
Seagate considered multiple solutions to their scalability challenge, including proprietary software, but chose to develop solutions in-house so that they would own the technology and create capability within Seagate.
“Cloud native computing, in particular the Kubernetes project, is very friendly to the kind of scalability that we need. We are targeting AI computation, not just in the data center, but at the edge and in the endpoint, so Kubernetes became a very attractive proposition for us. As container orchestration in general is being standardized by CNCF, cloud native projects have become the platforms that we gravitate towards,” said Hadavi.
“We needed the ability to run inference and conduct machine learning. In doing that, we also needed the capability to monitor and maintain high levels of security, and to manage the data itself. So we looked to other cloud native projects, including Prometheus, to enable us to do that,” said Hall.
Impacts
Operational changes powered by edgeRX have fundamentally advanced Seagate operations. One such example is the ability to easily share and disseminate data across AI systems for quality control. Seagate’s quality control programs leverage state-of-the-art AI to optically inspect products and detect defects on trailing edges of the recording heads of hard disk drives — failing the parts when necessary. Before staff were burdened with the arduous task of reviewing components through microscopes, now not only has accuracy from visual inspections improved, edgeRX is able to process three million high-resolution images each day.
“The intent is to try to improve quality and improve throughput by doing it this way, rather than doing this manually,” said Hall.
EdgeRX has also greatly improved the accuracy and availability of real-time analytics at the edge, to enable teams to continually improve operations. Seagate follows the statistical process control lifecycle, championed by American engineer Dr. William Edwards Deming and cited as one of the inspirations for the boom in post-war Japanese manufacturing. This focuses on better design of products to improve service; higher level of uniform product quality; and improvement of product testing in the workplace and in research centers.
“EdgeRX is a good indicator of that type of thinking and continuous improvement from a quality standpoint. From an end-user standpoint, we want to be able to improve our design quality in such a way that customers will always be satisfied with our product quality. So far, we believe we’ve been successful in doing that and, in doing so, we’ve also been able to get a fairly major return on investment that goes well beyond the initial costs of developing this capability,” said Hall.
Leveraging cloud native projects has also driven a deeper cultural transformation within Seagate, which has moved to agile ways of working and lean product development over 10-week increments.
“Adopting container technology and CNCF standards has helped us to craft an architecture that’s friendly to our methodology of delivering services to end customers. When a customer asks us to describe a service they’d like to have, that service is translated immediately to containers and pods — a one-to-one relationship to what the customer is asking. Before we were delivering in the form of monolithic apps, not microservices. There was a lot of ambiguity in terms of what services the customer was going to receive. Success for us is not just through technology adoption. It’s also through organizational alignment,” said Hadavi.