Designing Cloud-Native AI Infrastructure: A Framework for High-Performance, Fault-Tolerant, and Compliant Machine Learning Pipelines

Abstract

Big Tech companies have spent the last decade designing scalable, elastic, and resilient cloud infrastructures to support their business applications. However, the emergence of machine learning as a service-area-in-silico and the urgent need for operationalizing compliance in highly regulated industries have required them to invest an additional effort in designing cloud-native infrastructures to support ML workloads. These infrastructures must provide the required elasticity and efficiency to support high-performance, fault-tolerant, blameless, and compliant ML pipelines. The principled redesign of cloud architectures to overcome the challenges of serving ML workloads at scale is essential for accelerating their maturity; however, it has yet to start in earnest. This paper contributes a framework to guide the design of cloud-native infrastructures for ML workloads that links high-level design requirements with architectural dimensions. The framework enables architecture teams to compose the design of cloud-native architectures for ML workloads by exposing the architectural trade-offs involved in configuring elasticity, performance, fault-tolerance, compliance, cost, and risk for ML workloads. We describe the design framework properties using concrete examples that optimize for elasticity, cost, and risk. Finally, we argue that the principled design of cloud architectures for ML workloads is paramount for accelerating their further adoption and maturity in enterprise environments.