An ML System is a comprehensive infrastructure supporting the entire lifecycle of machine learning models, including development, deployment, monitoring, and continuous improvement. According to Google scientists in their paper “Hidden Technical Debt in Machine Learning Systems,” ML systems are intricate, interconnected structures consisting of various components critical for ensuring model reliability, adaptability to changes, and sustained impact in production environments.
1. Configuration
Purpose: Defines parameters, settings, and system requirements for the model and its environment.
Role: Ensures that all elements—like feature settings, thresholds, and tuning parameters—are consistent and well-documented across environments.
2. Data Collection
Purpose: Gathers raw data from various sources that will be used for training, validating, and testing the ML model.
Role: It’s foundational for the ML model’s accuracy and relevance. Effective data collection includes pipelines and tooling to gather, clean, and preprocess data.
3. Feature Extraction
Purpose: Transforms raw data into meaningful features used by the model.
Role: Helps the model make accurate predictions. It requires consistency across environments, ensuring the same process applies in both training and production.
4. Data Verification
Purpose: Checks data quality to ensure consistency, accuracy, and reliability.
Role: Catches data issues like missing values, outliers, and incorrect labels before they affect model performance, maintaining data integrity across the pipeline.
5. ML Code
Purpose: The actual model code that learns from data and makes predictions.
Role: This includes training algorithms, prediction logic, and optimization functions, forming the “core” of the ML system.
6. Machine Resource Management
Purpose: Manages computational resources like GPUs and CPUs required for model training and deployment.
Role: Ensures efficient use of resources, especially in large-scale environments, controlling costs and managing load for smooth model operation.
7. Analysis Tools
Purpose: Provide insights into model performance and behavior.
Role: Used for debugging, assessing feature importance, and validating results. They help refine models and diagnose issues throughout the ML lifecycle.
8. Process Management Tools
Purpose: Orchestrate different components in the ML pipeline, including data preprocessing, model training, validation, and deployment.
Role: Automates repetitive processes and schedules jobs, often with CI/CD pipelines. Examples include Apache Airflow, Kubeflow, and MLflow.
9. Serving Infrastructure
Purpose: Hosts and serves the model in production to make predictions available to users or applications.
Role: Includes REST APIs or streaming services to deliver predictions in real-time or batch, such as using AWS SageMaker or Google’s AI Platform.
10. Monitoring
Purpose: Continuously tracks model performance, data drift, and system health after deployment.
Role: Detects issues like prediction errors, data changes, and latency spikes to trigger alerts for retraining, redeployment, or further investigation.