MLOps test (Advanced)

Sample questions

5 sample questions from the MLOps test, with the answers. Samples are taken out of the live test, so you will not meet these when you sit it.

1. What is a common mistake during the MLflow experiment lifecycle management that can lead to missing metadata and versioning information?

   1. The experiment was not deployed to production.
   2. The experiment was not run on the correct tracking server.
   3. The experiment was not saved as a model artifact.
   4. The experiment was not properly logged with MLflow, leading to missing metadata and versioning information.

   Answer: D. The experiment was not properly logged with MLflow, leading to missing metadata and versioning information.

   When an MLflow experiment is not correctly logged, it can lead to missing metadata and versioning information, which makes it difficult to track and reproduce the experiment. This is a common mistake during the lifecycle management of experiments.

2. Which of the following correctly describes the difference in use cases between MLflow's experiment comparison tools and TensorBoard?

   1. MLflow is only useful for logging metrics, while TensorBoard is the only tool for comparing experiments.
   2. MLflow's experiment comparison tools are better suited for detailed model performance analysis and auditability, whereas TensorBoard is more focused on real-time visualization during training.
   3. TensorBoard is better for comparing experiments due to its built-in comparison features, while MLflow is more suited for visualization.
   4. Both MLflow and TensorBoard are equally effective for comparing experiments and visualizing training metrics.

   Answer: B. MLflow's experiment comparison tools are better suited for detailed model performance analysis and auditability, whereas TensorBoard is more focused on real-time visualization during training.

   MLflow's experiment tracking is designed for organizing, comparing, and auditing experiments, especially when working with multiple models and hyperparameters. It provides a structured way to log metrics, parameters, and artifacts, making it ideal for post-training analysis. TensorBoard, on the other hand, is optimized for real-time visualization of training metrics, graphs, and distributions, which is more useful during the training phase. Thus, the correct choice is MLflow for detailed comparison and auditability.

3. Which method is commonly used in MLOps for detecting data drift by comparing statistical properties of data distributions over time?

   1. Drift detection with machine learning models
   2. Drift detection with statistical tests
   3. Kubeflow pipeline logging
   4. MLflow tracking server

   Answer: B. Drift detection with statistical tests

   Drift detection with statistical tests is a method used to identify changes in data distributions over time, which is a fundamental aspect of MLOps for maintaining model performance. Other options may relate to drift detection, but they are not as precise or specific as statistical tests, which are a well-established technique in this domain.

4. Which method is commonly used in MLOps to detect data drift by comparing statistical properties of data from different periods?

   1. Drift detection through manual inspection
   2. Drift detection with statistical tests
   3. Drift detection using visual analysis only
   4. Drift detection with machine learning models

   Answer: B. Drift detection with statistical tests

   Drift detection with statistical tests is a method to identify changes in data distributions over time by comparing statistical properties of data from different periods. This approach is commonly used in MLOps to ensure model performance remains consistent in production. Other options, such as 'Drift detection with machine learning models,' refer to alternative approaches, but the question specifically tests knowledge of statistical drift detection methods.

5. Which of the following is a valid approach for collecting model metrics in a KServe deployment?

   1. KServe requires a separate metrics server to collect model metrics from deployed models.
   2. KServe uses a central metrics store to collect model metrics from all deployed models.
   3. KServe can collect metrics via Prometheus by exposing metrics endpoints on the InferenceService pods.
   4. KServe provides a built-in dashboard for visualizing model metrics without external tools.

   Answer: C. KServe can collect metrics via Prometheus by exposing metrics endpoints on the InferenceService pods.

   KServe supports metrics collection through integration with Prometheus by exposing metrics endpoints on the InferenceService pods. This allows for monitoring of model performance and operational metrics. The other options are incorrect because KServe does not use a central metrics store by default, does not require a separate metrics server for model metrics, and does not provide a built-in dashboard for metrics visualization.