Advanced AWS Glue test (Advanced)

Sample questions

5 sample questions from the Advanced AWS Glue test, with the answers. Samples are taken out of the live test, so you will not meet these when you sit it.

1. Which of the following frameworks is most focused on efficient metadata management, including schema evolution and time travel capabilities, in large-scale data lakes?

   1. Apache Iceberg
   2. Apache Hudi
   3. Glue Data Quality
   4. Lake Formation

   Answer: A. Apache Iceberg

   Apache Iceberg is designed for efficient metadata management in large-scale data lakes, supporting schema evolution, time travel, and ACID transactions. Hudi also supports metadata management but is not as focused on this aspect as Iceberg. Glue Data Quality is a separate service for data validation, and Lake Formation is used for data governance and permissions.

2. Which statement accurately describes the behavior of job bookmarks in AWS Glue?

   1. Job bookmarks are automatically disabled for all Glue jobs and must be manually enabled by the user.
   2. Job bookmarks are not used in Glue jobs and have no impact on performance or data processing.
   3. Job bookmarks are automatically enabled for all Glue jobs and track the progress of data processing to avoid reprocessing the same data.
   4. Job bookmarks are only enabled for Glue jobs that use the G.1X worker type and are manually configured by the user.

   Answer: C. Job bookmarks are automatically enabled for all Glue jobs and track the progress of data processing to avoid reprocessing the same data.

   Job bookmarks are indeed automatically enabled for all Glue jobs and are used to track the progress of data processing. This ensures that data is not reprocessed unnecessarily, improving performance and reducing costs. The other options either describe incorrect behaviors or are not directly related to the internal workings of job bookmarks.

3. Which AWS Glue feature enables users to define and enforce data quality rules to ensure that data meets specific standards before being processed further?

   1. Glue Data Quality
   2. Hudi metadata compression
   3. Lake Formation data classification
   4. Iceberg metadata compaction

   Answer: A. Glue Data Quality

   AWS Glue Data Quality is a feature that allows users to define and enforce data quality rules on their data pipelines, ensuring that data meets specific standards before it is processed further. This is a key component of advanced AWS Glue usage, as it helps maintain data integrity and reliability.

4. How does an AWS Glue ETL job scale automatically when processing large datasets?

   1. The job will not scale automatically and requires manual configuration for scaling.
   2. The job will scale by increasing the number of executors based on the number of transformations in the script.
   3. The job will scale by increasing the number of executors based on the size of the data partition.
   4. The job will scale by increasing the number of executors based on the number of files in the input.

   Answer: C. The job will scale by increasing the number of executors based on the size of the data partition.

   AWS Glue ETL jobs scale automatically by increasing the number of executors based on the size of the data partitions. This behavior is managed by the Glue service, which dynamically allocates resources to handle larger workloads efficiently.

5. Which statement best describes DPU calculation in AWS Glue and how it compares to EC2 spot instances, and when to choose each?

   1. DPU calculation in AWS Glue is based on the amount of data processed and the complexity of the transformations, while EC2 spot instances are billed based on the instance type and runtime. Choose Glue for serverless, scalable ETL jobs and EC2 for predictable, long-running workloads.
   2. DPU calculation in AWS Glue is based on the instance type and runtime, while EC2 spot instances are billed based on the amount of data processed. Choose Glue for long-running workloads and EC2 for scalable ETL jobs.
   3. DPU calculation in AWS Glue is based on the number of nodes used, while EC2 spot instances are billed based on the instance type and runtime. Choose Glue for scalable ETL jobs and EC2 for predictable, long-running workloads.
   4. DPU calculation in AWS Glue is based on the amount of data processed and the complexity of the transformations, while EC2 spot instances are billed based on the instance type and runtime. Choose Glue for predictable, long-running workloads and EC2 for scalable ETL jobs.

   Answer: A. DPU calculation in AWS Glue is based on the amount of data processed and the complexity of the transformations, while EC2 spot instances are billed based on the instance type and runtime. Choose Glue for serverless, scalable ETL jobs and EC2 for predictable, long-running workloads.

   AWS Glue uses DPU (Data Processing Unit) to measure the compute capacity required for a job, and it is calculated based on the data processed and the complexity of the job. EC2 spot instances, on the other hand, are billed based on the instance type and the time they run. Glue is ideal for ETL tasks that require scalability and are short-lived, while EC2 is better suited for long-running, predictable workloads.