Deep Learning test (Intermediate)

Sample questions

5 sample questions from the Deep Learning test, with the answers. Samples are taken out of the live test, so you will not meet these when you sit it.

1. What is the primary purpose of a pooling operation in a convolutional neural network?

   1. It applies a non-linear transformation to the input, introducing non-linearity into the network.
   2. It computes the difference between the predicted and actual values, used in classification tasks.
   3. It reduces the spatial dimensions of the feature maps, helping to decrease computational load and control overfitting.
   4. It adjusts the learning rate during training to improve convergence and performance.

   Answer: C. It reduces the spatial dimensions of the feature maps, helping to decrease computational load and control overfitting.

   Pooling operations, such as max pooling or average pooling, reduce the spatial dimensions of feature maps by summarizing the information within small regions. This helps to decrease the computational load and control overfitting. The other options describe features of convolutional layers, activation functions, or loss functions, which are not directly related to the behavior of pooling operations.

2. Which of the following best describes a key feature of the AdamW optimiser variant in a realistic project scenario?

   1. AdamW applies weight decay directly to the gradient update, similar to standard Adam.
   2. AdamW uses a different learning rate schedule compared to standard Adam.
   3. AdamW is primarily used for reinforcement learning tasks.
   4. AdamW decouples weight decay from the gradient update, allowing for more stable and effective training.

   Answer: D. AdamW decouples weight decay from the gradient update, allowing for more stable and effective training.

   AdamW is an extension of the Adam optimiser that decouples weight decay from the gradient update. This allows for more stable and effective training, as weight decay can be applied independently of the gradient scaling. Other options may describe features of Adam or other optimisers but are not specific to AdamW.

3. Which of the following statements accurately describes the advantage of using learning rate scheduling over a fixed learning rate when training deep networks?

   1. Learning rate scheduling is preferred over a fixed learning rate when training deep networks because it allows the learning rate to decrease over time, which can help the model converge more effectively and avoid overshooting the optimal solution.
   2. Learning rate scheduling is only useful for shallow networks and not for deep learning models.
   3. Learning rate scheduling is unnecessary because modern optimizers like Adam automatically adjust the learning rate without any manual intervention.
   4. A fixed learning rate is always better for training because it ensures consistent updates to the model parameters.

   Answer: A. Learning rate scheduling is preferred over a fixed learning rate when training deep networks because it allows the learning rate to decrease over time, which can help the model converge more effectively and avoid overshooting the optimal solution.

   Learning rate scheduling dynamically adjusts the learning rate during training, which can improve model convergence and generalization. A fixed learning rate may lead to suboptimal performance or instability during training. The other options are not directly related to the concept of learning rate scheduling.

4. Which activation function is commonly used in deep neural networks due to its ability to mitigate vanishing gradients and introduce non-linearity?

   1. Sigmoid activation
   2. ReLU activation
   3. Softmax activation
   4. Linear activation

   Answer: B. ReLU activation

   ReLU (Rectified Linear Unit) is a popular activation function in deep learning because it introduces non-linearity and helps mitigate the vanishing gradient problem. It outputs the input directly if it is positive, otherwise, it outputs zero. Other activation functions like sigmoid and tanh are less commonly used in deep neural networks due to issues like vanishing gradients.

5. Which activation function is typically used in the output layer of a neural network for multi-class classification tasks?

   1. ReLU activation
   2. Tanh activation
   3. Sigmoid activation
   4. Softmax activation

   Answer: D. Softmax activation

   Softmax is used for multi-class classification problems as it converts raw scores into probabilities that sum to 1. Sigmoid is used for binary classification, ReLU is a common choice for hidden layers due to its non-linearity and ease of optimization, and Tanh is another activation function but is less commonly used in modern networks compared to ReLU.