PyTorch test (Intermediate)

Sample questions

5 sample questions from the PyTorch test, with the answers. Samples are taken out of the live test, so you will not meet these when you sit it.

1. Which of the following statements best describes the relationship between in-place operations and autograd in PyTorch?

   1. Using in-place operations on tensors that require gradient tracking can lead to errors during the backward pass, as autograd cannot track changes to the tensor's values properly.
   2. In-place operations are always safe to use with autograd as long as the tensor is not part of a computation graph.
   3. In-place operations are only problematic when used with non-floating point tensors.
   4. In-place operations are preferred over non-in-place operations when using autograd because they improve memory efficiency.

   Answer: A. Using in-place operations on tensors that require gradient tracking can lead to errors during the backward pass, as autograd cannot track changes to the tensor's values properly.

   In-place operations on tensors that require gradient tracking can disrupt the computation graph that autograd relies on, leading to errors during the backward pass. Therefore, it is generally recommended to avoid in-place operations when using autograd. Other options may describe valid scenarios or consequences, but they do not directly address the issue with in-place operations and autograd.

2. In a realistic project scenario, how would you use torch.jit.trace to convert a PyTorch model into a TorchScript module?

   1. Use torch.jit.trace to define the model's architecture and initialize its parameters.
   2. Use torch.jit.trace to capture the model's execution and generate a TorchScript module.
   3. Use torch.jit.script to trace the model's execution and generate a TorchScript module.
   4. Use torch.jit.trace to save the model's parameters and state_dict for later use.

   Answer: B. Use torch.jit.trace to capture the model's execution and generate a TorchScript module.

   torch.jit.trace is used to trace a model's execution and convert it into a TorchScript module, which can be deployed in production environments. This method records the operations performed during a forward pass and generates a script module that can be used independently of Python. Other options either describe incorrect uses of TorchScript or are unrelated to tracing.

3. What is a key requirement for successfully tracing a model with torch.jit.trace in PyTorch?

   1. The model's input size must be fixed, and the model must not contain any dynamic control flow (e.g., loops with varying lengths or conditionals that depend on input data).
   2. TorchScript tracing automatically adapts to varying input sizes and dynamic control flow.
   3. The model must have dynamic control flow, and the input size must be fixed during tracing.
   4. The model can have dynamic control flow, and the input size can vary during tracing.

   Answer: A. The model's input size must be fixed, and the model must not contain any dynamic control flow (e.g., loops with varying lengths or conditionals that depend on input data).

   When using torch.jit.trace to trace a model, the input size must be fixed because tracing records operations based on a specific input example. Additionally, dynamic control flow (like loops with varying lengths or conditionals dependent on input) cannot be properly captured by tracing and may cause errors. This is why the correct option highlights these limitations.

4. A PyTorch model with a custom layer is scripted using TorchScript, but it fails to run. What is the most likely cause of this issue?

   1. The model fails to run in TorchScript because the custom layer does not have a defined forward() method.
   2. The model fails to run in TorchScript because the custom layer uses Python scripting instead of TorchScript.
   3. The model fails to run in TorchScript because the custom layer does not have a defined __init__() method.
   4. The model fails to run in TorchScript because the custom layer is not a subclass of nn.Module.

   Answer: A. The model fails to run in TorchScript because the custom layer does not have a defined forward() method.

   TorchScript requires all custom layers to have a defined forward() method for proper scripting. If this method is missing, TorchScript cannot trace or script the model, leading to runtime errors. The other options do not directly relate to the core requirement for TorchScript compatibility.

5. Which of the following is a correct and essential requirement for the forward method in a custom nn.Module?

   1. The forward method must return the output tensor(s) of the layer.
   2. The forward method must include gradient calculation.
   3. The forward method must be explicitly called during training.
   4. The forward method must be defined only for custom layers.

   Answer: A. The forward method must return the output tensor(s) of the layer.

   The forward method in nn.Module defines the computation performed at every call. It must return the output tensor(s) of the layer. Other options may describe valid practices or related concepts, but they are not the essential requirement for the forward method.