Neural network architectures. Both the dynamics model and the policy NN take same input: concatenations of past actions and observations. They also share the same fully convolutional structure in the first layers. At the output layer, the policy model includes the KL-filtering scheme (upper right corner) and the dynamics model output is multiplied with the WFS mask (lower right corner). See Sect. 5.2 for details.