Final input composition. Following the example in Fig. 4, 20% of the masked values are replaced by random magnitudes while changing the “1”s in the attention mask vector to “0”s. Similarly, we make another 20% of the masking visible, keeping the actual observations. Doted-line squares indicate both random and real observations. At this point, we should keep a second mask containing the initial 50% masked values to evaluate the loss function.