Astromer’s multi-head attention mechanism. The light curve consists of magnitudes and timestamps (in MJDs). The magnitude values are processed through a FF neural network, while the timestamps of each observation are encoded using the positional encoder. The two representations are summed to form the input representation, which is then fed into the attention mechanism. This representation is projected into query (Q), key (K), and value (V) matrices using learned weight matrices. The attention mechanism computes attention weights (α_i) using a softmax function applied to a scaled dot-product operation of the query and key vectors. The final multi-head attention output consists of multiple attention heads [h₁, . . ., h_#heads] concatenated and then transformed by a learned weight matrix. This transformed output is then used as the input representation for the next multi-head attention layer.