An extension done for diffusion models in sequential data. Assign a set for a set of frames and add noise to them accordingly.
For example,
For a given set of frames ,
We can have a fixed set of signal levels
For what means, see Flow Matching.
The point of this is to force the model to create temporal dependencies on past inputs while creating the current frame as well as learn to denoise from multiple levels.
A question I had while reading this was whether this was used as a sort of positional encoding, but thatβs done separately in most cases. Another thing to note is that the values are fixed for a fixed length of frames. Every must be applied to its corresponding frame.