AttentionLayer

AttentionLayer[] represents a trainable net layer that learns to pay attention to certain portions of its input.

AttentionLayer[net] specifies a particular net to give scores for portions of the input.

AttentionLayer[net, opts] includes options for weight normalization, masking and other parameters.

Examples

Create an attention layer:

AttentionLayer[]

Please visit the official Wolfram Language Reference for more details.