Constructor SingleHeadAttention

SingleHeadAttention(string, string, string, string, string, string, string, string, string, string, float)

Creates a fused Multihead Attention layer based on the PyTorch implementation.

Declaration

public SingleHeadAttention(string name, string query, string q_weight, string q_bias, string k_weight, string k_bias, string v_weight, string v_bias, string out_weight, string out_bias, float scaling_factor)

Parameters

Type	Name	Description
string	name	The name to use for the output tensor of the layer.
string	query	The name to use for the query tensor of the layer.
string	q_weight	The name to use for the weight tensor of query.
string	q_bias	The name to use for the bias tensor of the query.
string	k_weight	The name to use for the weight tensor of key.
string	k_bias	The name to use for the bias tensor of key.
string	v_weight	The name to use for the weight tensor of value.
string	v_bias	The name to use for the bias tensor of value.
string	out_weight	The name to use for the weight tensor of the out-projection layer.
string	out_bias	The name to use for the bias tensor of the out-projection layer.
float	scaling_factor	The scaling factor for scaling the query.