Constructor SingleHeadAttention
SingleHeadAttention(string, string, string, string, string, string, string, string, string, string, float)
Creates a fused Multihead Attention
layer based on the PyTorch implementation.
Declaration
public SingleHeadAttention(string name, string query, string q_weight, string q_bias, string k_weight, string k_bias, string v_weight, string v_bias, string out_weight, string out_bias, float scaling_factor)
Parameters
Type | Name | Description |
---|---|---|
string | name | The name to use for the output tensor of the layer. |
string | query | The name to use for the query tensor of the layer. |
string | q_weight | The name to use for the weight tensor of query. |
string | q_bias | The name to use for the bias tensor of the query. |
string | k_weight | The name to use for the weight tensor of key. |
string | k_bias | The name to use for the bias tensor of key. |
string | v_weight | The name to use for the weight tensor of value. |
string | v_bias | The name to use for the bias tensor of value. |
string | out_weight | The name to use for the weight tensor of the out-projection layer. |
string | out_bias | The name to use for the bias tensor of the out-projection layer. |
float | scaling_factor | The scaling factor for scaling the query. |