Class DefaultTruncator

Placeholder implementation of the truncation. Does not truncate, only concatenates the primary and secondary sequence of tokens.

Inheritance

object

DefaultTruncator

Implements

ITruncator

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Namespace: Unity.InferenceEngine.Tokenization.Truncators

Assembly: Unity.InferenceEngine.Tokenization.dll

Syntax

public class DefaultTruncator : ITruncator

Methods

Truncate(IReadOnlyList<Token>, IReadOnlyList<Token>, int, Output<IEnumerable<Token>>, Output<IEnumerable<Token>>)

Splits sequences of tokens into smaller collection of sequences.

Declaration

public void Truncate(IReadOnlyList<Token> inputA, IReadOnlyList<Token> inputB, int numAddedTokens, Output<IEnumerable<Token>> outputA, Output<IEnumerable<Token>> outputB)

Parameters

Type	Name	Description
IReadOnlyList<Token>	inputA	The primary sequence of tokens (mandatory).
IReadOnlyList<Token>	inputB	The optional secondary sequence of tokens.
int	numAddedTokens	The number of tokens that the IPostProcessor steps will add.
Output<IEnumerable<Token>>	outputA	The target container of the truncated subsequences of inputA.
Output<IEnumerable<Token>>	outputB	The target container of the truncated subsequences of inputB.

Implements

ITruncator