Class ByteLevelDecoder

Converts byte-level characters to unicode characters, then concat them into a single string.

Inheritance

object

ByteLevelDecoder

Implements

IDecoder

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Namespace: Unity.InferenceEngine.Tokenization.Decoders

Assembly: Unity.InferenceEngine.Tokenization.dll

Syntax

public class ByteLevelDecoder : IDecoder

Constructors

ByteLevelDecoder()

Initializes a new instance of the ByteLevelDecoder type.

Declaration

public ByteLevelDecoder()

Methods

Decode(IReadOnlyList<string>, Output<string>)

Applies modifications to the input detokenized strings.

Declaration

public void Decode(IReadOnlyList<string> tokens, Output<string> output)

Parameters

Type	Name	Description
IReadOnlyList<string>	tokens	The string values to modify.
Output<string>	output	The recipient of modified strings.

Implements

IDecoder