Class SplitDelimiterContiguous
Aggregates all the successive delimiters into a single split and keeps all the content splits as is.
Inherited Members
Namespace: Unity.InferenceEngine.Tokenization.SplitDelimiterBehaviors
Assembly: Unity.InferenceEngine.Tokenization.dll
Syntax
public class SplitDelimiterContiguousProperties
Instance
Gets a shared instance of the SplitDelimiterContiguous.
Declaration
public static SplitDelimiterContiguous Instance { get; }Property Value
| Type | Description |
|---|---|
| SplitDelimiterContiguous |
Methods
Apply(SubString, IReadOnlyList<(Range offsets, bool isMatch)>, Output<SubString>)
Applies the delimiter behavior to the list of splits and stores the result into the
specified output.
Declaration
public void Apply(SubString source, IReadOnlyList<(Range offsets, bool isMatch)> splits, Output<SubString> output)Parameters
| Type | Name | Description |
|---|---|---|
| SubString | source | The source string of the splits. |
| IReadOnlyList<(Range offsets, bool isMatch)> | splits | The list of splits, indicating of the split if a content, or a delimiter. |
| Output<SubString> | output | The target list where updated splits are added. |