Use case: Metadata Instance Modifier

In order to support large models, the number of meshes in a given scene cannot be too high. This is handled by processing models with a transformation workflow that merges adjacent meshes together into batches, which are in turn merged into progressively larger batches. This process repeats until the model can be represented by a single mesh. These merged meshes are referred to as HLODs, because they are organized together in a Hierarchical tree structure and represent varying Levels Of Detail of the original meshes.

The information about the original meshes is preserved by associating an InstanceId to each one and mapping them to a texture that can be projected onto the merged meshes. This can be used to determine which object is represented by a given pixel on the screen, regardless of which HLOD contains the object in its merged mesh. This can also be done in reverse, by using a given InstanceId to determine which parts of the meshes in the scene represent a given object.

Instance Modifier and Updater

An InstanceModifier can be used to apply a highlight or change the visibility of a group of instances. This can be either automatic while objects are streamed into the scene, or based on user input (ex: in response to clicking on an object). Every InstanceModifier contains an InstanceUpdater, which can be used as follows:

ModelStreamId modelStreamId;
IEnumerable<InstanceId> instanceIds;
Color32 highlightColor;
bool isVisible;

InstanceUpdater.SetHighlight(modelStreamId, instanceIds, highlightColor);
InstanceUpdater.SetVisibility(modelStreamId, instanceIds, isVisible);

Note

These changes are not applied immediately; they are batched together and applied before the following frame.

Async Overrides

public virtual Task LoadAsync(ModelStreamId modelStreamId, 
    IEnumerable<InstanceGeometricErrorState> states)
public virtual Task UpdateAsync(ModelStreamId modelStreamId, 
    IEnumerable<InstanceGeometricErrorState> states)
public virtual Task UnloadAsync(ModelStreamId modelStreamId, 
    IEnumerable<InstanceGeometricErrorState> states)

These functions can be overridden to apply modifiers to specific groups of instances. They are called internally during the streaming process for batches of instances.

• As new sections of the model are loading, the InstanceModifier will be informed about newly discovered instances via the LoadAsync method.
• As the framework moves to more detailed versions of objects, the states of existing instances may need to be updated for newly discovered levels of detail. Those instances can be updated using the UpdateAsync method.
• During this process, the framework may consider some instances to no longer be relevant and will inform the modifier via the UnloadAsync method.

The following example will apply a blue highlight to each instance as it is loaded into the scene.

Color32 highlightColor = new Color32(0, 0, 255, 255);

public override Task LoadAsync(ModelStreamId modelStreamId, 
    IEnumerable<InstanceGeometricErrorState> states)
{
    var instanceIds = states.Select(x => x.InstanceId);
    InstanceUpdater.SetHighlight(modelStreamId, instanceIds, highlightColor);
}

This next example will highlight each instance with a random color. Note that the color applied to each instance is cached in a Dictionary so that the color remains consistent later, when a different Mesh containing the same instance could be loaded.

Dictionary<InstanceId, Color32> highlightColors = new();

Color32 GetHighlightColor(InstanceId instanceId)
{
    if (!highlightColors.TryGetValue(instanceId, out var highlightColor)
    {
        highlightColor = new Color32(Random.Range(0, 255), 
            Random.Range(0, 255), Random.Range(0, 255), 255);
        highlightColors.Add(instanceId, highlightColor);
    }
    return highlightColor;
}

public override Task LoadAsync(ModelStreamId modelStreamId, 
    IEnumerable<InstanceGeometricErrorState> states)
{
    foreach (var instanceId in states.Select(x => x.InstanceId))
    {
        highlightColor = GetHighlightColor(instanceId);
        InstanceUpdater.SetHighlight(modelStreamId, new[] { instanceId }, highlightColor);
    }
}

public override Task UpdateAsync(ModelStreamId modelStreamId, 
    IEnumerable<InstanceGeometricErrorState> states)
{
    foreach (var instanceId in states.Select(x => x.InstanceId))
    {
        highlightColor = GetHighlightColor(instanceId);
        InstanceUpdater.SetHighlight(modelStreamId, new[] { instanceId }, highlightColor);
    }
}

public override Task UnloadAsync(ModelStreamId modelStreamId, 
    IEnumerable<InstanceGeometricErrorState> states)
{
    foreach (var instanceId in states.Select(x => x.InstanceId))
    {
        // no need to reset the highlight since the instance is being unloaded anyway
        highlightColors.Remove(instanceId);
    }
}

Metadata Queries

Another case would be to highlight a subset of instances based on some pre-determined criteria. This section will show how to filter the instances being loaded based on metadata values associated with them.

Note

See the Set up a IMetadataRepository page of the Metadata package documentation.

MetadataRepository metadataRepository = new(...);
string[] someKey = { "SomeKey", "value" };
string someValue = "SomeValue";
InstanceId[] instanceIds = {...};

// start building a metadata query
var query = await metadataRepository.Query()   
    // where the value of someKey must match someValue
    .WhereKeyEquals(someKey, someValue)     
    // returning only the InstanceIds (excluding the actual metadata)
    .SelectOnlyId()           
    // constrained to the given InstanceIds
    .IncludedIn(instanceIds) 
    // and finally executed asynchronously    
    .ExecuteAsync();                            

When working with larger models, queries can be too costly to process on every single instance in the model. By limiting the query to a subset of instances during the loading process, the execution time is significantly reduced. This example shows how to execute a metadata query in LoadAsync with a scope limited to the most recent batch of loaded instances.

MetadataRepository metadataRepository = new(...);
string[] someKey = { "SomeKey", "value" };
string someValue = "SomeValue";
Color32 highlightColor = new Color32(0, 0, 255, 255);

public override async Task LoadAsync(ModelStreamId modelStreamId, 
    IEnumerable<InstanceGeometricErrorState> states)
{
    var instanceIds = states.Select(x => x.InstanceId).ToArray();
    
    var query = await metadataRepository.Query()
        .WhereKeyEquals(someKey, someValue)
        .SelectOnlyId()
        .IncludedIn(instanceIds)
        .ExecuteAsync();
        
    InstanceUpdater.SetHighlight(modelStreamId, query.Keys, highlightColor);
}