Using Culling Callbacks for Visibility Control

Let’s imagine that we want to render a scene from the outside to get a good overview of our model. We may want to see through walls to get a good understanding of the layout (imagine a flat seen from the top, with no roof for instance). So we may want a culling method capable of filtering elements from the rendering, or from part of the rendering.

Culling callbacks available on the RED::IShape are a way to deal with any kind of customization of the culling. In that tutorial, we’ll focus on a small box with a ball inside. We’ll remove the wall just in front of us (in our line of sight) so that we can always see the ball:

The visibility filtering performed in the tutorial

In this very simple example, we define 4 quadrants (+x/-x/+y/-y) plus a top quadrant (+z). We discard the wall or roof that corresponds to our quadrant number. Then, on a camera movement, HOOPS Luminate will callback the culling method set in place and take the appropriate action regarding the visibility of the shape for which the callback is used.

Culling Callback Setup

The culling callback is easy to set. Simply call RED::IShape::SetContextSwitchCallback or RED::IShape::SetVisibilitySwitchCallback. There are however two ways to proceed:

1. The callback may be set on a node, once. In this case, the callback will be called once for the node, and the returned values will rule the visibility for all children of that node.

2. The callback is set on individual leaf shapes (the tutorial’s method) and in this case the callback is raised for each shape that has it set. In that configuration, each shape’s callback only rules its own visibility.

In our example, we set the callback on the 4 walls & roof shapes:

RED::Object* root;
RC_TEST( icamera->GetRootShape( root ) );
RED::IShape* iroot = root->As< RED::IShape >();

int i, count;
RED::Object* shape;
RC_TEST( iroot->GetChildrenCount( count ) );

for( i = 0; i < count; i++ )
{
    RC_TEST( iroot->GetChild( shape, i ) );

    if( shape->GetID() >= 0 && shape->GetID() < 5 )
    {
        RED::IShape* ishape = shape->As< RED::IShape >();

        if( g_context == true )
        {
            RC_TEST( ishape->SetVisibilitySwitchCallback( NULL, NULL, iresmgr->GetState() ) );
            RC_TEST( ishape->SetContextSwitchCallback( CullingCallback, NULL, iresmgr->GetState() ) );
        }
        else
        {
            RC_TEST( ishape->SetVisibilitySwitchCallback( CullingCallback, NULL, iresmgr->GetState() ) );
            RC_TEST( ishape->SetContextSwitchCallback( NULL, NULL, iresmgr->GetState() ) );
        }
    }
}

Context Switch or Visibility Switch Callback?

There are two callback methods on the RED::IShape: one is called the context switch callback, the other is the visibility switch callback. As explained here: Culling Methods in the Using Culling Callbacks section, each callback has a different meaning. If we press the red button in the toolbar, we’ll see the effect of changing the type of callback being used:

The effect of one or the other callbacks

If the context switch callback is used, then the shape which gets filtered is removedd from ALL rendering passes. Therefore, it does not cast any shadows. If the visibility switch callback is used, then the shape which gets filtered is ONLY removed from the PRIMARY rendering pass. Therefore, it’s not directly visible, but it continues to cast shadows.