Writing Custom Render Shaders

In addition to the built-in render shaders, HOOPS Luminate allows to completely customize your effects by hand made shader programming.

Shader Target Selection

The RED engine is a multi-pass rendering engine. Therefore, a vertex / pixel shader pair may be executed many times for the rendering of a single frame. A simple rendering example with the RED engine uses at least the following workflow as described here: The Shading Pipeline

Pre-lighting pass

Lighting passes (1 pass for each active light in the scene)

Post-lighting pass

A shader is registered and added to one or many passes in this list. Then inside each pass, a shader may use multiple rendering configurations. We call a ‘shader target’ the scope of a shader item (variables, programs, binding configuration). Below is the list of valid shader targets for each possible rendering pass. Passes are defined by the RED::MATERIAL_PASS enumeration.

Pre-Lighting or Post-Lighting Passes

Shader items rendered for these passes have only one single possible shader target:

RED_SHAD_TARGET_LIGHT_NO_LIGHT               // Target a not-a-light rendering pass

All parameters, programs, binding information, must use this flag as shader target. Any other target value will cause the faulty item to be ignored during the rendering. This specific target is often referred as the RED_L0.

Lighting Passes

Shader items rendered for these passes must specify at least one of the following shader targets:

RED_SHAD_TARGET_LIGHT_DIRECTIONAL            // Parameter will be available for directional lights only
RED_SHAD_TARGET_LIGHT_BEAM                   // Parameter will be available for beam lights only
RED_SHAD_TARGET_LIGHT_POINT                  // Parameter will be available for point lights only
RED_SHAD_TARGET_LIGHT_AREA                   // Parameter will be available for area lights only
RED_SHAD_TARGET_LIGHT_POINT_SPOT             // Parameter will be available for point spot lights only
RED_SHAD_TARGET_LIGHT_AREA_SPOT              // Parameter will be available for area spot lights only

For example, the vertex and pixel shader programs used for a directional light are usually not the same as the programs used for a point light, simply because the lighting calculations differ.

Ray-Tracer Passes

Shader items rendered for the ray-tracer must specify at least one of the following shader targets:

RED_SHAD_TARGET_REFLECTION_VECTOR            // Used for the definition of the first ray bounce reflection direction
RED_SHAD_TARGET_REFRACTION_VECTOR            // Used for the definition of the first ray bounce refraction direction
RED_SHAD_TARGET_INDIRECT_REFLECTION_VECTOR   // Used for the definition of the second+ ray bounce reflection direction
RED_SHAD_TARGET_INDIRECT_REFRACTION_VECTOR   // Used for the definition of the second+ ray bounce refraction direction
RED_SHAD_TARGET_REFLECTION_CUTOFF            // Used for the definition of the first ray bounce reflection cutoff threshold
RED_SHAD_TARGET_INDIRECT_REFLECTION_CUTOFF   // Used for the definition of the second+ ray bounce reflection cutoff threshold
RED_SHAD_TARGET_REFRACTION_CUTOFF            // Used for the definition of the first ray bounce refraction cutoff threshold
RED_SHAD_TARGET_INDIRECT_REFRACTION_CUTOFF   // Used for the definition of the second+ ray bounce refraction cutoff threshold
RED_SHAD_TARGET_GI_DIFFUSE_COLOR             // Used for the definition of the GI diffusion property of the material
RED_SHAD_TARGET_GI_REFLECTION_COLOR          // Used for the definition of the GI reflectivity property of the material
RED_SHAD_TARGET_GI_TRANSMISSION_COLOR        // Used for the definition of the GI transmission property of the material
RED_SHAD_TARGET_GI_NORMAL                    // Used for the definition of the GI surface normal of the material

These targets are all used within the RED::MTL_RAYTRACE pass of a material.

From Geometry Data to Rendering Programs

In this section, we will see how to define the data to be transferred from the mesh geometry to the vertex shader using the RED::RenderCode class.

A RED::RenderShader class needs the following minimal set of parameters to work:

A vertex shader program (RED::RenderShader::SetVertexProgramId)

A pixel shader program (RED::RenderShader::SetPixelProgramId)

A RED::RenderCode (RED::RenderShader::SetRenderCode)

The RED::RenderCode performs the association between mesh data channels and vertex shader channels. Vertex shaders have 16 generic input attributes that can be used. Meshes have 16 generic channels that can store various data.

../../../../_images/bk_bm_custom_custom_geom_01.png — **The RED::RenderCode swizzles mesh channels to setup the vertex shader**

A mesh data channel may be replicated using the RED::RenderCode at the vertex shader’s entrance. A mesh data channel may although be ignored if it’s not needed for the shader in the rendered configuration. But a valid RED::RenderCode is mandatory to fill the shader’s inputs.

The RED::RenderCode::BindChannel function gives the ability to define which mesh channel will be connected to the shader inputs. Two enumerations are generally used to define the mesh channels and vertex inputs: RED::MESH_CHANNEL and RED_VSH_INPUT.

Note that the RED engine uses the generic naming for all vertex shader input attributes. Therefore, a vertex shader must use the vertex.attrib syntax defined by the ARB vertex program extension. It can’t use the vertex.position, vertex.texcoord syntax, as the engine provides generic inputs, and does not use any conventional input channel name.

The RED::RenderCode object exposes several other options like sending the model or the view matrix to the shader using the functions RED::RenderCode::SetModelMatrix and RED::RenderCode::SetViewMatrix, or to normalize a channel (RED::RenderCode::SetNormalizedChannel).

Shader Parameters

Shader parameters are inserted in the rendering workflow as detailed on the figure below:

../../../../_images/bk_bm_custom_custom_param_01.png — **View of the full shader pipeline**

Values

The RED::RenderShaderParameter class enumerates all possible values that can be made available to a shader program:

RED::Vector (4 floats)

RED::Color (RGBA, 4 floats)

RED::Matrix (4x4 homogeneous matrices, floats)

Image (2D, composite, cube)

These values are bound by the specification of three items:

The shader program targeted by the parameter: vertex or pixel

The name of the parameter: a RED::String that identify the parameter for later searches from the shader API

The binding position of the parameter

The binding position is understood in a way that is specific for each kind of parameter:

RED::Vector: Number of the program.local constant that receive the vector

RED::Color: Number of the program.local constant that receive the color

RED::Matrix: Numbers of the program.local constants that receive the matrix. 4 consecutive constants are used at the specified binding position. If we bind at the 4th constant, the parameter will use program.local[4, 5, 6, 7]. A matrix is bound using the line major convention: e.g. program.local[4] matches the first matrix row in our example

Image: Texture channel bound to the texture. We’ve at least 16 texture channels available

References

References are HOOPS Luminate specific shader parameters whose values are automatically computed by the engine based on the current rendering pass. Refer to the RED::RenderShaderParameter::TYPE enumeration for a list of all possible references.

If we consider for example a shader using the dimensions of the viewport, then if we use this shader in two different windows, the shader will be called twice, each time with the corresponding viewport parameters. The same applies for all lights parameters: if we have more than one light in a scene, then all positions, directions, textures, colors of the light change during each lighting pass, hence the need for references. References are specified within the RED::RenderShaderParameter, indicating the enumerated value for the corresponding wished reference.

Note

A reference uses one of the possible values of a shader parameter: a reference may be a RED::Color, a RED::Vector, an image (RED::Object), a RED::Matrix.

Camera Level Parameters

Sometimes, defining global parameters that are common to all materials in the context of a scene graph can be very useful. HOOPS Luminate provides such a mechanism by defining camera shader parameters.

The RED::IViewpoint::AddRenderShaderParameter is used to define shader parameters at the camera level. It defines global parameters for all the render shaders contained in the viewpoint scene graph.

The two shader parameter functions RED::RenderShaderParameter::SetCameraParameterValue and RED::RenderShaderParameter::GetCameraParameterValue could then be called to define which global parameter must be used by the shader in the scene graph during the rendering phase. The shader parameter stores the reference to the camera parameter, it is of type RED::RenderShaderParameter::REF_CAMERA_PARAMETER.

../../../../_images/API_CameraLevelParameters.png — **The camera level parameters model**

Each camera in the scene has a parameter storing a color: yellow for the first camera, blue for the second camera. During the rendering of each camera scene graph, the geometry shading will look for the referenced parameter in the list of parameters available from the camera.

Adding a Shader Parameter to a Rendering Shader

// shader is a custom RED::RenderShader added to a RED::MTL_PRELIT pass. Hence the RED_L0 parameter.
// param1 is a float parameter named "parameter1" for the vertex shader program.
RED::RenderShaderParameter param1( "parameter1", 0, RED::RenderShaderParameter::VSH );
param1.SetValue( 10.0 );
RC_TEST( shader.AddParameter( param1, RED_L0 ) );

// param2 is a color parameter named "parameter2" for the pixel shader program.
RED::RenderShaderParameter param2( "parameter2", 1, RED::RenderShaderParameter::PSH );
param2.SetValue( RED::Color( 1.0, 0.0, 0.0, 1.0 ) );
RC_TEST( shader.AddParameter( param2, RED_L0 ) );

In this sample code, two parameters are added to a custom RED::RenderShader. The render shader is added to the RED::MTL_PRELIT pass of a material, hence the RED_L0 parameter.

The first parameter is of type double and is intended for the vertex shader: RED::RenderShaderParameter::VSH.

The second parameter is of type RED::Color and is intended for the pixel shader: RED::RenderShaderParameter::PSH.

By calling the appropriate RED::RenderShaderParameter::SetValue method, the parameter type RED::RenderShaderParameter::TYPE is automatically set.

// paramshadow is a reference to the shadow image texture.
// It is named "shadowimage" and is intended for the pixel shader program.
// It targets the generic hardware platform and the software platform.
RED::RenderShaderParameter paramshadow( "shadowimage", 0, RED::RenderShaderParameter::PSH );
paramshadow.SetReference( RED::RenderShaderParameter::REF_LIGHT_SHADOW_IMAGE_TEX );
RC_TEST( shader.AddParameter( paramshadow, RED_LALL ) );
RC_TEST( shader.AddParameter( paramshadow, RED_LALL, RED::HW_SOFT_TRACER ) );

In the previous example, the shader is added to the RED::MTL_LIT pass of a material. The RED_LALL parameter indicates that we focus on accessing parameters from any kind of light.

The parameter added is a reference to the shadow image (RED::RenderShaderParameter::REF_LIGHT_SHADOW_IMAGE_TEX). It targets the RED::HW_GENERIC platform by default and the RED::HW_SOFT_TRACER target for software rendering.