Custom Shaders

Defining a Custom Shader

Before a custom shader can be used, it needs to be defined through a call to HC_Define_Shader. HC_Define_Shader will return the shader definition key upon success, or -1 if an error occurred. In order to define a shader, three things are needed:

a name – this field is not used by Visualize, and is only available to allow developers to associate the shader definition with a string.
a stage – this field determines which stage of the graphics pipeline the shader will be bound to. Currently the only two options available are vertex and pixel.
the shader source – this is a string containing the source which will be injected in the Visualize shaders. There is a line length limit of 1024 characters (including escaped sequences). Either GLSL or HLSL source is allowed. If you wish to run your code using different graphics driver, the source you provide should compile on all of them. The shader source must be contained in a function called custom_main with the following signature:

// for vertex shaders
void custom_main(inout VertexShaderContext ctx)

// for pixel shaders
void custom_main(inout PixelShaderContext ctx)

The VertexShaderContext and PixelShaderContext are the two structures which will be used to interact with Visualize shader code. They are defined as follows:

struct VertexShaderContext {
        vertex_input vinput;            // Input vertex data, unchanged by Visualize
        vertex_output voutput;          // Output vertex data, already processed by Visualize vertex shader. Will be passed to the pixel shader.
};

struct PixelShaderContext {
        pixel_input pinput;             // Input pixel data, unchanged by Visualize
        pixel_output poutput;           // Output pixel data, already processed by Visualize pixel shader. Will be passed to Output Merger pipeline stage.
        shared_struct sharedvalues;
};

Using Custom Shaders

Once a custom shader has been defined it needs to be set on a segment, like any other attribute, in order for it to take effect during drawing. This can be done through the HC_Set_Shader function. The function allows to set both a vertex and a pixel shader at the same time, but only one of the two is required.

The target parameter for the function determines which kind of geometry will be affected by the shader, with the possible choices being triangles, lines, points.

There is no one-to-one mapping between HOOPS geometry types and the target used by the Set_Shader function. For example, text might be rendered as triangles, lines or points, depending on the rendering options which are selected.

Since custom shaders are attributes, they will inherit down the scene graph.

Limitations

Using a custom vertex shader to change the output vertex position will cause issues when using highlight and selection features.

Example

This example shows how to define and set a custom shader which will always draw triangles and lines in red:

        static char const ps_source[] = {"void custom_main(inout PixelShaderContext ctx) { \n"
                                         "\tctx.poutput.FinalColor = float4(1.0, 0.0, 0.0, 0.0);\n"
                                         "}\n\n"};

        HC_KEY shader_key = HC_Define_Shader("red_shader", "pixel", ps_source);
        HC_Set_Shader("triangles, lines", HC_ERROR_KEY, shader_key);

Passing Custom Data Between Vertex and Pixel Shaders

By default, the pixel shader has access to the contents of the vertex_output / pixel_input structures (the two structures are always identical). It is possible to add custom fields to this structure and to assign a value to them in the vertex shader, so that they can be optionally interpolated by the graphics pipeline before becoming available as an input to the pixel shader.

The Set_Shader_Outputs function is used to define new vertex output fields. Three things are needed in order to define a new output field:

its name – in order to avoid conflict with other variables defined by the Visualize shaders, user_ will be automatically prepended to the name selected
its type
whether or not the value of the field should be interpolated

Note that Set_Shader_Outputs can only be called on a custom vertex shader.

Example

        // The custom field can be accessed in the vertex shader as
        // ctx.voutput.user_customField
        static char const vs_source[] = {"void custom_main(inout VertexShaderContext ctx) { \n"
                                         "\tctx.voutput.user_customField = float3(1.0, 0.0, 0.0);\n"
                                         "}\n\n"};

        // The custom field can be accessed in the pixel shader as
        // ctx.pinput.user_customField
        static char const ps_source[] = {
            "void custom_main(inout PixelShaderContext ctx) { \n"
            "\tfloat calculation = dot(ctx.pinput.user_customField, float3(ctx.pinput.position.xyz));\n"
            "\tcalculation = saturate(calculation);\n"
            "\tctx.poutput.FinalColor.x = calculation;\n"
            "}\n\n"};

        HC_KEY vertex_shader_key = HC_Define_Shader("v_shader", "vertex", vs_source);
        HC_KEY pixel_shader_key = HC_Define_Shader("p_shader", "pixel", ps_source);
        HC_Set_Shader("triangles, lines", vertex_shader_key, pixel_shader_key);
        HC_Set_Shader_Outputs(vertex_shader_key, "customField = (type = float3, interpolation)");

Limitations

There is a finite amount of fields that can be output from the vertex shader. This number changes from driver to driver and sometimes also based on the GPU running the code. Additionally, Visualize will use a variable number of outputs based on the complexity of the scene that needs to be rendered.

This means that you should use the minimum amount of outputs possible in order to avoid encountering this limitation.

IMPORTANT: The source text in user_customField must always be lowercase, or it won’t work.

Guaranteeing Data Availability

The contents of the vertex_input and pixel_input structs change from one draw call to the next, based on what Visualize determines is necessary for rendering the geometry being drawn. This means that a custom shader that uses a vertex’s normal to complete some calculation might fail to compile in the case where Visualize determined that a scene did not need normals in order to be rendered.

To work around this issue it is possible to request that some inputs to the vertex and pixel shader be always made available through a call to Set_Shader_Inputs. Different shader types have different fields that can be requested:

Input	Available stages	Available field
normal	vertex	`vinput.normal`
eye position	pixel	`pinput.eye_position`
eye normal	pixel	`pinput.eye_normal`
object view	pixel	`pinput.object_view`
object normal	pixel	`pinput.object_normal`
texcoord[0-7]	vertex, pixel	`vinput.texcoord[0-7]`, `pinput.texcoord[0-7]`

Example

        // My custom shader uses the vertex normal for calculations, therefore I need to make sure
        // it is available to the shader
        static char const vs_source[] = {
            "void custom_main(inout VertexShaderContext ctx) { \n"
            "\tctx.voutput.user_customField = normalize(mul(ctx.vinput.normal, GetNormalizedModelMatrix(ctx.vinput)));\n"
            "}\n\n"};

        static char const ps_source[] = {
            "void custom_main(inout PixelShaderContext ctx) { \n"
            "\tfloat calculation = dot(ctx.pinput.user_customField, float3(ctx.pinput.position.xyz));\n"
            "\tcalculation = saturate(calculation);\n"
            "\tctx.poutput.FinalColor.x = calculation;\n"
            "}\n\n"};

        HC_KEY vertex_shader_key = HC_Define_Shader("v_shader", "vertex", vs_source);
        HC_KEY pixel_shader_key = HC_Define_Shader("p_shader", "pixel", ps_source);
        HC_Set_Shader("triangles, lines", vertex_shader_key, pixel_shader_key);
        HC_Set_Shader_Outputs(vertex_shader_key, "customField = (type = float3, interpolation)");
        HC_Set_Shader_Inputs(vertex_shader_key, "normal");

Limitations

Texture coordinates can only be requested in order: HC_Set_Shader_Inputs(key, "texcoord2, texcoord0, texcoord1") is a valid call because it requests texture coordinates 0 through 2, while HC_Set_Shader_Inputs(key, "texcoord2, texcoord0") is not a valid call because it requests texture coordinates 0 and 2 while omitting 1.
Certain options will only provide the requested input at runtime if the geometry being drawn supports that option. For example, texture coordinates will only be available when drawing geometry that can be textured: shells and meshes.
If texture coordinates are requested as inputs, any kind of texturing will be disabled (this includes bump mapping, diffuse texturing, etc…) and Visualize will expect that 4 vertex parameters will be available for each vertex of triangles drawn under such a shader. Each 4th vertex parameter should be set to 1 if that vertex will be transformed, as is the case with textures.

Structs

Only struct members which are deemed of interest to our partners are explained here.

The description for the availability column is an approximation. To find out exactly when every field is available it is always best to refer to the code directly, since the circumstances can change.

What follows is a description of the fields in the structs which are used in the vertex and pixel shader.

The contents of the structs change from one draw call to the other based on what is needed to render the draw call. As such, when writing custom shaders, customers are only allowed to refer to a field if the conditions in the Availability column for that field are satisfied.

When the Availability value of a field is listed as Needs to be requested, the user should request the field through a call to Set_Shader_Input.

Vertex_Input

The type of the position field is POS_TYPE.

POS_TYPE can be either a float3 or a float4 depending on how transform data is bound to the shader.

When POS_TYPE is a float4 its w component is an index into a structured buffer.

If the geometry was inserted with double precision and the underlying graphics API supports passing doubles as vertex attributes (presently only true for OpenGL2), POS_TYPE will be either double3 or double4.

Type	Interpolated	Name	Availability	Description
`POS_TYPE`	yes	position	Always available	Vertex position in object space
`float3`	yes	normal	Needs to be requested	Vertex normal in object space
`float4`	yes	tangent	Only available for triangles when bump textures are active	Vertex tangent in object space
`float2`	yes	findex	Only available when color is assigned by fractional index (example: because of a call to ``HC_Set_Color_By_FIndex`)	UV coordinates used to lookup color of geometry colored by FIndex
`float4`	yes	texcoord0	Only defined if there is at least one diffuse texture bound to the shader or if requested by the user	Contains the UVW coordinates for the texture. The alpha component is not used.
`float4`	yes	texcoord1	Only defined if there are at least two diffuse textures bound to the shader or if requested by the user	Contains the UVW coordinates for the texture. The alpha component is not used.
`float4`	yes	texcoord2	Only defined if there are at least three diffuse textures bound to the shader or if requested by the user	Contains the UVW coordinates for the texture. The alpha component is not used.
`float4`	yes	texcoord3	Only defined if there are at least four diffuse textures bound to the shader or if requested by the user	Contains the UVW coordinates for the texture. The alpha component is not used.
`float4`	yes	texcoord4	Only defined if there are at least five diffuse textures bound to the shader or if requested by the user	Contains the UVW coordinates for the texture. The alpha component is not used.
`float4`	yes	texcoord5	Only defined if there are at least six diffuse textures bound to the shader or if requested by the user	Contains the UVW coordinates for the texture. The alpha component is not used.
`float4`	yes	texcoord6	Only defined if there are at least seven diffuse textures bound to the shader or if requested by the user	Contains the UVW coordinates for the texture. The alpha component is not used.
`float4`	yes	texcoord7	Only defined if there are at least eight diffuse textures bound to the shader or if requested by the user	Contains the UVW coordinates for the texture. The alpha component is not used.
`float4`	yes	backtexcoord0	Only defined if there is at least one back face diffuse texture bound to the shader	Contains the UVW coordinates for the texture. The alpha component is not used.
`float4`	yes	backtexcoord1	Only defined if there are at least two back face diffuse textures bound to the shader	Contains the UVW coordinates for the texture. The alpha component is not used.
`float4`	yes	backtexcoord2	Only defined if there are at least three back face diffuse textures bound to the shader	Contains the UVW coordinates for the texture. The alpha component is not used.
`float4`	yes	backtexcoord3	Only defined if there are at least four back face diffuse textures bound to the shader	Contains the UVW coordinates for the texture. The alpha component is not used.
`float4`	yes	backtexcoord4	Only defined if there are at least five back face diffuse textures bound to the shader	Contains the UVW coordinates for the texture. The alpha component is not used.
`float4`	yes	backtexcoord5	Only defined if there are at least six back face diffuse textures bound to the shader	Contains the UVW coordinates for the texture. The alpha component is not used.
`float4`	yes	backtexcoord6	Only defined if there are at least seven back face diffuse textures bound to the shader	Contains the UVW coordinates for the texture. The alpha component is not used.
`float4`	yes	backtexcoord7	Only defined if there are at least eight back face diffuse textures bound to the shader	Contains the UVW coordinates for the texture. The alpha component is not used.
`float3`	yes	uvsmirror	Only available if a mirror texture is bound to the shader	When using spherical parameterization source it contains a non-normalized vector pointing in the direction to be sampled. Otherwise it contains the UV coordinates for sampling the mirror texture
`float3`	yes	uvsspec	Only available if a specular texture is bound to the shader	When using spherical parameterization source it contains a non-normalized vector pointing in the direction to be sampled. Otherwise it contains the UV coordinates for sampling the specular texture
`float3`	yes	uvsbump	Only available if a bump texture is bound to the shader	When using spherical parameterization source it contains a non-normalized vector pointing in the direction to be sampled. Otherwise it contains the UV coordinates for sampling the bump texture
`float3`	yes	uvsemission	Only available if an emission texture is bound to the shader	When using spherical parameterization source it contains a non-normalized vector pointing in the direction to be sampled. Otherwise it contains the UV coordinates for sampling the emission texture

Vertex_Output / Pixel_Input

Type	Interpolated	Name	Availability	Description
`float4`	yes	position	Always available	As a vertex output, vertex position in homogeneous coordinates. As a pixel input, interpolated vertex position in device coordinates
`float4`	no	material_color_and_specular_intensity	Always available	The xyz component contains the diffuse color, the alpha component contains the specular intensity. The color contained in this variable was passed in either as a vertex attribute or as a uniform value.
`float4`	yes	eye_position	Needs to be requested	Vertex position in camera coordinates
`float3`	no	eye_normal	Needs to be requested	Vertex normal in camera coordinates. Not guaranteed to be normalized.
`float4`	no	diffuse_light_and_ambient_blend	Only available when eye_normal has not been requested, or when lighting is calculated per-vertex rather than per-pixel	the xyz component contains the diffuse color, the alpha component contains an ambient blend factor between 0 and 1. The color contained in this variable was calculated based on the surface and lighting characteristics of the scene.
`float4`	yes	eye_tangent	Only available for triangles when bump textures are active	Vertex tangent in camera coordinates. Not guaranteed to be normalized.
`float2`	yes	findex	Only available when color is assigned by fractional index (example: because of a call to HC_Set_Color_By_FIndex)	When drawing geometry with color specified by findex a texture containing all the possible colors is created. findex contains the UV coordinates necessary to index the color texture.
`float4`	yes	texcoord0	Only defined if there is at least one diffuse texture bound to the shader or if requested by the user	Contains the UVW coordinates for the texture. The alpha component is not used.
`float4`	yes	texcoord1	Only defined if there are at least two diffuse textures bound to the shader or if requested by the user	Contains the UVW coordinates for the texture. The alpha component is not used.
`float4`	yes	texcoord2	Only defined if there are at least three diffuse textures bound to the shader or if requested by the user	Contains the UVW coordinates for the texture. The alpha component is not used.
`float4`	yes	texcoord3	Only defined if there are at least four diffuse textures bound to the shader or if requested by the user	Contains the UVW coordinates for the texture. The alpha component is not used.
`float4`	yes	texcoord4	Only defined if there are at least five diffuse textures bound to the shader or if requested by the user	Contains the UVW coordinates for the texture. The alpha component is not used.
`float4`	yes	texcoord5	Only defined if there are at least six diffuse textures bound to the shader or if requested by the user	Contains the UVW coordinates for the texture. The alpha component is not used.
`float4`	yes	texcoord6	Only defined if there are at least seven diffuse textures bound to the shader or if requested by the user	Contains the UVW coordinates for the texture. The alpha component is not used.
`float4`	yes	texcoord7	Only defined if there are at least eight diffuse textures bound to the shader or if requested by the user	Contains the UVW coordinates for the texture. The alpha component is not used.
`float4`	yes	backtexcoord0	Only defined if there is at least one back face diffuse texture bound to the shader	Contains the UVW coordinates for the texture. The alpha component is not used.
`float4`	yes	backtexcoord1	Only defined if there are at least two back face diffuse textures bound to the shader	Contains the UVW coordinates for the texture. The alpha component is not used.
`float4`	yes	backtexcoord2	Only defined if there are at least three back face diffuse textures bound to the shader	Contains the UVW coordinates for the texture. The alpha component is not used.
`float4`	yes	backtexcoord3	Only defined if there are at least four back face diffuse textures bound to the shader	Contains the UVW coordinates for the texture. The alpha component is not used.
`float4`	yes	backtexcoord4	Only defined if there are at least five back face diffuse textures bound to the shader	Contains the UVW coordinates for the texture. The alpha component is not used.
`float4`	yes	backtexcoord5	Only defined if there are at least six back face diffuse textures bound to the shader	Contains the UVW coordinates for the texture. The alpha component is not used.
`float4`	yes	backtexcoord6	Only defined if there are at least seven back face diffuse textures bound to the shader	Contains the UVW coordinates for the texture. The alpha component is not used.
`float4`	yes	backtexcoord7	Only defined if there are at least eight back face diffuse textures bound to the shader	Contains the UVW coordinates for the texture. The alpha component is not used.
`float3`	yes	object_view	Needs to be requested	Vector from the vertex position to the camera eye position.
`float3`	yes	object_normal	Needs to be requested	Vertex normal in world coordinates. Nor guaranteed to be normalized.
`float3`	yes	uvenv	Only available if an environment texture is bound to the shader.	When using a cube map, uvenv contains the reflection vector used to sample the environment texture cube. Otherwise it contains the UV coordinates for sampling the environment texture.
`float4`	yes	uvstmirror	Only available if a mirror texture is bound to the shader	When using spherical parameterization source it contains a non-normalized vector pointing in the direction to be sampled. Otherwise it contains the UV coordinates for sampling the mirror texture
`float4`	yes	`uvstspec	Only available if a specular texture is bound to the shader	When using spherical parameterization source it contains a non-normalized vector pointing in the direction to be sampled. Otherwise it contains the UV coordinates for sampling the specular texture
`float4`	yes	uvstbump	Only available if a bump texture is bound to the shader	When using spherical parameterization source it contains a non-normalized vector pointing in the direction to be sampled. Otherwise it contains the UV coordinates for sampling the bump texture
`float4`	yes	uvstemission	Only available if an emission texture is bound to the shader	When using spherical parameterization source it contains a non-normalized vector pointing in the direction to be sampled. Otherwise it contains the UV coordinates for sampling the emission texture

Pixel_Output

Type	Name	Availability	Description
`float4`	FinalColor	Always available	The output color for the pixel

Shared_Struct

Type	Name	Default Value
`float3`	`surface.position`	May be set by `vertex_output.eye_position.xyz` if available
`float3`	`surface.normal`	May be set by `vertex_output.eye_normal` if available
`float3`	`surface.tangent`	May be set by `vertex_output.eye_tangent.xyz` if available
`float3`	`surface.bitangent`	May be set by `vertex_output.eye_tangent` if available
`float3`	`surface.object_normal`	May be set by `vertex_output.object_tangent` if available
`float3`	`surface.object_bitangent`	May be set by `vertex_output.object_tangent` if available
`float2`	`surface.marker_uv`	May be set if texture available
`float3`	`lighting.ambient`	Set with ambient light value from vertex shader
`float3`	`lighting.diffuse`	`vertex_output.diffuse_light_and_ambient_blend.rgb` Should be set if there is a light in the scene
`float3`	`lighting.specular`	`vertex_output.material_color_and_specular_intensity` Should be set if there is a light in the scene
`float3`	`effects.mirror`	May be set there is environment texture set
`float3`	`effects.environment`	May be set there is environment texture set
`float3`	`color.diffuse`	`uniformFragment.MaterialColorSpecular`
`float3`	`color.specular`	`vertex_ouput.material_color_and_specular_intensity`

Uniforms

Common Uniform

The common uniform is accessible from both the vertex and pixel shader. Its fields can be accessed as follows: uniformCommon.fieldName.

All matrices contained in this uniform are declared row major for use in HLSL.

Type	Name	Description
`float3`	EyeCameraPosition	The camera’s eye position in camera space
`float3`	DistantLightsDirection[]	The size of the array is `number of distant lights + 1`. Directions of distant lights in camera space
`float4`	DistantLightsDiffuse[]	The size of the array is `number of distant lights + 1`. Diffuse color of distant lights
`float3`	DistantLightsSpecular[]	The size of the array is `number of distant lights + 1`. Diffuse color of distant lights modulated by the specular component
`float3`	DistantLightsHalfway[]	The size of the array is `number of distant lights + 1`. Normalized halfway vector between distant light direction and camera position
`float3`	PointLightsPosition[]	The size of the array is `number of point lights + 1`. Position of point lights in camera space
`float4`	PointLightsDiffuse[]	The size of the array is `number of point lights + 1`. Diffuse color of point lights
`float3`	PointLightsSpecular[]	The size of the array is `number of point lights + 1`. Diffuse color of point lights modulated by the specular component
`float3`	SpotLightsPosition[]	The size of the array is `number of spot lights + 1`. Position of spot lights in camera space
`float3`	SpotLightsDirection[]	The size of the array is `number of spot lights + 1`. Direction of spot lights in camera space
`float2`	SpotLightsAngles[]	The size of the array is `number of spot lights + 1`. Values used for spot light attenuation
`float4`	SpotLightsDiffuse[]	The size of the array is `number of spot lights + 1`. Diffuse color of spot lights
`float3`	SpotLightsSpecular[]	The size of the array is `number of spot lights + 1`. Diffuse color of spot lights modulated by the specular component
`float4x4`	IndexTextureMatrix	Texture transform used for geometry colored by FIndex
`float4x4`	TextureMatrix0	Transform for diffuse texture 0
`float4x4`	TextureMatrix1	Transform for diffuse texture 1
`float4x4`	TextureMatrix2	Transform for diffuse texture 2
`float4x4`	TextureMatrix3	Transform for diffuse texture 3
`float4x4`	TextureMatrix4	Transform for diffuse texture 4
`float4x4`	TextureMatrix5	Transform for diffuse texture 5
`float4x4`	TextureMatrix6	Transform for diffuse texture 6
`float4x4`	TextureMatrix7	Transform for diffuse texture 7
`float4x4`	BumpTextureMatrix	Transform for bump texture
`float4x4`	SpecularTextureMatrix	Transform for specular texture
`float4x4`	MirrorTextureMatrix	Transform for mirror texture
`float4x4`	EnvironmentTextureMatrix	Transform for environment texture
`float4x4`	TransmissionTextureMatrix	Transform for transmission texture
`float4x4`	EmissionTextureMatrix	Transform for emission texture

Vertex Uniform

The vertex uniform is accessible from the vertex shader. Its fields can be accessed as follows: uniformVertex.fieldName.

Type	Name	Description
`float3`	CameraPosition	Camera position in object space
`float4`	MaterialColorDiffuse	Material diffuse value
`float4`	MaterialColorSpecular	Material specular value
`float`	ColormapSize	Size of the colormap used when coloring geometry by FIndex
`float2`	InvViewport	Inverse of the viewport width and height, in pixels

Fragment Uniform

The fragment uniform is accessible from the pixel shader. Its fields can be accessed as follows: uniformFragment.fieldName.

Type	Name	Description
`float4`	MaterialColorDiffuse	Material diffuse value
`float4`	MaterialColorSpecular	Material specular value
`int`	AlphaTest	Whether alpha test is active

Utility Functions

Compatibility Macros

The following macros are defined in Visualize shaders and can be used to make it easier to write shader code which compiles on all our supported drivers. The following table indicates the macro and its equivalent evaluation in the various drivers:

Macro	OpenGL	DirectX	Metal (GLSL / MSL)
`float2`	vec2	float2	vec2 / float2
`float3`	vec3	float3	vec3 / float3
`float4`	vec4	float4	vec4 / float4
`float2x2`	mat2	float2x2	mat2 / float2x2
`float3x3`	mat3	float3x3	mat3 / float3x3
`float4x4`	mat4	float4x4	mat4 / float4x4
`double2`	dvec2	double2	Unsupported
`double3`	dvec3	double3	Unsupported
`double4`	dvec4	double4	Unsupported
`int2`	ivec2	int2	ivec2 / int2
`int3`	ivec3	int3	ivec3 / int3
`int4`	ivec4	int4	ivec4 / int4
`uint2`	uvec2	uint2	uvec2 / uint2
`uint3`	uvec3	uint3	uvec3 / uint3
`uint4`	uvec4	uint4	uvec4 / uint4
`bool2`	bvec2	bool2	bvec2 / bool2
`bool3`	bvec3	bool3	bvec3 / bool3
`bool4`	bvec4	bool4	bvec4 / bool4
`ddx`	dFdx	ddx	dFdx / dfdx
`ddy`	dFdy	ddy	dFdy / dfdy
`mul(a, b)`	((b) * (a))	mul(a, b)	((b) * (a)) / ((b) * (a))
`saturate(a)`	clamp((a), 0.0, 1.0)	saturate(a)	clamp((a), 0.0, 1.0) / saturate(a)``
lerp(x, y, s)	mix(x, y, s)	lerp(x, y, s)	mix(x, y, s) / mix(x, y, s)
frac(x)	fract(x)	frac(x)	fract(x) / fract(x)
atan2(y, x)	atan(y, x)	atan2(y, x)	atan(y, x) / atan2(y, x)
equal(a, b)	equal(a, b)	((a) == (b))	equal(a, b) / ((a) == (b))
notEqual(a, b)	notEqual(a, b)	((a) != (b))	notEqual(a, b) / ((a) != (b))
greaterThan(a, b)	greaterThan(a, b)	((a) > (b))	greaterThan(a, b) / ((a) > (b))
lessThan(a, b)	lessThan(a, b)	((a) < (b))	lessThan(a, b) / ((a) < (b))
not(a)	not(a)	(!(a))	not(a) / (!(a))

Vertex Shader

The way transform matrices are passed to the shader changes based on how the scene will be rendered. In order to access the correct values regardless of how the shader is organized, the following utility functions should be used.

The following utility functions are only available when the geometry is not being drawn in display coordinates.

GetProjectionMatrix

float4x4 GetProjectionMatrix(const vertex_input vertex)

vertex	the `vinput` component of the `VertexShaderContext`
result	the projection matrix

GetModelMatrix

float4x4 GetModelMatrix(const vertex_input vertex)

vertex	the `vinput` component of the `VertexShaderContext`
result	the world matrix

GetViewMatrix

float4x4 GetViewMatrix(const vertex_input vertex)

vertex	the `vinput` component of the `VertexShaderContext`
result	the view (camera) matrix

GetModelViewMatrix

float4x4 GetModelViewMatrix(const vertex_input vertex)

vertex	the `vinput` component of the `VertexShaderContext`
result	the product of the model and view (camera) matrices

GetNormalizedModelMatrix

float3x3 GetNormalizedModelMatrix(const vertex_input vertex)

vertex	the `vinput` component of the `VertexShaderContext`
result	the normalized 3x3 adjoint of the world matrix

GetNormalizedModelViewMatrix

float3x3 GetNormalizedModelViewMatrix(const vertex_input vertex)

vertex	the `vinput` component of the `VertexShaderContext`
result	the product of the normalized world matrix and the view (camera) matrices

Internal Implementation

There are three main parts to how the modern custom shaders are implemented internally:

Custom shader definition
Custom shader attribute
Usage of the attribute and definition by each graphics API

Custom Shader Definition

The custom shader definition is an object that exists to hold onto all the relevant data for a shader so that we can refer to it elsewhere. Defining a custom shader doesn’t actually use it, it simply defines it. Much like we can define a line style, but never use it as a line pattern when drawing, or define a glyph, but never use it as a marker symbol when drawing.

This is represented internally by a Custom_Shader_Definition object, and is created when HC_Define_Shader is used to define a modern custom shader.

This object will be stored globally on a linked list hanging off of HOOPS::WORLD.

It is the object which contains all the relevant settings that were used to define the shader, basically, the name, what stage it applies to, and the source for the custom_main function.

Additionally, it will hold onto any settings which operate directly on an HC_KEY representing a custom shader. In particular, this is where the following functions will also store, query, or remove any data they set:

HC_Set_Shader_Inputs
HC_Set_Shader_Outputs
HC_Set_Shader_Samplers
HC_Set_Shader_Textures
HC_Set_Shader_Uniform

All of these functions merely store the necessary data on the definition which will get referred to later when we need to actually start creating our shaders when rendering.

Samplers and textures behave slightly differently than the other settings here in that they also need to get defined (though they are immutable) and are referred to by HC_KEY, so there are some additional definition functions for those types:

HC_Define_Shader_Sampler - creates a Custom_Shader_Sampler internally
HC_Define_Shader_Texture - creates a Custom_Shader_Texture internally

Sampler and texture definitions are also global objects and are stored as linked lists hanging off HOOPS::WORLD as well.

Custom Shader Attribute

To actually tell the scene graph we want to use a custom shader, we need to set an attribute on a segment. We do this by calling HC_Set_Shader on the segment where we want our previously defined shader to get used, and this will create a Custom_Shader attribute object which will be associated with the segment.

The Custom_Shader attribute will store the Custom_Shader_Definition pointers we want to use as vertex or pixel shaders for a particular type of primitive.

Additionally it will need to store any uniform data that needs to be set for a particular usage of a custom shader if that custom shader had a uniform added to it. This is because we could use the same custom shader in a different segment, but we may want to have the custom shader use different uniform data. For example, we may have different modelling matrices in two different segments, so it should be possible to specify each of those modelling matrices separately, but otherwise use the same shader.

Debugging Shaders

Shaders can be complicated pieces of code and errors may be hard to spot. HOOPS Visualize 3DF has a few functions you can use to aid in finding syntax errors in your shader code.

HC_Compile_Shader
HC_Show_Compilation_Results
HC_Show_Compilation_Results_Size

For a quick way to test whether Visualize can find your shader code where it expects, you can use HC_Compile_Shader:

    HC_KEY keys[]{mySegmentKey, view->GetModelKey(), view->GetViewKey()};
    int keycount = 3;

    if (HC_Compile_Shader(keycount, keys, "geometry = (shell = triangles), stage = pixel") == false) {
        // something went wrong
    }

The first line simply sets a key path from the segment in which your shader exists back to the view segment. The actual call to HC_Compile_Shader will fail if:

the segment path is invalid
no segment with a defined shader exists on the given key path
the current driver doesn’t support shaders
shader compilation failed

At this step, shader compilation results are stored and you need to call HC_Show_Compilation_Results_Size and HC_Show_Compilation_Results to get the results. The code below shows an example of how we can get the original shader source and errors:

static bool check_compilation_results(
    char const* stage, char const* primitive, HC_KEY* shader_key, char const* source_contains, bool check_non_empty_error = false)
{
    int source_size = 0;
    int errors_size = 0;

    HC_Show_Compilation_Results_Size(stage, primitive, &source_size, &errors_size);

    char* source = (char*)malloc((source_size + 1) * sizeof(char));
    char* error = nullptr;
    if (errors_size > 0) {
        error = (char*)malloc((errors_size + 1) * sizeof(char));
    }

    bool result = true;
    HC_Show_Compilation_Results(stage, primitive, shader_key, &source_size, source, &errors_size, error);

    if (strstr(source, source_contains) == nullptr) {
        result = false;
    }

    if (check_non_empty_error) {
        result = errors_size > 0;
    }

    free(source);
    if (error != nullptr) {
        free(error);
    }

    return result;
}

So what kind of error should you expect? To test this, we’ll use some shader code with a missing semi-colon at the end of the line:

    char const* wrong_shader_ps = "void custom_main(inout PixelShaderContext ctx) { \n"
                                  "\tctx.poutput.FinalColor  = ctx.pinput.user_color_override\n"
                                  "}\n\n";

For this source, we get the error below:

C:\path\to\my\project\Shader@0x0000015AC60535A0(388,28-57): error X3018: invalid subscript 'user_color_override'

Inside the parenthesis above (388,28-57), the first number represents the line in the source file and helps you to find what is wrong.