• Technical Overview
  • Programming Guide
  • Reference Manual
  • HOOPS Visualize
TechSoft3d

HOOPS/OOC Programming Guide (Technical Preview)

The HOOPS/OOC module is provided in this release as a technical preview. It is currently under development and for use as a technical demonstration of out of core processing and rendering of very large datasets.

1.0 Introduction

2.0 Preprocessing Data

  • 2.1 Using the ooc.exe preprocessor

3.0 Loading and Rendering Data

  • 3.1 Using the HOOPS/OOC HIO Plug-in
  • 3.2 Optimizing Rendering

4.0 Platform Support


1.0 Introduction

The HOOPS/OOC module is an out-of-core system that processes and renders large amounts of data. This system is designed take as its input an ASCII point cloud file. In a multi-stage process, HOOPS/OOC transforms the raw point cloud data into a compressed format and renders it with high visual quality without sacrificing performance.

To begin, a prepocessor loads raw input data. It restructures and compresses the data. The preprocessor creates an OOC file, a special purpose HSF file, as well as a directory of data node files. This processed data can be as small as a third of the size of the original raw input. The resultant OOC file can be loaded and rendered in the HOOPS graphics system.

OOCHighLevelWorkflow.png
Figure 1.0a shows the HOOPS/OOC Workflow.


2.0 Preprocessing Data

HOOPS/OOC provides a simple command line executable for processing your data. It is called ooc.exe and can be found in the bin directory of the HOOPS installation.

When the preprocessor receives a point cloud file, it performs three passes of the data. In each pass, every single data point is examined. As a result, this stage can take a significant amount of time. We suggest that you allow for sufficient preprocessing time prior to loading and rendering your model.

OOCDetailedPreprocess.png
Figure 2.0a shows the three phases of preprocessing

The following is a description of the three phases of preprocessing input data goes through:

  1. HOOPS/OOC loads in all the data points and calculates a bounding box that contains the entire data set. One or more temporary binary file(s) are created to house the data for rapid access in subsequent phases.

  2. HOOPS/OOC reads the temporary binary file(s) and then spatially sorts the points. If a user-specified bounding box was defined, HOOPS/OOC will exclude any points outside of this volume. Additionally, users can specify a subsample percentage that tells HOOPS/OOC to sort only a portion of the points in the dataset.

    HOOPS/OOC uses a version of the binary space partition algorithm to sort the data. For each partition, it creates a node that stores a user-specifiable number of points. This number will be the size of the shell that later is loaded into the HOOPS database. If the number of data points exceeds the limit, then HOOPS/OOC subdivides the node further until the threshold is not surpassed.

    While sorting the points, HOOPS/OOC adheres to a memory limit. If the memory usage reaches the limit, HOOPS/OOC writes the least recently used nodes out to disk to free up memory so that it can continue processing the remaining data.

  3. After the data has been spatially sorted, HOOPS/OOC organizes the points into an appropriate scene graph so that HOOPS can render the data efficiently. Beginning at the root node, HOOPS/OOC creates a shell and assigns a randomized user-specificed number of points to it. As HOOPS/OOC moves down the tree, the child nodes are representations of the partitions created in phase two. Each child node is populated with a percentage of the data corresponding to the fraction of data that resides in its respective partitioned area. HOOPS/OOC moves recursively until the leaves of the tree are reached and there are no more partitions. This tree structure is written to an OOC file (<name>.ooc) while the data that belongs to each node is written into a node file (*.node). Node files live in a directory with the same root name and at the same level as the OOC file.

2.1 Using the ooc.exe preprocessor

To preprocess a point cloud file, run the MS Windows command prompt, cmd.exe. Navigate to the HOOPS bin directory. Now you can run the ooc.exe utility. This program takes a number of options but the most important is the input file name which is passed with the -f option. The following example show how you might preprocess a the MyPointCloud.ptx ASCII point cloud file:

> ooc -f ../data/MyPointCloud.ptx -l ../data/MyPointCloudLog.txt

The above command line entry tells ooc.exe to process the ../data/MyPointCloud.ptx file. In the data directory, it creates a MyPointCloudLog_ptx.ooc file along with a MyPointCloudLog_ptx_nodes directory that contains a series of *.node data files. Additionally, it will also create a MyPointCloudLog.txt log file. The preprocessor takes a number of options that lets you control how your data is analyzed and organized. They are as follows:

  • Input File Name: use -f to specify the name and path of the ASCII point cloud file.
  • Output File Name: use -o to specify the name and path of the output file which will be appended with ".ooc" for the index file and "_node" for the data directory.
  • Log File Generation: use -l to specify a log file where data about pre-processing should be written in addition to stdout.
  • Maximum Memory Usage: use -m to specify the maximum amount of memory in MB that will be used in preprocessing. The default is 512. Note that this is an approximate value.
  • Maximum Shell Size: use -s to specify the maximum number of points for a shell at a given node in the point cloud. The default is 10,000.
  • Overwrite Existing Files: use -r to indicating that HOOPS/OOC should overwrite existing files.
  • Culling Bounding Box: use -b to specify a bounding box outside of which points will be disregarded.
  • Subsample Percentage: use -p to specify a percentage of overall points to import.


3.0 Loading and Rendering Data

Once an OOC file and its associated directory of node files have been generated, the OOC file can be passed as input file to the HIOManager in HOOPS/MVO. The HIOManager uses the HOOPS/OOC HIO plug-in to load data into the HOOPS database.

OOCDetailedLoad.png
Figure 3.0a shows the loading and rendering of point cloud data

The following is the workflow describing what happens when an OOC file is loaded into HOOPS:

  1. The OOC file is read by the HOOPS/OOC HIO plug-in which is part of HOOPS/MVO. The class that implements this plug-in is HIOUtilityOOC.
  2. With the information from the OOC file, HIOUtilityOOC can build a scene graph in the HOOPS/3dGS database. This scene graph is a spatial representation of the point cloud. Each node in the tree corresponds a portion of the point cloud spatially.
  3. At render time, HOOPS/3dGS traverses the segment tree. For each node, it determines if that portion of the point cloud will be drawn to the screen. If a specific area of the point cloud is visible to the camera, then HOOPS/3dGS will request data for that node from HIOUtiltyOOC.
  4. HIOUtilityOOC will then find the data file corresponding to the correct node and load it into memory. At this point, the data is put into a queue which is used for performance and memory management.
  5. HIOUtilityOOC works with HOOPS/3dGS to determine when node data can be loaded into the database.
  6. HOOPS/3dGS draws the point cloud data on the screen.

Because the loading and storing of data in memory can be a higly intensive process, HOOPS implements several mechanisms to balance the throughput of data with system performance. HOOPS/3dGS and HOOPS/OOC work in concert to control the amount of data loaded at any given time. Additionally, HOOPS/3dGS also carefully manages the amount of memory used. As parts of the model move out of the view frustrum or are extent culled, HOOPS/3dGS depopulates the data from the associated segments. Geometry is flushed from memory. As areas become visible again, data is reloaded. Note that HOOPS/3dGS does use caching logic such that in situations of quickly panning back and forth or zooming in and out, data is not disposed of prematurely.

3.1 Using the HOOPS/OOC HIO Plug-in

The HOOPS/OOC HIO plug-in lets you import OOC files into HOOPS. It is part of the HIO Plug-in architecture. To use this integration, you must be running the HOOPS/MVO module.

To use the HOOPS/OOC HIO plug-in, please follow the steps below:

  1. Create a hio_plugins directory in your application's working directory.
  2. Copy the hio_ooc.hio file found in <hoops>/bin/<platform>/hio_plugins/hio_ooc to the hio_plugins directory created in the previous step.
  3. Ensure that the hoops_ooc<hoops_version>_vc<version>.dll is installed in your the HOOPS bin directory.
  4. Run your application.

During start-up, when HOOPS/MVO finds the HOOPS/OOC HIO plug-in in your application's path, it will perform the following steps:

  1. Load in the necessary HOOP/OOC libraries.
  2. Create the appropriate input handler.
  3. Register this handler and the associated file extensions to the HIOManager.

The input handler that is registered with the HIOManager is HIOUtilityOOC. You can load files directly with this input handler as shown in the sample code below:

HIOUtilityOOC ooc_reader;
HInputHandlerOptions ooc_opt;
ooc_opt.m_pHBaseView = view;
HFileInputResult result = ooc_reader.FileInputByKey("c:\\temp\\mypointcloud.ooc", &ooc_opt);

3.2 Optimizing Rendering

In addition to the intensive memory and data management HOOPS performs to ensure visual quality and performance for point cloud rendering, there are additional options developers can set that further assist in the optimization of point cloud visualization.

Within HOOPS/MVO, you can use HBaseView::SetFramerateMode passing FrameRateMode::FramerateFixed indicating a frame rate as well as a culling threshold. Setting the framerate lets HOOPS understand how best to balance performance with visual acuity.

By default, the data points will be rendered as vertex markers but you can enable point splatting so that the points give an appearance of a smooth surface. To enable this feature, use the function HBaseView::SetSplatRendering. With HBaseView::SetSplatSize and HBaseView::SetSplatSymbol, you can determine the size and symbol of your splats respectively. If hardware acceleration is available, you can allow HOOPS to leverage it if you call HBaseView::SetFastMarkerDrawing.

Within HOOPS/3dGS, you set several options that will improve rendering performance. We recommend that for point cloud data, shadows should be disabled. In HOOPS, there are two kinds of shadows: Simple shadows and shadow maps. These are both rendering options and can be disabled by calling Set_Rendering_Options. The following code snippets shows how you can turn these options off:

HC_Set_Rendering_Options("no simple shadow, shadow map = off");

There also is another rendering option you can use to improve performance: vertex decimation. Although HOOPS does use culling logic in the fixed frame rate mode, vertex decimation is a quick and direct way to tell HOOPS to only draw a percentage of the vertices in the scene. If you are decimated vertices, the randomize vertices rendering option will cause vertices that are compiled into display lists to be inserted in random order. This option is intended for a more uniform point distribution when applying vertex decimation to non-randomized data.

4.0 Platform Support

HOOPS/OOC is supported on the Windows platform for Visual Studio 2005 and 2008.