==================
Preprocessing data
==================

HOOPS/OOC provides a simple command line executable for processing your data. It is called :doc:`ooc.exe </prog_guide/misc/ooc_preprocessor>` and can be found in the *bin* directory of the HOOPS installation.

When the preprocessor receives a point cloud file, it performs three passes of the data. In each pass, every single data point is examined. As a result, this stage can take a significant amount of time. We suggest that you allow for sufficient preprocessing time prior to loading and rendering your model

.. image:: images/OOCDetailedPreprocess.png

*The three phases of preprocessing*

The following list describes the three phases of input data preprocessing:

1. HOOPS/OOC loads in all the data points and calculates a bounding box that contains the entire data set.  One or more temporary binary file(s) are created to house the data for rapid access in subsequent phases.

2. HOOPS/OOC reads the temporary binary file(s) and then spatially sorts the points. If a user-specified bounding box was defined, HOOPS/OOC will exclude any points outside of this volume.  Additionally, users can specify a subsample percentage that tells HOOPS/OOC to sort only a portion of the points in the dataset.

HOOPS/OOC uses a version of the binary space partition algorithm to sort the data. For each partition, it creates a node that stores a user-specifiable number of points. This number will be the size of the shell that later is loaded into the HOOPS database.  If the number of data points exceeds the limit, then HOOPS/OOC subdivides the node further until the threshold is not surpassed.

While sorting the points, HOOPS/OOC adheres to a memory limit. If the memory usage reaches the limit, HOOPS/OOC writes the least recently used nodes out to disk to free up memory so that it can continue processing the remaining data.

3. After the data has been spatially sorted, HOOPS/OOC organizes the points into an appropriate scene graph so that HOOPS can render the data efficiently.  Beginning at the root node, HOOPS/OOC creates a shell and assigns a randomized user-specified number of points to it.  As HOOPS/OOC moves down the tree, the child nodes are representations of the partitions created in phase two. Each child node is populated with a percentage of the data corresponding to the fraction of data that resides in its respective partitioned area. HOOPS/OOC moves recursively until the leaves of the tree are reached and there are no more partitions. This tree structure is written to an OOC file (*<name>.ooc*) while the data that belongs to each node is written into a node file (*.node*). Node files live in a directory with the same root name and at the same level as the OOC file.


Using the ooc.exe preprocessor
==============================

To preprocess a point cloud file, run :doc:`ooc.exe </prog_guide/misc/ooc_preprocessor>` from the command line. This program takes a number of options - but the most important is the input file name, which is passed with the *-f* option. The following example shows how you might preprocess the *MyPointCloud.ptx* ASCII point cloud file::

	> ooc -f ../data/MyPointCloud.ptx -l ../data/MyPointCloudLog.txt

The above command line entry tells :doc:`ooc.exe </prog_guide/misc/ooc_preprocessor>` to process the *../data/MyPointCloud.ptx* file. In the data directory, it creates a *MyPointCloudLog_ptx.ooc</span> file along with a *MyPointCloudLog_ptx_nodes* directory that contains a series of *.node* data files. Additionally, it will also create a *MyPointCloudLog.txt* log file. The preprocessor takes a number of options that lets you control how your data is analyzed and organized. They are as follows:

* **Input filename**: use *-f* to specify the name and path of the ASCII point cloud file. To specify multiple input files, provide multiple *-f filename8 arguments to the  preprocessor or give a *-F filename* with a file containing a list of  point cloud files (one per line).

* **Output filename**:  use *-o* to specify the name and path of the output file which will be appended with ".ooc" for the index file and "_node" for the data directory.	

* **Log file generation**:  use *-l* to specify a log file where data about pre-processing should be written in addition to stdout.

* **Maximum memory usage**: use *-m* to specify the maximum amount of memory in MB that will be used in preprocessing. The default is 512.  Note that this is an approximate value.

* **Maximum shell size**: use *-s* to specify the maximum number of points for a shell at a given node in the point cloud. The default is 10,000.

* **Overwrite existing files**: use *-r* to indicating that HOOPS/OOC should overwrite existing files.

* **Culling bounding box**: use *-b* to specify a bounding box outside of which points will be disregarded.

* **Subsample percentage**: use *-p* to specify a percentage of overall points to import.
