The Three Stages of a Dataset
Data acquisition and processing is done in three steps:
Scanning: Records images and data such as laser scans and Wi-Fi signals, and stores it on a disk.
Post-processing: Turns raw data into finished Point clouds, maps, and panorama images. While those artifacts are technically available live during the scan, they are re-computed during post-processing with higher quality and greater accuracy.
Web processing: Prepares the post-processed data for NavVis IVION.
The data from each of these steps is kept in directories called datasets. A dataset contains all the data of the corresponding scan run according to the current stage of the processing. A project directory looks like this:
<base directory>
datasets_rec/<dataset>
datasets_proc/<dataset>
datasets_web/<dataset>
Wherein:
<base directory>
is the project directory, typically named for the project's working title or the customer's name.datasets_rec
contains the raw Recorded datasets.datasets_proc
contains the processed datasets obtained by post-processing the Recorded datasets.datasets_web
contains the datasets ready for NavVis IVION.The actual name of
<dataset>
is the date and time when it was recorded.
Note: While it is technically possible to process data in arbitrary places, we strongly recommend you follow the directory structure above. The tools working on multiple datasets at the same time require this structure.
Note: Never delete files or directories from the datasets under
datasets_rec
. These represent the original data; all other files can be generated from there. That means the minimum required for backup are all the folders indatasets_rec/*
.
Note: Project directory names must not contain spaces or umlauts.
Transferring Data to a PC
Before you start uploading, create the directories described above on your computer. Make sure you have access to those directories.
Transferring Data from the SSD
Take the hard drive out of the powered-down device and connect it to your PC using a SATA-to-USB adapter. Once your PC recognizes the disk, you can drag and drop the directories you need to copy using your file explorer. By default, recorded data is stored on the SSD in data/datasets_rec/<date_time>/
.
After you have successfully transferred all data, delete the data from the device to make room for future scans.
FAQ
What are the three stages of a dataset?
The three stages of a dataset are Scanning, Post-processing, and Web processing.
What is the purpose of the scanning stage?
The scanning stage records images and data such as laser scans and Wi-Fi signals, and stores it on a disk.
What does post-processing do to the raw data?
Post-processing turns raw data into finished Point clouds, maps, and panorama images with higher quality and greater accuracy.
What is stored in the datasets_rec directory?
The datasets_rec directory contains the raw Recorded datasets.
Can I delete files from the datasets_rec directory?
No, you should never delete files or directories from the datasets_rec directory as they represent the original data.
How should I transfer data to my PC?
Create the necessary directories on your computer, then connect the hard drive to your PC using a SATA-to-USB adapter to drag and drop the required directories.
What should I do after transferring all data from the device?
After transferring all data, delete the data from the device to make room for future scans.
What is the naming convention for the dataset folders?
The actual name of the dataset is the date and time when it was recorded.
Is it necessary to follow the recommended directory structure?
Yes, it is strongly recommended to follow the directory structure to ensure compatibility with tools working on multiple datasets.