Panorama Streaming API Usage Guide

Overview

Use this guide when you want to show IVION panoramas in a third-party application. You need access to an IVION instance with a site that contains panoramas. Follow this guide to download panorama tiles and depth images and place them in the site coordinate system, aligned with point clouds and maps.

Note:

• You download panorama files with pre-signed URLs from the IVION API, in the same way as point cloud files in the Point Streaming API Usage Guide.

• Each panorama is stored in a single NVR archive that contains all of its tiles. See the NVR File Format Specification.

• The streaming APIs require API access, which is included in IVION Professional and IVION Enterprise but not in IVION Core. Without it, API requests fail with HTTP 403 and the message API not licensed.

• You can use the C++ IVION Streaming API Sample Application as a reference in addition to this document. It shows panoramas, maps and point clouds together.

• All placeholders in this document use curly brackets {}. For example, {instance_url}

For a description of all IVION API endpoints, see the IVION API documentation.

Logging in

Third-party applications log in with a login mandate. The application requests a mandate, the user authorizes it in the IVION web application, and the application exchanges it for an access token and a refresh token. For more information, see the sections Logging In and Refreshing the Access Token in Point Streaming API Usage Guide.

All API requests in this guide require the header X-Authorization: Bearer {access_token}. Downloads with pre-signed URLs do not require it.

Note:

Some IVION instances allow anonymous access. On such instances, you can skip the login: all requests in this guide, including the request for a pre-signed download URL, also work without the X-Authorization header, with the permissions of an anonymous user.

Listing sites

Retrieve the list of sites available to the user through the following endpoint:

Endpoint:

GET {instance_url}/api/sites
X-Authorization: Bearer {access_token}

Response Code & Status:

200 OK

Response Body:

[
  {
    "id": number,
    "name": string,
    "description": string | null,
    ...
  },
  ...
]

Listing datasets

Panoramas belong to datasets. Retrieve the datasets of a site through the following endpoint:

Endpoint:

GET {instance_url}/api/site/{site_id}/datasets
X-Authorization: Bearer {access_token}

Response Code & Status:

200 OK

Response Body:

[
  {
    "id": number,
    "name": string,
    "title": string,
    "storage_prefix": string,
    "reference_system_id": number,
    "has_depth_images": boolean,
    "is_epano_dataset": boolean,
    ...
  },
  ...
]

Listing panoramas

Retrieve the panoramas of a dataset through the following endpoint:

Endpoint:

GET {instance_url}/api/site/{site_id}/images/filter?dataset={dataset_id}
X-Authorization: Bearer {access_token}

Response Code & Status:

200 OK

Response Body:

[
  {
    "id": number,
    "dataset_id": number,
    "file_id": string,
    "storage_file_path": string,
    "scs_location": [number, number, number],
    "scs_orientation": [number, number, number, number],
    ...
  },
  ...
]

The endpoint supports further filters, for example by bounding box or by distance from a location. To retrieve a single panorama, use GET {instance_url}/api/site/{site_id}/images/{id}. See the API documentation for all parameters.

Requesting a pre-signed download URL

Panorama files are downloaded with pre-signed URLs, in the same way as point cloud files. A pre-signed URL is scoped to a storage prefix and can be reused for all files under that prefix.

Endpoint:

POST {instance_url}/api/site/{site_id}/storage/download/prefix/signed/url
X-Authorization: Bearer {access_token}

Request Body:

{
  "prefix_in_site": "{storage_prefix}",
  "expiry_in_millis": number
}

Response Code & Status:

200 OK

Response Body:

{
  "url": string,
  "issued_at_millis": number,
  "expires_at_millis": number
}

The field expiry_in_millis is optional. If you omit it, the instance's default validity is used, which is usually 5 minutes. The instance rejects requests for a longer validity than its maximum.

To download a file, insert the file's storage path into the path of the returned URL, before the query string:

{signed_base_url}/{storage_file_path}{signed_query_string}

Note:

• The storage path of a file starts with the site ID, for example {site_id}/{storage_prefix}/pano/00398.nvr.

• Monitor expires_at_millis and request a new URL before it expires. Do not rely on a fixed validity, because it depends on the instance configuration.

• Cache one signed URL per storage prefix. Avoid requesting several signed URLs for the same prefix in parallel.

• Usually, the URL points to a CDN. On instances without a CDN, the URL is a path on the IVION instance. If the URL starts with /, resolve it against the origin (scheme, host and port) of the instance.

• Signed URLs support HTTP Range requests. No X-Authorization header is needed.

For panoramas, use the dataset's storage_prefix as prefix_in_site. The URL of a panorama's archive is then:

{signed_base_url}/{storage_file_path}{signed_query_string}

Reading the NVR archive

All tiles of a panorama are stored in a single NVR archive. The archive's index is stored at the end of the file, so a client can download any tile with an HTTP Range request after reading the index once. The format is described in the NVR File Format Specification. In short:

1. Request the last 2048 bytes of the archive with Range: bytes=-2048.

2. Read the 42-byte footer at the end: number of files N (uint32, big-endian, at offset 32) and archive type (uint8, at offset 36).

3. Read the index: N big-endian uint32 file lengths directly before the footer. If the index does not fit into the first response, request the last 4 × N + 42 bytes.

4. The first file starts at byte 4. Each following file starts where the previous one ends.

5. Download a file with Range: bytes={offset}-{offset + length - 1}. Files with a length of 0 are missing.

Panorama layout

IVION stores panoramas in one of two projections. The projection follows from the archive type in the footer of the NVR archive:

Both projections have four color resolution levels (0 = lowest, 3 = highest) and one level of depth tiles (PNG). The file index in the archive is:

color tile t at level r:  r × T + t
depth tile t:             4 × T + t

All tiles of a level have the same size in pixels. Take the size from the decoded image.

Texture coordinates

In this guide, the texture coordinates (u, v) of a tile have their origin at the bottom-left corner of the tile image: u runs from left (0) to right (1), and v from bottom (0) to top (1). Directions are given in the panorama's local frame, which has Z pointing up.

Equirectangular tiles

An equirectangular panorama is divided into 8 columns and 4 rows of tiles, each covering 45° × 45°. Tile t lies in column t mod 8 and row t div 8, where row 0 is at the top:

Azimuth is measured counter-clockwise from the local +X axis, seen from above. Column 0 covers azimuth 180° to 135°, and azimuth decreases from left to right, so the center of the full panorama faces +X. The direction for a texture coordinate (u, v) of tile t is:

column    = t mod 8
row       = t div 8
azimuth   = 180° − (column + u) × 45°
elevation = 90° − (row + 1 − v) × 45°
direction = (cos(elevation) × cos(azimuth),
             cos(elevation) × sin(azimuth),
             sin(elevation))

To display a tile, approximate its part of the sphere with a grid of vertices, for example 11 × 11, and map the tile image onto it.

Cubemap tiles

A cubemap panorama has six faces. Each face is divided into 2 columns and 4 rows of tiles, so each tile covers half the width and a quarter of the height of a face. Tile t belongs to face t div 8; within the face, it lies in column (t mod 8) mod 2 and row (t mod 8) div 2, where row 0 is at the top. The faces lie on a cube with edge length 1 centered at the panorama center:

With the face normal n, the image right axis r and the image up axis w from the table, the point on the cube for a texture coordinate (u, v) of tile t is:

column = (t mod 8) mod 2
row    = (t mod 8) div 2
s      = (column + u) / 2 − 0.5
h      = 0.5 − (row + 1 − v) / 4
point  = 0.5 × n + s × r + h × w

Each tile is a flat rectangle, so its four corners are enough to display it.

Choosing a resolution level

Load level 0 of all tiles first. This gives a complete, low-resolution panorama with few requests. Then load higher levels only for the tiles in the field of view, starting with the tiles closest to the view direction. For wide fields of view, level 2 is usually sufficient; for narrow fields of view, use level 3.

Positioning panoramas

To display a panorama in the site coordinate system, place the panorama's tile geometry at scs_location and rotate it with the quaternion scs_orientation. A direction d in the panorama's local frame corresponds to the site direction q × d × q⁻¹, where q = (w, x, y, z) is scs_orientation.

When the camera is at the panorama center, the size of the tile geometry does not matter. Render the panorama without depth testing, or scale it so that it lies between the near and far planes.

Site coordinates:

• Panorama poses and map poses are given in the site coordinate system, a right-handed metric coordinate system with Z pointing up.

• Point clouds are stored in dataset coordinates. To show them together with panoramas and maps, apply the dataset's site transformation, as described in the Applying Alignment section of the Point Streaming API Usage Guide.

Depth images

If a dataset has depth images (has_depth_images is true), each panorama archive contains one depth tile per color tile. Depth tiles use the same layout and texture coordinates as the color tiles, at a single resolution level. They are stored as PNG images with three 8-bit channels. The distance from the panorama center along the viewing ray, in millimeters, is:

distance_mm = R + 256 × G + 65536 × B

A value of 0 means that no depth is available for this pixel. Read depth images with nearest neighbor sampling; interpolating between neighboring pixels produces invalid distances.

To compute the site coordinate of a pixel, multiply the normalized site direction of the pixel by the distance:

point = scs_location + normalize(q × d × q⁻¹) × distance_mm / 1000

A panorama archive may contain 0-length entries for all depth tiles, for example if depth images were not generated. Treat this the same as has_depth_images being false.

360-degree camera panoramas

Panoramas captured with external 360-degree cameras are stored in datasets with is_epano_dataset set to true. They use the same endpoints and the same storage format as other panoramas and are usually equirectangular. Such datasets usually have no point cloud and no depth images. The images filter endpoint includes them by default; add include_epano_images=false to exclude them.

Maps

To show the panoramas on a floor map, see the Map Streaming API Usage Guide. Maps use the same site coordinate system.

FAQ