Improved Image Viewer for 3D Point Cloud Labeling

By Tobias Cornille on March 27th, 2023

At Segments.ai, we’re constantly striving to improve our platform for labeling 3D point clouds. One of the ways we’re doing this, is by enabling you to upload camera images alongside your point clouds, providing important context for labelers. To view these images, we recently shipped an improved image viewer with a number of new features:

  1. Synchronized Image Viewer

    When hovering in the perspective or top view, a synchronized image viewer appears in the bottom right of the screen, zoomed to the location of the pointer in the closest camera image. This feature helps labelers quickly identify and annotate objects in the scene.

  2. View the Active Cuboid in the Synchronized Image Viewer

    When a cuboid is selected, the synchronized image viewer shows the active cuboid in every camera image where it appears, in a carousel. This feature makes it easier for users to verify and correct the labeling of objects.

  3. Select Images using Thumbnails

    The main viewer now displays all the available camera images as thumbnails in the main labeling window. When you press on a thumbnail, it opens the new image viewer in the main labeling window.

  4. Zoom/Pan in the Image Viewer

    In the new image viewer, you can zoom and pan to explore the camera image in more detail, while the 3D point cloud and objects, such as cuboids and vectors, are overlaid on top of it. This allows labelers to get a better understanding of the relationship between the 3D scene and the camera image.

    • To zoom on the image, scroll or double click on the image.
    • To pan on the image, hold Command/Control and drag the image. This key can be changed in the hotkey settings.

  5. Toggle Point Cloud Overlay

    You can now toggle the point cloud overlay on and off in the image viewer. Disabling the point cloud overlay allows you to see the camera image in more detail, while enabling it allows you to see if the camera and lidar are well calibrated.

    • To toggle the point cloud overlay on and off, press the hotkey (Command/Control + q by default) or click on the icon in the toolbar on the right.

We believe that these features will greatly enhance the labeling experience for our users, providing them with more context and insights to make more accurate and efficient labeling decisions.

To use the new features, start by uploading images alongside your point clouds. It’s important to note that for 3D overlays to work in the image viewer, you need to upload camera calibration parameters (intrinsics and extrinsics). Please have a look at our documentation for more details.

We hope that you’ll find these updates useful, and as always, we welcome your feedback and suggestions on how we can continue to improve Segments.ai.

Tobias
Tobias Cornille
Share: