Modern machine learning models can be used to detect and track objects in 3D point clouds, or to classify every single point (3D point cloud segmentation). To create such a model, you need a lot of data, i.e., labeled point clouds. Labeling point clouds manually is typically very difficult and time-consuming. But what if point cloud labeling could actually get easier over time? That’s exactly what model-assisted labeling promises.
Model-assisted labeling means using model predictions to speed up the labeling process. The trick is to train a model on a small number of labeled point clouds and then using that model to create predictions for the unlabeled point clouds. The predictions won’t be perfect, but correcting annotations is still a whole lot easier than creating them from scratch. Better still, as you label more and more point clouds, you can retrain your model to improve its predictions. Labeling will thus get faster over time.
In this guide, we’ll set up model-assisted labeling using Segments.ai, a data annotation platform for computer vision teams. Segments.ai’s Python SDK makes it easy to implement model-assisted labeling for both point cloud segmentation and 3D object detection (cuboid annotation).
Read along here, or try it out yourself using our interactive notebooks.
Notebooks on Colab
- Fast point cloud cuboid annotation:
- Fast point cloud segmentation:
1. Create a dataset and import data
The first step is to create a dataset on Segments.ai and import the point clouds you want to label.
- Create a dataset using the Python SDK with the
client.add_dataset()method or using the web interface. - Import data using
client.add_sample()or via the Add Samples button on the Samples tab of the dataset in the web interface. When adding samples, you can first upload your files to Segments.ai’s asset storage service usingclient.upload_asset()or simply pass the links to the data stored in a cloud bucket. More information: https://docs.segments.ai/guides/import-data.
2. Label a part of the data
To kickstart model-assisted labeling, you’ll need to label a subset of your data manually. If you want to train a model for object detection, this means drawing cuboids (3D bounding boxes) around every object in a point cloud. If you’re creating a model for semantic or panoptic segmentation, this means labeling all the individual points in the point cloud.
Luckily, Segments.ai has simple interfaces for both point cloud segmentation and cuboid annotation. Take a look at the docs to discover how to use the labeling interfaces.
3. Train a model
Once you have labeled some point clouds, you can train your first model. Depending on the task you want to achieve, you might be able to fine-tune an existing model instead of training a model from scratch. In the notebooks, we use a pre-trained SqueezeSegV3 model for point cloud segmentation and a pre-trained CenterPoint model for 3D object detection.
To get the labeled point clouds from Segments.ai, first create a release using client.add_release() or using the web interface. Then, you can create a SegmentsDataset containing only the labeled point clouds.
1
2
release = client.get_release(dataset_identifier, release_name)
dataset = SegmentsDataset(release, labelset='ground-truth', filter_by=['LABELED', 'REVIEWED'])
4. Generate and upload label predictions
Next, you simply have to run the model on the unlabeled samples in your dataset and upload the model predictions. Use the client.add_label() method to upload the model predictions to Segments.ai.
1
2
3
4
5
6
7
8
9
10
release = client.get_release(dataset_identifier, release_name)
dataset = SegmentsDataset(release, labelset='ground-truth', filter_by='UNLABELED')
for sample in dataset:
pred_annotations = run_model(sample)
attributes = {
'format_version': '0.2',
"annotations": pred_annotations,
}
client.add_label(sample['uuid'], 'ground-truth', attributes)
5. Correct the predictions and repeat
Now you can go back to labeling the rest of your data. This time, you won’t have to start labeling from scratch though. You simply have to correct the predictions your model made. After labeling some more point clouds, you can go back to step 3 to improve your model. This way, labeling the remaining point clouds will become faster and faster.
Conclusion
Model-assisted labeling is a technique where you correct model predictions to speed up data annotation. Model-assisted labeling can make labeling point clouds a lot faster. In this guide, you learned how you can use Segments.ai to implement model-assisted learning. Don’t forget to check out the full notebooks on Colab if you want to try it yourself.
Hope this was useful! If you have any questions or suggestions, feel free to send us an email at support@segments.ai
