In today’s workforce landscape, where labor shortages and aging populations pose significant challenges, automation has become an urgent necessity. Thanks to advancements in artificial intelligence, robots could form a solution for these pressing issues. With crucial components like batteries and processors becoming more affordable, the potential for widespread robotic adoption across industries is on the horizon. This exciting prospect has sparked a wave of innovative startups and business ventures, all eager to build autonomous robots and vehicles.
Unfortunately, starting a robotics venture is no walk in the park. It’s well-known that “hardware is hard”, but the software needed for robotics is also a challenge on its own. While giants like Tesla can afford to develop everything in-house, most robotics startups simply don’t have the time or resources for that. Luckily, there are software tools specifically geared towards robotics companies. These tools make developing robots a more manageable task by eliminating the need for hard-to-maintain in-house tools.
In 2024, there’s an increasingly wide range of tools available, from intuitive visualization platforms to advanced simulation environments. At Segments.ai, we’re doing our bit by creating data labeling tools, with the aim of making obtaining labeled sensor data easier and more efficient. To shed some light on other helpful tools, we’ve created a market map of software for robotics companies.
In this blog post, we’ll zoom in on the market map and explore the landscape of software tools for robotics in 2024. We’ll look at the key players, from established companies to startups, and see how they can help speed up the development of robots and autonomous vehicles. Feel free to share the market map, and let us know if we forgot any useful tools.
Calibration
Once you’ve decided on the appropriate sensors for your robot and successfully assembled them, the next critical phase in the process is calibration. Calibration is the process of finding the parameters to model each sensor, and the spatial relationships between the sensors. Creating accurate calibration tools can be complex and time-consuming. Advanced calibration software tools, with their sophisticated algorithms and user-friendly interfaces, can streamline this process. Here are three tools you can use for sensor calibration:
Tangram Vision
Tangram Vision offers a comprehensive suite of sensor calibration tools for various devices like cameras, 3D sensors, LiDAR, IMU, and radar. The suite includes products like MetriCal for calibration during the production line and deployment, and AutoCal for online calibration health checks and adjustments. Tangram Vision’s tools provide solutions for precise and accurate calibrations, simplifying development with self-calibration capabilities and offering features like target correction, error covariance, and calibration quality analysis visualizations.
Main Street Autonomy
Main Street Autonomy makes localization, mapping, and calibration software for robotics. Their software offers accurate 3D maps and 6-DOF (degrees of freedom) localization. They offer a fiducial-free calibration system, which only requires a regular log from the robot in its normal environment.
Deepen
Deepen offers sensor calibration tools, as well as a multi-sensor data labeling tool. Their sensor calibration tools support cameras, lidars, radars, IMUs, and more. They are also involved in the Safety Pool initiative, a global initiative to aid the evolution and deployment of safe autonomous driving systems through smart sharing of safety-related data and standards.
Data Visualization
Data visualization tools are crucial for understanding your robot. These tools enable you to visualize the streams of data coming from the various sensors on your robot, such as cameras, lidars, and radars. They also allow you to examine annotations (see below) and the output of your models and algorithms, which is essential for debugging and improving your perception stack.
Rerun.io
Rerun.io is an open-source SDK for visualizing multimodal data that evolves over time. The SDK is available for Rust, C++, and Python. It allows visualizing streams comprising of diverse data types such as images, point clouds, and text. The SDK can be used by engineers at robotics companies to visualizing sensor data from robots, debug outputs of algorithms and machine learning models, and build demos.
Foxglove
Foxglove is an observability platform for robotics developers, offering a comprehensive solution for visualizing and managing robotics data. It allows users to analyze live or pre-recorded data through visualizations like images and point clouds, and supports data hosting, searching, and streaming. Foxglove is also the force behind the open-source MCAP file format, aimed at logging multimodal data from robots efficiently.
rviz
RViz, short for ROS visualization, is an open-source 3D visualization software tool for robots, sensors, and algorithms within the Robot Operating System (ROS) framework. It’s written in C++, and also has Python bindings. It allows engineers to visualize a robot’s perception of its environment, utilizing sensor data to create an accurate representation of the robot’s surroundings.
Voxel51
Voxel51 is the company behind FiftyOne, an open-source toolkit for visualizing and curating computer vision datasets. FiftyOne allows you to visualize sensor data, annotations and model outputs. The toolkit also has powerful data curation/selection features that let you build high-quality datasets which can be used to train computer vision models.
Formant
Formant offers a data platform for robotics. The platform focuses on ingesting live or historic data from various robotic devices or sensors, visualizing this data, and enabling its transfer to external databases for further analysis. Formant also offers solutions for teleoperation, fleet management, and device monitoring.
Data Management
Data visualization tools are crucial for understanding your robot. These tools enable you to visualize the streams of data coming from the various sensors on your robot, such as cameras, lidars, and radars. They also allow you to examine annotations (see below) and the output of your models and algorithms, which is essential for debugging and improving your perception stack.
Roboto.ai
Roboto AI is a startup founded by two Amazon Robotics researchers. They’re building a web-based platform for sensor data management, offering features such as storing sensor data in the cloud, analyzing data for patterns, and transforming data with custom actions. Their stand-out feature is the ability to search through your data in natural language using AI.
Model-Prime
Model-Prime is a cloud platform designed to help robotics companies efficiently manage and analyze the large volumes of logs generated by their robots. It provides tools for rapid data ingestion, fast search capabilities, and metadata enrichment. It also allows you to easily send the data to data visualization and machine learning platforms.
Kognic
Kognic is a data management platform for autonomous vehicles. Kognic helps you assemble ground-truth data pipelines for sensor-fusion. The platform allow you to explore, shape, and explain datasets.
Data Annotation
Because machine learning is playing an increasingly large role in enabling robot autonomy, data annotation has become a crucial task. It involves labeling sensor data, such as images from cameras or point clouds from lidar sensors, to create a dataset that accurately represents the real world. This labeled data serves as the ground truth for training and validating machine learning models. Quality data annotation tools help streamline this labor-intensive process, making it easier for robotics companies to focus on innovation and speed up the development cycle.
Here are three of the best tools for data annotation. Check out our recent blog post on the topic for a longer list of point cloud annotation tools.
Segments.ai
Segments.ai (that’s us!) is a multi-sensor data labeling platform specifically designed for autonomous vehicles and robotics companies. Our platform can be used for segmentation or for annotating objects with cuboids or other vector shapes. Our multi-sensor interface even allows you to label multiple modalities in a single interface. We also have various advanced tools to speed up labeling, an easy-to-use Python SDK, and extensive documentation. You can access the platform and label your data internally, or you can work with one of our professional labeling partners to get data labeled with even less effort.
Scale
Scale offers a data engine for robotics, automotive, logistics companies, and governments. With Scale Rapid, you can completely outsource your data labeling to Scale’s annotation workforce. If you want to access the labeling platform, you can use Scale Studio. It supports images, video, text, audio, and lidar data, although lidar labeling is only available for enterprise customers.
Deepen
Covered under #Calibration above.
Kognic
Covered under #Data Management above.
Localization & Mapping
For mobile robots, whether they’re delivery drones or autonomous vehicles, understanding their position within an environment (localization) and creating or utilizing maps of those environments (mapping) are essential capabilities. These tasks enable robots to navigate effectively, avoid obstacles, and interact with their surroundings meaningfully. Software tools in this category use sophisticated localization and mapping algorithms, giving robots the spatial awareness needed to operate autonomously.
Slamcore
Slamcore is a software platform that provides tools for creating, integrating, and deploying SLAM (Simultaneous Localization and Mapping) systems. The platform offers a range of features for developers, including real-time localization, mapping, and visualization.
Main Street Autonomy
Covered under #Calibration above.
Simulation
Simulation software allows you to test algorithms, hardware configurations, and scenarios in a virtual environment before real-world deployment. This not only accelerates the development cycle by enabling rapid iteration without the need for physical prototypes but also enhances the safety and effectiveness of robots. By providing synthetic data and simulating challenging conditions, these tools allow developers to train and refine machine learning models under a wide range of conditions, ensuring that robots are well-prepared for the complexities of the real world.
Gazebo
Gazebo is an open-source 2D/3D robotics simulator developed by the ROS community. Gazebo provides realistic rendering of environments, supports various physics engines like ODE, and models sensors such as laser range finders and cameras.
AWS RoboMaker
AWS RoboMaker is a cloud-based platform for developing, testing, and deploying robotics applications. The platform offers tools for simulating and deploying robotics applications, as well as integrations with popular robotics frameworks. AWS RoboMaker is designed to be scalable and can handle large amounts of data from multiple sources.
NVIDIA Isaac SIM
NVIDIA Isaac SIM is a simulation platform designed for robotics research and development. It realistic physics simulations, sensor models, and integration with popular robotics frameworks to accelerate the development of autonomous systems. Additionally, it facilitates synthetic data generation for efficient model training and provides learning resources for users.
Unity
Unity provides a comprehensive toolkit for robotics simulation, enabling developers to visualize and debug the internal state of simulations accurately. This platform allows for the creation of highly customizable real-world situations for robot design and simulations, accelerating prototyping with over a million assets available on the Asset Store. Unity offers high-fidelity physics, realistic environments, and supports ROS 2 integration.
Cyberbotics
Cyberbotics is the company behind Webots, an open-source robot simulator. Webots provides a complete development environment to model, program and simulate robots, vehicles and mechanical systems.
MuJoCo
MuJoCo is an open-source tool for physics simulation maintained by Google Deepmind. It is a C/C++ library with a C API, intended for researchers and developers. You can use it as an interactive virtual environment for simulating robots.
Sources
