LiDAR (Light Detection and Ranging) sensors emit laser pulses and measure the time they take to return, producing dense 3D maps of the environment as clouds of millions of data points. Unlike cameras, LiDAR is unaffected by lighting conditions and provides precise depth information — which is why it is the primary sensor for autonomous vehicle safety systems and industrial robotics.
What Is a Point Cloud?
A point cloud is a collection of data points in 3D space, each defined by X, Y, and Z coordinates. High-density automotive LiDAR sensors produce 100,000 to 2,000,000 points per second. Each point may also carry intensity (reflectivity) and timestamp data. The result is a volumetric representation of the scene that captures depth and spatial relationships that cameras cannot.
Core Point Cloud Annotation Tasks
3D Bounding Box Annotation
The most common task: fitting a 3D box (defined by center position, length, width, height, and yaw angle) around each object in the scene. Unlike 2D bounding boxes, 3D boxes must be oriented correctly — a car's bounding box must align with the vehicle's heading, not with the sensor coordinate frame. This requires annotators to manipulate 3D visualizations from multiple viewing angles simultaneously.
Semantic Segmentation of Point Clouds
Assigning a class label (road, building, vegetation, vehicle, pedestrian, cyclist) to every point in the cloud. This is computationally and ergonomically demanding — annotators must classify millions of points per scan, typically using a combination of automated pre-labeling and human correction. Semantic segmentation of point clouds is essential for HD map production and terrain analysis.
Instance Segmentation
Beyond class labels, instance segmentation identifies each unique object — distinguishing between individual pedestrians, vehicles, and cyclists rather than labeling them all as a single semantic class. Required for multi-object tracking and interaction modeling.
Multi-Frame Tracking
Fusing consecutive LiDAR scans and maintaining consistent object IDs across frames — the 3D equivalent of video object tracking. As vehicles and pedestrians move through the scene, their 3D bounding boxes must be updated per frame with the correct position, orientation, and ID. Handling occlusion in 3D is particularly complex: a pedestrian occluded from the sensor's view in one frame must reappear with the same ID.
Applications Beyond Automotive
- Construction and surveying: Annotating building structures, terrain features, and site equipment from drone-mounted LiDAR.
- Forestry and agriculture: Classifying tree species, canopy density, and crop health from aerial point clouds.
- Industrial robotics: Labeling parts, bins, and workspace boundaries for robot manipulation and navigation.
- Infrastructure inspection: Identifying cracks, corrosion, and structural anomalies in bridges, pipelines, and utility networks.
- Indoor mobile robots: Mapping floor plans, obstacles, and dynamic objects for warehouse automation.
Technical Challenges in Point Cloud Annotation
- Sparsity at distance: LiDAR point density decreases with distance from the sensor, making distant small objects (pedestrians 50m away) very difficult to annotate accurately.
- Occlusion and gaps: Objects partially blocked by other objects have missing points that annotators must interpolate based on contextual understanding.
- Sensor-specific calibration: Annotation boxes must account for the specific mounting position and orientation of the sensor on the vehicle.
- Annotation tool performance: Point cloud viewers must render millions of points in real-time while supporting complex interaction — annotator productivity depends heavily on tool performance.
Quality Standards for 3D Annotation
- 3D IoU threshold: Target 3D IoU > 0.7 for vehicle classes, > 0.5 for pedestrians (smaller objects have higher variance).
- Orientation accuracy: Heading angle error < 10 degrees for vehicles — critical for motion prediction models.
- Completeness: All objects above the minimum size threshold must be annotated — missed objects cause false negatives that are especially dangerous in safety-critical applications.
- Multi-frame consistency: Object dimensions (length, width, height) should not vary significantly frame-to-frame for a stationary or slow-moving object.