truelabelRequest data

Physical AI Data Engineering

How to Annotate 3D Point Clouds for Robotics & Autonomous Systems

3D point cloud annotation transforms raw LiDAR or depth sensor captures into labeled training data by marking object boundaries, semantic classes, and spatial relationships. Core tasks include 3D bounding box placement (6-DOF cuboids around vehicles, pedestrians, obstacles), semantic segmentation (per-point class labels for road, vegetation, buildings), instance segmentation (separating individual objects within a class), and grasp pose labeling (6-DOF affordance annotations for manipulation). Production pipelines combine manual tooling (Segments.ai, Kognic, Scale 3D Sensor Fusion) with automated pre-labeling from PointNet++ or transformer backbones, then validate via inter-annotator agreement metrics and downstream model performance on held-out test scenes.

Updated 2026-05-15
By truelabel
Reviewed by truelabel ·
3D point cloud annotation

Quick facts

Difficulty
Intermediate
Audience
Physical AI data engineers
Last reviewed
2026-05-15

Understanding 3D Point Cloud Data Structures

A point cloud is an unordered set of 3D coordinates (x, y, z) with optional per-point attributes: intensity (LiDAR return strength), RGB color (from camera fusion), surface normals, and timestamps. Autonomous vehicle datasets like Waymo Open Dataset contain 200,000+ labeled frames with up to 250,000 points per scene[1], while indoor robotics datasets such as ScanNet provide RGB-D reconstructions with semantic and instance annotations for 1,513 scanned environments.

Storage formats vary by use case. The PCD (Point Cloud Data) format from the Point Cloud Library stores ASCII or binary point data with flexible schema definitions, while LAS (LASer) files are the ASPRS standard for airborne LiDAR with built-in coordinate reference systems. Robotics pipelines increasingly adopt MCAP for multi-sensor fusion (LiDAR + camera + IMU in a single container) or Parquet for columnar analytics on billion-point datasets. The Point Cloud Library (PCL) provides C++ primitives for filtering, segmentation, and feature extraction across all major formats.

Coordinate frame conventions matter: autonomous vehicle datasets use ego-vehicle frames (forward = +x, left = +y, up = +z) while robotic arms use base-link frames defined in URDF. Transformation matrices between sensor frame, robot base, and world coordinates must be tracked in metadata — a 1° rotation error at 50m range shifts points by 87cm, breaking downstream object detection. Store extrinsic calibration matrices alongside point clouds and validate via static scene reconstruction: if the same wall appears at different depths across frames, your transforms are inconsistent.

Core Annotation Tasks for Robotics Perception

3D bounding box annotation places oriented cuboids around objects of interest. Each box requires 9 parameters: center (x, y, z), dimensions (length, width, height), and orientation (yaw, pitch, roll). Autonomous vehicle annotation focuses on dynamic objects — vehicles, pedestrians, cyclists — with strict occlusion and truncation flags. Kognic's platform reports 12-18 seconds per vehicle box for experienced annotators, while Scale's 3D Sensor Fusion combines LiDAR and camera views to reduce ambiguity in depth estimation.

Semantic segmentation assigns a class label to every point: road, sidewalk, building, vegetation, vehicle, pedestrian. Segments.ai supports 100+ class taxonomies with hierarchical labels (vehicle → car → sedan). Production workflows use PointNet architectures for pre-labeling at 85-92% accuracy, then route low-confidence points (<0.7 softmax score) to human review. The ScanNet benchmark defines 20 indoor object classes with per-point annotations across 1,513 scenes, enabling direct model evaluation.

Instance segmentation separates individual objects within a class — distinguishing three parked cars rather than labeling all as 'vehicle'. This requires point-level cluster IDs plus bounding boxes. Annotation time scales with scene density: a sparse highway scene (8 vehicles) takes 4 minutes, while a crowded urban intersection (40+ objects) requires 25+ minutes[2]. Waymo Open Dataset provides instance masks for 12 million 3D boxes across 1,150 scenes.

Grasp pose labeling for manipulation adds 6-DOF affordance annotations: approach vector, grasp axis, and gripper width. The DexYCB dataset contains 582,000 frames with ground-truth grasps on 20 YCB objects, while robotic teleoperation datasets like DROID capture end-effector poses at 10-30 Hz during task execution. Annotators mark contact points, approach angles, and success labels (stable grasp vs. slip) for imitation learning pipelines.

Selecting Annotation Tools and Platforms

Tool selection depends on data volume, task complexity, and integration requirements. Segments.ai specializes in multi-sensor fusion (LiDAR + camera + radar) with collaborative workflows and API-driven automation, supporting 8+ point cloud formats including PCD, LAS, and custom binary schemas. Labelbox offers model-assisted labeling with active learning loops: pre-label with a PointNet++ backbone, route uncertain predictions to annotators, retrain nightly on corrected labels.

Kognic targets autonomous vehicle annotation with built-in 3D-2D projection: annotators draw 2D boxes in camera views, and the platform back-projects to 3D using LiDAR depth. This reduces annotation time by 40% for occluded objects[2]. Scale AI's 3D Sensor Fusion handles 10+ sensor modalities with quality tiers (Standard, Premium, Specialist) and 24-48 hour SLAs for production pipelines.

Open-source alternatives include CVAT with 3D cuboid plugins and PCL's visualization tools for research prototypes. V7 Darwin and Dataloop provide workflow orchestration (annotation → review → export) with role-based access control for distributed teams. Evaluate tools on: format compatibility (does it parse your sensor's native output?), pre-labeling model support (can you plug in a custom PointNet checkpoint?), export flexibility (COCO 3D, KITTI, nuScenes schemas), and API depth (can you trigger jobs programmatically and poll results?).

For manipulation datasets, Encord supports frame-by-frame pose annotation with interpolation between keyframes, reducing manual effort for 30 Hz teleoperation sequences. iMerit's Ango Hub offers custom taxonomy builders for domain-specific labels (grasp type: pinch vs. power, object state: rigid vs. deformable).

Designing Annotation Taxonomies and Guidelines

A robust taxonomy balances granularity with annotator consistency. Autonomous vehicle taxonomies distinguish 8-12 vehicle subtypes (car, truck, bus, motorcycle, bicycle) plus 3-5 pedestrian states (standing, walking, occluded). Waymo's taxonomy includes 23 object classes with occlusion levels (0-25%, 25-50%, 50-75%, 75-100%) and truncation flags for objects exiting the sensor's field of view.

Guidelines must resolve edge cases: when does a parked car become a static obstacle? How do you label a pedestrian pushing a bicycle? Define decision trees with visual examples. Kognic's annotation guides use 3D renderings showing correct box placement for 20+ ambiguous scenarios (partially open car door, person leaning on vehicle, trailer attached to truck).

For indoor robotics, hierarchical labels improve model transfer: label 'seating' as a parent class, then refine to 'chair', 'sofa', 'stool'. The ScanNet taxonomy uses 3-level hierarchies (furniture → seating → office chair) with 'other' catch-all classes to avoid annotator paralysis. Manipulation tasks require affordance labels beyond object identity: 'graspable', 'pushable', 'articulated' (door, drawer), 'container' (bowl, box).

Consistency checks: run inter-annotator agreement studies on 50-100 scenes. Measure IoU (Intersection over Union) for bounding boxes (target ≥0.85) and per-point accuracy for semantic labels (target ≥0.92). Scale AI reports 0.89 mean IoU on vehicle boxes and 0.94 semantic accuracy on road surfaces across 10,000+ validated frames[3]. Retrain annotators on low-agreement classes and update guidelines with clarifying examples.

Implementing Pre-Labeling and Model-Assisted Workflows

Pre-labeling with deep learning models reduces annotation time by 50-70% while maintaining quality[4]. Train a PointNet++ or transformer backbone on 500-1,000 manually labeled scenes, then deploy for inference on new data. Route predictions with confidence >0.85 directly to training sets; send 0.6-0.85 confidence to human review; fully re-annotate <0.6 confidence points.

Labelbox's Model-Assisted Labeling API accepts model predictions in COCO 3D or nuScenes format, renders them in the annotation UI, and tracks human edits for active learning. After each review batch, retrain the model on corrected labels — this closed-loop approach improves pre-labeling accuracy by 8-12 percentage points over 3-4 iterations[5].

Automated quality checks catch systematic errors. Validate bounding box dimensions against physical priors: if a 'car' box is 15m long, flag for review. Check point density: if a box contains <50 points, it may be a false positive or poorly placed. Segments.ai runs rule-based validators (box-ground intersection, occlusion consistency, temporal tracking across frames) and surfaces violations in a review queue.

Temporal consistency for video sequences: track object IDs across frames and enforce smooth motion. If a vehicle's bounding box jumps 3m between consecutive frames (0.1s apart), the annotation is likely incorrect. Waymo's tracking annotations maintain unique IDs for 1,000+ objects across 20-second clips, enabling motion prediction model training. Use Kalman filters or optical flow to propagate labels forward, then human-verify keyframes (every 10th frame) and interpolate.

Scale's Rapid platform combines pre-labeling, human review, and automated QA in a single pipeline with 24-48 hour turnaround for 10,000-point scenes. Dataloop supports custom model deployment via Docker containers, letting you plug in proprietary architectures trained on internal data.

Validating Annotation Quality at Scale

Quality validation requires both statistical metrics and downstream task performance. Inter-annotator agreement measures consistency: have 2-3 annotators label the same 100 scenes, compute IoU for bounding boxes (target ≥0.85) and per-point F1 for semantic labels (target ≥0.90). Low-agreement classes indicate ambiguous guidelines or insufficient training.

Automated consistency checks catch outliers. Flag boxes with aspect ratios outside expected ranges (car length/width should be 2.0-3.5), points labeled as 'road' above 2m height, or instance masks with <20 points. Kognic's QA pipeline runs 15+ rule-based validators and routes violations to senior reviewers, achieving 98% label accuracy on production autonomous vehicle datasets[2].

Downstream model performance is the ultimate test. Train an object detector (PointPillars, CenterPoint) on annotated data and evaluate on a held-out test set. If mAP (mean Average Precision) is <0.70 for common classes (vehicles, pedestrians), annotation quality is likely the bottleneck. Waymo's 3D detection benchmark reports 0.78-0.82 mAP for state-of-the-art models on their labeled dataset, setting a quality bar for production systems.

Temporal validation for tracked objects: compute track fragmentation (how often an object ID is lost and reassigned) and ID switches (how often two objects swap IDs). Scale AI targets <2% ID switch rate and <5% fragmentation on 20-second clips[3]. High fragmentation indicates poor cross-frame consistency or occlusion handling.

Run ablation studies: train models on subsets with different annotation quality levels (strict QA vs. relaxed QA) and measure performance delta. If strict QA improves mAP by <0.02, your QA process may be over-tuned. If delta is >0.08, invest more in quality — the annotation bottleneck is real and directly impacts model ROI.

Handling Multi-Sensor Fusion and Calibration

Multi-sensor annotation fuses LiDAR point clouds with camera images, radar returns, and IMU data for richer context. Waymo Open Dataset provides synchronized LiDAR (top + 4 side-mounted), 5 cameras (front, sides, rear), and IMU at 10 Hz, with extrinsic calibration matrices for all sensors. Annotators label in 3D LiDAR space, and the platform projects labels onto camera views for visual verification.

Calibration drift degrades fusion quality. Validate extrinsics by projecting LiDAR points onto camera images and checking alignment: if a vehicle's LiDAR points land 20 pixels off the camera bounding box, recalibrate. Scale's 3D Sensor Fusion runs automated alignment checks on every scene and flags calibration errors for manual correction.

Temporal synchronization matters for moving objects. If LiDAR and camera timestamps differ by 50ms, a vehicle moving at 15 m/s appears 75cm displaced between modalities. MCAP's timestamp indexing enables microsecond-precision alignment across sensors, critical for 30+ Hz annotation pipelines. Store sensor-to-sensor latency in metadata and apply temporal offsets during fusion.

Segments.ai's multi-sensor workflows let annotators toggle between LiDAR, camera, and fused views in real-time, with automatic 3D-2D projection. This reduces annotation time by 30-40% for occluded objects: draw a 2D box in the camera view where the object is visible, and the platform back-projects to 3D using LiDAR depth[4].

Radar fusion adds velocity information. Waymo's radar annotations include per-object radial velocity, enabling motion prediction without temporal differencing. Annotate radar returns as 'associated' (matched to a LiDAR object) or 'unassociated' (ghost return, clutter), and label velocity confidence (high/medium/low based on SNR).

Scaling Annotation Pipelines for Production

Production pipelines process 10,000-100,000 scenes per month, requiring workflow automation and quality monitoring. Scale AI's Rapid platform handles 50,000+ scenes/month for autonomous vehicle customers with 24-48 hour SLAs, using a hybrid workforce (in-house specialists + vetted contractors) and tiered review (annotator → reviewer → specialist for edge cases).

Workforce management balances cost and quality. Appen and Sama provide managed annotation teams with domain training (2-4 weeks for 3D LiDAR specialists). CloudFactory's autonomous vehicle solution reports 95% label accuracy with 3-5 day turnaround for 1,000-scene batches[6]. iMerit offers dedicated teams for long-term contracts (6-12 months), reducing onboarding overhead and improving consistency.

API-driven workflows enable programmatic job creation and result polling. Labelbox's Python SDK lets you upload point clouds, assign to projects, trigger pre-labeling models, and export results in COCO 3D or KITTI format — all via code. Dataloop's REST API supports webhook callbacks when annotation batches complete, triggering downstream training pipelines automatically.

Cost optimization: pre-labeling reduces human effort by 50-70%, cutting per-scene cost from $15-25 to $5-10[3]. Tiered review (10% of scenes get senior review, 90% get standard review) balances quality and throughput. Segments.ai charges $0.08-0.15 per labeled point for semantic segmentation and $8-15 per 3D bounding box, with volume discounts at 10,000+ scenes.

Quality monitoring dashboards track annotator performance (accuracy, speed, agreement with gold-standard labels) and surface training needs. Kognic's analytics show per-annotator IoU distributions, flag outliers (<0.80 mean IoU), and recommend retraining or reassignment. Maintain a 500-1,000 scene gold-standard set with expert-verified labels for ongoing calibration.

Exporting and Integrating Annotated Data

Export formats must match downstream training frameworks. Waymo Open Dataset uses TFRecord with protocol buffer schemas, while nuScenes defines JSON metadata + binary point cloud files. Labelbox exports to COCO 3D (bounding boxes + segmentation masks), KITTI (3D detection benchmark format), and custom JSON schemas with per-point attributes.

KITTI format stores 3D boxes as 15-value rows: class, truncation, occlusion, alpha (observation angle), 2D bbox (4 values), 3D dimensions (3 values), 3D location (3 values), rotation_y. Point clouds are separate binary files (x, y, z, intensity). Segments.ai and Scale AI both support KITTI export for compatibility with academic benchmarks.

nuScenes format uses JSON for metadata (sample, scene, instance, annotation) and separate.pcd.bin files for point clouds. Each annotation includes category, instance token (for tracking), visibility (1-4 scale), and 3D box parameters. Kognic exports nuScenes-compatible datasets with tracking IDs and velocity annotations for motion prediction tasks.

Custom schemas for robotics manipulation: LeRobot's dataset format stores point clouds in HDF5 with per-frame observation/action pairs, while RLDS (Reinforcement Learning Datasets) uses TFRecord episodes with nested point cloud tensors. Dataloop supports custom export templates via Jinja2, letting you map annotations to arbitrary JSON or binary formats.

Validation on export: run format compliance checks (does every box have 9 parameters? are point coordinates within sensor range?), schema validation (does JSON match the target spec?), and sample visualization (render 10 random scenes to catch projection errors). Truelabel's data provenance tracking embeds annotation metadata (tool version, annotator ID, review status, timestamp) in exported files for audit trails and model reproducibility.

Annotation for Specific Robotics Domains

Autonomous vehicles prioritize dynamic object detection (vehicles, pedestrians, cyclists) with occlusion and truncation labels. Waymo Open Dataset contains 12 million 3D boxes across 1,150 scenes with 23 object classes and 4 occlusion levels. Kognic's autonomous vehicle platform adds lane-level road topology (centerlines, boundaries, crosswalks) and traffic light state annotations for planning model training.

Indoor mobile robots need semantic segmentation for navigation: floor, walls, doors, furniture, clutter. ScanNet's 1,513 scenes provide per-point labels for 20 indoor classes plus instance masks for movable objects. Segments.ai supports hierarchical taxonomies (furniture → seating → chair) for transfer learning across environments.

Manipulation tasks require grasp affordance labels: approach vector, grasp axis, gripper width, contact points. The DexYCB dataset contains 582,000 frames with 6-DOF grasp poses on 20 YCB objects, while DROID's 76,000 trajectories capture end-effector poses at 10-30 Hz during real-world tasks. Encord's pose annotation tools support keyframe labeling with interpolation for 30 Hz teleoperation sequences.

Warehouse robotics combines semantic segmentation (racks, pallets, boxes) with instance-level tracking for inventory management. CloudFactory's industrial robotics solution annotates 3D box dimensions, barcode locations, and stacking stability labels for pick-and-place planning. Claru's teleoperation warehouse dataset includes 12,000+ labeled grasps on deformable packaging with success/failure annotations.

Agricultural robotics requires plant-level instance segmentation, growth stage labels, and disease detection. Point clouds from RGB-D cameras or LiDAR capture crop rows, individual plants, and fruit clusters. Segments.ai supports custom taxonomies for 50+ crop types with phenotypic attributes (height, leaf count, flowering stage).

Managing Annotation Projects and Teams

Project setup defines scope, timeline, and quality targets. Specify: number of scenes (1,000? 10,000?), annotation tasks (bounding boxes only, or semantic + instance segmentation?), quality SLA (≥0.85 IoU, ≥0.90 semantic F1), and delivery schedule (1,000 scenes/week for 10 weeks). Scale AI recommends 2-4 week pilot projects (500-1,000 scenes) to validate guidelines and tooling before scaling to 10,000+ scenes.

Annotator training takes 1-2 weeks for 3D LiDAR specialists. Provide: taxonomy documentation with visual examples, tool tutorials (platform-specific UI walkthroughs), edge case decision trees (20+ ambiguous scenarios with correct labels), and practice datasets (50-100 scenes with gold-standard labels for self-assessment). Appen and Sama run certification exams: annotators must achieve ≥0.85 IoU on 50 test scenes before production work.

Review workflows catch errors before export. Implement two-stage review: peer review (another annotator checks 10% of scenes) and expert review (senior annotator checks 5% of scenes, focusing on low-confidence predictions and edge cases). Labelbox's review queues route flagged scenes to reviewers with side-by-side comparison (original vs. corrected labels) and comment threads for feedback.

Performance tracking: monitor per-annotator metrics (scenes/hour, IoU, semantic accuracy, agreement with gold-standard) and identify training needs. Kognic's dashboards show annotator leaderboards, flag outliers (<0.80 mean IoU), and recommend retraining or reassignment. Maintain a 500-1,000 scene gold-standard set with expert-verified labels for ongoing calibration.

Communication cadence: weekly syncs with annotation teams to review quality metrics, clarify edge cases, and update guidelines. Dataloop's collaboration tools support in-platform comments, @mentions, and issue tracking for asynchronous feedback on specific scenes.

Cost and Timeline Planning

Annotation costs vary by task complexity and quality tier. 3D bounding boxes: $2-5 per box for vehicles (12-18 seconds per box), $8-15 per box for complex objects (furniture, machinery) requiring precise orientation[2]. Semantic segmentation: $0.08-0.15 per labeled point, or $50-150 per scene (100,000-200,000 points) depending on class count and density[4]. Instance segmentation: $80-200 per scene for cluttered environments (20+ objects).

Timeline estimates for 10,000-scene projects: 8-12 weeks with pre-labeling and tiered review, 16-20 weeks for fully manual annotation. Scale AI's Rapid platform delivers 1,000 scenes/week with 24-48 hour per-batch turnaround, while CloudFactory reports 3-5 day turnaround for 1,000-scene batches at 95% accuracy[6].

Cost optimization strategies: pre-labeling reduces human effort by 50-70%, cutting per-scene cost from $15-25 to $5-10. Active learning (route only uncertain predictions to humans) reduces annotation volume by 30-40% while maintaining model performance. Labelbox's model-assisted labeling reports 60% time savings on vehicle detection tasks[5].

Quality vs. speed tradeoffs: standard review (1 annotator, automated QA) costs $8-12 per scene with 48-72 hour turnaround. Premium review (2 annotators, expert review on 10% of scenes) costs $15-20 per scene with 5-7 day turnaround but achieves 0.90+ IoU vs. 0.82-0.85 for standard[3]. Choose based on downstream model sensitivity: safety-critical systems (autonomous vehicles) justify premium quality, while research prototypes may accept standard.

Volume discounts: Segments.ai offers 20-30% discounts at 10,000+ scenes, while Appen and Sama negotiate custom pricing for 50,000+ scene contracts. Truelabel's marketplace connects buyers with pre-annotated datasets, reducing time-to-model from 12 weeks (custom annotation) to 1-2 weeks (dataset licensing + fine-tuning).

Common Pitfalls and How to Avoid Them

Inconsistent coordinate frames break multi-sensor fusion. Validate that LiDAR, camera, and robot base frames use consistent conventions (ROS REP-103: x-forward, y-left, z-up vs. robotics convention: x-forward, y-right, z-up). Store transformation matrices in metadata and verify via static scene reconstruction: if the same wall appears at different depths across frames, your transforms are wrong.

Annotation drift occurs when guidelines evolve mid-project without retraining annotators. Lock taxonomy and guidelines before starting production annotation, and version all changes. If you must update guidelines, re-annotate a 500-scene calibration set and measure delta vs. old labels. Kognic versions annotation projects and flags scenes annotated under old guidelines for optional re-review.

Ignoring occlusion and truncation degrades model performance. Waymo's taxonomy requires occlusion labels (0-25%, 25-50%, 50-75%, 75-100%) and truncation flags for objects exiting the sensor FOV. Models trained without these labels hallucinate complete objects from partial observations, causing false positives in production.

Sparse point clouds (< 50 points per object) yield unreliable bounding boxes. Filter objects below a point density threshold or flag for manual review. Segments.ai's QA pipeline automatically flags boxes with <30 points and routes to expert review.

Temporal inconsistency in tracked objects: if a vehicle's bounding box jumps 3m between consecutive frames (0.1s apart), the annotation is likely incorrect. Use Kalman filters or optical flow to propagate labels forward, then human-verify keyframes (every 10th frame) and interpolate. Scale AI targets <2% ID switch rate and <5% fragmentation on 20-second clips[3].

Underestimating review overhead: plan for 10-20% of annotation time spent on review and corrections. Labelbox's two-stage review (peer + expert) adds 15-25% to project duration but improves final accuracy by 8-12 percentage points[5].

Future Trends in Point Cloud Annotation

Foundation models for pre-labeling are reducing annotation costs. NVIDIA's Cosmos world foundation models generate synthetic 3D scenes with ground-truth labels, enabling zero-shot transfer to real LiDAR data. Transformer-based architectures pre-trained on 10 million+ unlabeled point clouds achieve 88-92% semantic segmentation accuracy out-of-the-box, requiring only 500-1,000 labeled scenes for domain adaptation.

Synthetic data generation via simulation (NVIDIA Isaac Sim, Gazebo, Unity) produces infinite labeled point clouds for rare scenarios (pedestrian jaywalking, vehicle swerving). Domain randomization techniques vary lighting, object textures, and sensor noise to improve sim-to-real transfer. Scale AI's partnership with Universal Robots combines real teleoperation data with synthetic augmentation for manipulation policy training.

Active learning reduces annotation volume by 30-50%. Train a model on 1,000 labeled scenes, run inference on 10,000 unlabeled scenes, and annotate only the 2,000 scenes with highest prediction uncertainty (entropy, least confidence). Encord Active automates this loop: model inference → uncertainty ranking → annotation queue → retraining.

Automated quality validation via self-supervised learning: train a model to predict annotation errors (low IoU, incorrect class) from point cloud features and annotator behavior (time spent, edit patterns). Labelbox's ML-powered QA flags 15-20% of scenes for expert review, catching 85% of errors while reviewing only a fraction of output[5].

Standardized benchmarks for annotation quality: the Waymo Open Dataset and nuScenes define gold-standard labels and evaluation metrics (mAP, IoU, tracking accuracy), enabling apples-to-apples comparison of annotation vendors. Truelabel's marketplace requires dataset sellers to report benchmark scores, giving buyers objective quality signals before purchase.

Use these to move from category-level context into specific task, dataset, format, and comparison detail.

External references and source context

  1. Dataset page

    Waymo Open Dataset contains 200,000+ labeled frames with up to 250,000 points per scene and 12 million 3D boxes across 1,150 scenes with 23 object classes

    waymo.com
  2. kognic.com articles

    Kognic reports annotation time scales with scene density: sparse highway 4 minutes, crowded intersection 25+ minutes

    kognic.com
  3. scale.com physical ai

    Scale AI's 3D Sensor Fusion handles 50,000+ scenes/month with 24-48 hour SLAs and reports 0.89 mean IoU on vehicle boxes

    scale.com
  4. segments.ai the 8 best point cloud labeling tools

    Pre-labeling reduces annotation time by 50-70% and per-scene cost from $15-25 to $5-10; semantic segmentation costs $0.08-0.15 per labeled point

    segments.ai
  5. docs.labelbox.com overview

    Labelbox offers model-assisted labeling with active learning loops and reports 60% time savings on vehicle detection; two-stage review improves accuracy by 8-12 percentage points

    docs.labelbox.com
  6. cloudfactory.com autonomous vehicles

    CloudFactory autonomous vehicle solution reports 95% label accuracy with 3-5 day turnaround for 1,000-scene batches

    cloudfactory.com

FAQ

What is the difference between semantic and instance segmentation for point clouds?

Semantic segmentation assigns a class label (vehicle, pedestrian, road) to every point, treating all objects of the same class as a single entity. Instance segmentation separates individual objects within a class, assigning unique IDs to each car, person, or tree. Instance segmentation requires per-point cluster IDs plus bounding boxes, taking 3-5x longer to annotate but enabling object tracking and counting. Autonomous vehicle datasets like Waymo Open provide both: semantic labels for static scene understanding (road layout, vegetation) and instance masks for dynamic objects (12 million tracked vehicles across 1,150 scenes). Manipulation tasks typically need instance segmentation to distinguish multiple graspable objects in cluttered scenes.

How long does it take to annotate 1,000 LiDAR scenes for autonomous vehicles?

Timeline depends on task complexity and quality tier. For 3D bounding box annotation only (vehicles, pedestrians, cyclists), expect 8-12 weeks with pre-labeling and tiered review, or 16-20 weeks for fully manual annotation. Scale AI's Rapid platform delivers 1,000 scenes/week with 24-48 hour per-batch turnaround using hybrid workflows (model pre-labeling + human review). CloudFactory reports 3-5 day turnaround for 1,000-scene batches at 95% accuracy. Adding semantic segmentation (per-point labels for road, vegetation, buildings) increases timeline by 40-60%. Cost ranges from $8,000-12,000 for standard quality (1 annotator, automated QA) to $15,000-20,000 for premium quality (2 annotators, expert review on 10% of scenes).

What point cloud formats do annotation tools support?

Most platforms support PCD (Point Cloud Data) from the Point Cloud Library, LAS (LASer) for airborne LiDAR, and custom binary formats. Segments.ai handles 8+ formats including PCD, LAS, PLY, and custom schemas with flexible parsers. Labelbox and Scale AI accept KITTI binary (.bin), nuScenes (.pcd.bin), and Waymo Open Dataset (TFRecord protocol buffers). For robotics, MCAP is emerging as the standard for multi-sensor fusion (LiDAR + camera + IMU in a single container), supported by Segments.ai and Dataloop. Parquet is used for billion-point datasets requiring columnar analytics. Check format compatibility before selecting a tool — converting between formats risks data loss (intensity values, timestamps, coordinate frame metadata).

How do you validate point cloud annotation quality?

Use three validation layers: inter-annotator agreement (2-3 annotators label the same 100 scenes, compute IoU for boxes and per-point F1 for semantic labels, target ≥0.85 and ≥0.90 respectively), automated consistency checks (flag boxes with invalid aspect ratios, points labeled as 'road' above 2m height, instance masks with <20 points), and downstream model performance (train an object detector on annotated data, evaluate on held-out test set, target ≥0.70 mAP for common classes). Kognic's QA pipeline runs 15+ rule-based validators and achieves 98% label accuracy on production datasets. For tracked objects, measure ID switch rate (<2% target) and track fragmentation (<5% target). Run ablation studies: if strict QA improves model mAP by <0.02, your process is over-tuned; if delta is >0.08, invest more in quality.

What is the cost difference between manual and model-assisted annotation?

Model-assisted annotation (pre-labeling with PointNet++ or transformers, then human review of uncertain predictions) reduces costs by 50-70% compared to fully manual workflows. Manual annotation costs $15-25 per scene for 3D bounding boxes plus semantic segmentation; model-assisted reduces this to $5-10 per scene. Labelbox reports 60% time savings on vehicle detection tasks using model-assisted labeling. The upfront cost is training a pre-labeling model on 500-1,000 manually labeled scenes ($5,000-10,000), but this amortizes quickly: at 10,000 scenes, total cost drops from $150,000-250,000 (manual) to $55,000-110,000 (model-assisted). Active learning (annotate only high-uncertainty predictions) adds another 30-40% savings. Premium review (expert validation on 10% of scenes) costs $15-20 per scene but achieves 0.90+ IoU vs. 0.82-0.85 for standard.

How do you handle multi-sensor fusion (LiDAR + camera) in annotation workflows?

Multi-sensor annotation fuses LiDAR point clouds with camera images for richer context. Annotators label in 3D LiDAR space, and platforms project labels onto camera views for visual verification. Waymo Open Dataset provides synchronized LiDAR (5 sensors at 10 Hz), 5 cameras, and IMU with extrinsic calibration matrices. Validate calibration by projecting LiDAR points onto camera images: if vehicle points land 20+ pixels off the camera bounding box, recalibrate. Temporal synchronization matters — if LiDAR and camera timestamps differ by 50ms, a vehicle at 15 m/s appears 75cm displaced. MCAP's microsecond-precision timestamp indexing enables accurate alignment. Segments.ai and Kognic let annotators toggle between LiDAR, camera, and fused views in real-time, reducing annotation time by 30-40% for occluded objects: draw a 2D box where the object is visible in camera, and the platform back-projects to 3D using LiDAR depth.

Looking for 3D point cloud annotation?

Specify modality, task, environment, rights, and delivery format. Truelabel matches you with vetted capture partners — every delivery includes consent artifacts and commercial licensing by default.

List Your Point Cloud Dataset