Robotics · Calibration · Reliability

CalibGraph

A robust multi-camera calibration and health-monitoring system for cameras mounted on different moving links of an articulated robot. The project connects classical hand–eye calibration to observability, robust graph refinement, mechanical-drift isolation, camera–robot time synchronization, and held-out task-space error.

System thesis: calibration quality is not just a solver-selection problem. A deployable calibration stack must verify motion observability, detect violated noise assumptions, distinguish geometric failure from clock mismatch, monitor mount stability, and validate the final result through robot-frame task error.

AX=XB SE(3) Multi-Camera Articulated Robots Robust Least Squares Observability Time Synchronization Health Monitoring

Browse code on GitHub Download reproducible release Run experiments

Actual benchmark summaries from the final synthetic prototype: articulated sensing geometry, held-out task error, motion observability, drift monitoring, and time synchronization.

Headline results

The final evaluation uses separate calibration and validation trajectories. Results therefore measure downstream robot-frame task error rather than only fit residuals on calibration observations.

59.7%

Reduction in held-out grasp error using Joint Huber under 8% gross outliers

85.8%

Reduction in held-out grasp error using time-aware calibration under timestamp mismatch

100%

Detection for medium and large mechanical mount drift across 20 trials

34 passed

Automated geometry, solver, monitoring, time-sync, and task-space tests

Conditional policy: PARK remains the preferred clean-data baseline; robust joint optimization is activated for outlier contamination, and time-aware refinement is activated for camera–robot clock mismatch.

System architecture

Collect and qualify motionGenerate or record link poses and target observations, then reject rank-deficient or weak rotational trajectories.

Independent initializationCalibrate every camera against its carrying link using classical OpenCV hand–eye solvers.

Robust joint refinementOptimize all camera mounts and one shared target pose with linear or Huber residuals.

Diagnose time and driftEstimate per-camera clock offsets and monitor persistent geometric inconsistency after deployment.

Validate robot tasksMeasure held-out grasp-point, surface-point, approach-normal, and action-acceptance errors.

Frame model and optimization

The transform convention is T_A_B: coordinates in frame B mapped into frame A. For camera C_i on link L_i:

p_A   = T_A_B @ p_B
T_A_C = T_A_B @ T_B_C

T_B_T = T_B_L_i(q_t) @ T_L_i_C_i @ T_C_i_T(t)

Relative motions recover the classical equation A_k X = X B_k. The joint graph minimizes a normalized six-dimensional SE(3) consistency residual:

E_i,t = inverse(T_B_T) @ T_B_L_i(t) @ T_L_i_C_i @ T_C_i_T(t)

r_i,t = [
  translation(E_i,t) / sigma_translation,
  LogSO3(rotation(E_i,t)) / sigma_rotation
]

When the camera and robot clocks differ, the carrying-link pose must be evaluated at t + delta_i rather than at the camera timestamp alone.

T_B_T =
  T_B_L_i(t + delta_i)
  @ T_L_i_C_i
  @ T_C_i_T(t)

Motion observability before solver selection

The same solver can be accurate or catastrophic depending on the calibration trajectory. CalibGraph measures angular excitation, axis diversity, translation baseline, and design rank before trusting the result.

Motion regime	Rotation rank	Quality	Engineering interpretation
Diverse multi-axis motion	8	GOOD	Well observed
Single-axis motion	6	POOR	Rotation not uniquely constrained
Small rotations	8	POOR	Full rank but practically ill-conditioned
Translation only	0	POOR	No rotational observability

Translation error under robot motion degeneracy

Median calibration error grows catastrophically when the robot trajectory violates rotational observability assumptions.

Robust joint refinement is condition-dependent

Joint optimization was not automatically better. Under clean Gaussian observations, PARK was both faster and more accurate. Under 8% gross target-pose outliers, Huber refinement prevented corrupted factors from pulling all camera estimates away from the shared target.

Calibration condition	Method	Mean grasp error	Precision acceptance	Mean calibration runtime
Gaussian	PARK	0.727 mm	98.9%	14.1 ms
Gaussian	Joint Huber	0.765 mm	98.9%	427.5 ms
8% outliers	PARK	2.186 mm	45.6%	14.0 ms
8% outliers	Joint Huber	0.882 mm	96.1%	472.9 ms

Independent and joint calibration under outliers

Ordinary joint least squares propagated corrupted measurements; robust Huber refinement remained stable.

Held-out grasp error under calibration outliers

The robust improvement transfers to a separate robot trajectory and a downstream grasp-point metric.

Mechanical drift monitoring and camera isolation

The monitor learns robust healthy residual distributions from an initial calibration window. Persistent critical scores request recalibration. Raw cross-camera disagreement detects that the system is unhealthy; leave-one-camera-out consistency identifies the mount most likely to have moved.

z = max(0, (error - median(error)) / max(1.4826 * MAD, minimum_scale))

health = sqrt(z_translation² + z_rotation² + z_cross_camera²)

isolation(i)
  = mean distance(camera i, peers)
    - mean distance among the peers

Injected drift	Detection	Correct camera isolation	False alarms
1 mm / 0.2°	55%	55%	0%
3 mm / 0.5°	100%	95%	0%
8 mm / 1.5°	100%	100%	0%

Calibration health score after mechanical drift

A persistent health-state transition separates ordinary observation noise from meaningful mount movement.

Camera–robot time synchronization

On moving links, pairing an image with the wrong encoder timestamp creates an apparent extrinsic error. CalibGraph alternates between PARK calibration and bounded per-camera time-offset estimation.

100 ms offset-level metric	Zero-offset PARK	Time-aware PARK	Reduction
Extrinsic translation error	18.03 mm	2.39 mm	86.7%
Target localization error	66.35 mm	8.81 mm	86.7%
Cross-camera disagreement	70.97 mm	9.38 mm	86.8%
Mean time-offset error	70.0 ms assumption error	8.80 ms	87.4%

Per-camera offsets are estimated jointly with repeated extrinsic recalibration.

Held-out grasp error with time synchronization

Time correction reduces held-out fused grasp error from 19.78 mm to 2.80 mm.

Reproduction

python3 -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements.txt
python -m pip install -e .
python -m pytest -q

python scripts/benchmark_target_pose_noise.py
python scripts/benchmark_motion_observability.py
python scripts/benchmark_multicamera_independent.py
python scripts/benchmark_joint_multicamera.py
python scripts/benchmark_drift_monitoring.py
python scripts/benchmark_time_synchronization.py
python scripts/benchmark_task_space_validation.py

The release contains source code, 34 tests, phase-by-phase technical notes, CSV summaries, raw trial tables, environment information, and generated figures.

Limitations and next validation step

The current evidence is synthetic; no physical robot or real-camera experiment is claimed.
Target observations are simulated as noisy SE(3) poses rather than estimated from raw corner detections.
The nominal kinematic chain is exact unless a corruption experiment explicitly changes it.
The time-aware method is an offline diagnostic and is much slower than closed-form PARK.
Task-acceptance thresholds are explicit simulated geometric criteria, not a safety or production guarantee.

Next step: connect the same interfaces to real image detections, encoder timestamps, measured kinematic uncertainty, and a physical robot task while preserving the observability, health, and held-out validation gates.