Learning-Guided Dense SLAM for Structured Interior Reconstruction
A metric, uncertainty-aware RGB-D mapping system that extends a learned dense-SLAM backend with deployable scale anchoring, TSDF fusion, Manhattan structure recovery, visibility-based opening validation, and selective abstention when room-level evidence is insufficient.
Headline results
The project uses Freiburg1 Room as the positive structural reconstruction case and Freiburg1 Desk as a scale-stress and partial-coverage negative control. The same evidence policy accepts the room and abstains on Desk.
System architecture
The learned SLAM backend is treated as one subsystem inside a larger geometry and reliability stack. Each stage exports machine-readable diagnostics so later decisions can be audited rather than inferred from a final visualization.
1 · Trajectory evaluation exposed a scene-dependent scale failure
Freiburg1 Room showed similar SE(3) and Sim(3) trajectory error, but Freiburg1 Desk produced a very different pattern: the path shape was accurate after Sim(3) alignment while the metric translations were compressed. A matched path-length ratio of 1.121 and an Umeyama scale of 1.113 independently indicated approximately 11% scale error.
| Sequence / metric | SE(3) ATE RMSE | Sim(3) ATE RMSE | Translational RPE RMSE |
|---|---|---|---|
| Freiburg1 Room | 6.28 cm | 6.07 cm | 6.93 cm |
| Freiburg1 Desk | 9.25 cm | 1.64 cm | 11.07 cm |
2 · Ground-truth-free RGB-D metric scale anchoring
The diagnostic Sim(3) correction uses ground-truth poses and cannot be deployed. I therefore estimated a single metric translation scale from RGB-D odometry between nearby MASt3R keyframes. Candidate pairs were filtered using baseline range, rotation disagreement, translation-direction consistency, and scale bounds; a robust median with MAD filtering produced the final anchor.
T_i→j^SLAM = T_Wj^-1 · T_Wi
scale_ij = (t_SLAM · t_RGB-D) / (t_SLAM · t_SLAM)
s* = robust_median({scale_ij})
3 · Dense-fusion ablation
I fused the same synchronized RGB-D observations under three estimated-pose variants and compared each result with a ground-truth-pose TSDF fusion: uncorrected SE(3), ground-truth-derived Sim(3), and the deployable RGB-D metric scale. Surface distances were recomputed over five fixed sampling seeds to separate geometry differences from random sampling variation.
| Pose variant | Surface RMSE | Within 2 cm | Role |
|---|---|---|---|
| Uncorrected SE(3) | 4.405 ± 0.015 cm | 33.37 ± 0.19% | Deployable baseline |
| GT-derived Sim(3) | 2.267 ± 0.074 cm | 69.83 ± 0.17% | Diagnostic upper-bound alignment |
| RGB-D metric scale | 2.106 ± 0.008 cm | 71.04 ± 0.20% | Ground-truth-free correction |
4 · Metric room structure rather than only a dense map
The Room sequence was converted from a colored TSDF mesh into a structured representation. The pipeline estimates floor and ceiling, recovers a Manhattan frame, separates outer perimeter planes from internal persistent planes, and exports both a conservative closed model and a hypothesis model containing only visibility-supported architectural openings.
5 · Visibility-aware opening validation
A wall-occupancy detector initially produced several gaps, including a narrow doorway-like hypothesis. Occupancy alone cannot distinguish a true opening from missing coverage or foreground occlusion, so each wall cell was projected into aligned RGB-D frames. Measured depth was compared with the expected wall depth and aggregated across views:
Wall observed
Measured depth agrees with the expected wall depth within tolerance.
Occluded
Measured depth is closer than the wall; foreground geometry blocks the test cell.
Free-space evidence
Repeated valid depth is farther than the expected wall, supporting an opening hypothesis.
| Region | Dimensions | Confidence | Architectural decision |
|---|---|---|---|
| y-min region | 0.80 × 0.75 m | 0.816 | Probable window in hypothesis model only |
| x-max anomaly 1 | 0.15 × 0.50 m | 0.740 | Rejected: size and placement inconsistent with accepted opening rules |
| x-max anomaly 2 | 0.50 × 0.40 m | 0.701 | Rejected: size and placement inconsistent with accepted opening rules |
6 · Selective structural inference and abstention
A production-quality system should not always return a room. I introduced an envelope-level confidence policy that requires opposing persistent plane pairs, sufficient floor support, at least three strongly observed walls, and weak-but-supported evidence for any remaining perimeter wall. The exact same policy was evaluated on a positive and negative control.
Freiburg1 Room
- Reliable floor and ceiling evidence
- Opposing persistent wall-plane pairs
- Three strong walls
- One weak-but-supported wall
Freiburg1 Desk
- Furniture-dominated partial vertical geometry
- No persistent opposing X-wall pair
- No persistent opposing Y-wall pair
- No reliable ceiling plane
Engineering contributions
Geometry and estimation
- SE(3), Sim(3), ATE, RPE, and matched-path diagnostics
- Robust RGB-D pairwise metric-scale estimator
- Estimated-pose, scale-corrected, and GT-pose TSDF ablations
- Floor, ceiling, Manhattan wall, perimeter, and internal-plane extraction
- Multi-view expected-depth versus measured-depth opening tests
Reliability and systems
- Conservative versus hypothesis model variants
- Strong, weak-but-supported, and unsupported wall states
- Explicit structural abstention on partial scene coverage
- Repeated deterministic evaluation over fixed sampling seeds
- GLB/PLY/JSON exports, source release, checksums, and manifest validation
Evaluation scope and limitations
- The quantitative study currently covers two TUM RGB-D sequences, not a broad multi-dataset benchmark.
- The RGB-D scale estimator assumes synchronized metric depth and known camera intrinsics.
- TSDF comparisons reuse the same RGB-D data and do not replace evaluation against an independent high-accuracy scan.
- The probable window has geometric free-space support but no RGB semantic confirmation.
- The generic structural coverage gate is an acceptance mechanism; the refined Room pipeline provides the final dimensions.
- Furniture, reflective surfaces, thin geometry, missing depth, and motion blur can reduce wall and opening evidence.
Next technical extensions
Move from one global scale anchor to a robust temporal scale state with uncertainty and change detection.
Add RGB/VLM evidence for doors, windows, walls, materials, and functional room components.
Test complete and partial scans across TUM, Replica, ScanNet, and self-captured mobile RGB-D sequences.
Summary
I extended learned dense SLAM into an auditable interior-reconstruction system. On Freiburg1 Desk, a robust ground-truth-free RGB-D scale anchor estimated 1.099 versus a 1.113 diagnostic reference and reduced repeated TSDF surface RMSE from 4.405 cm to 2.106 cm. On Freiburg1 Room, the pipeline recovered a 5.599 × 4.347 × 2.898 m structured envelope, retained one geometry-supported window hypothesis, and exported conservative and hypothesis models. The same structural confidence policy accepted Room and abstained on the partial Desk sequence rather than inventing unsupported geometry.
Dense-SLAM backend: MASt3R-SLAM by Riku Murai, Eric Dexheimer, and Andrew J. Davison. This page describes an independent engineering and evaluation extension built on the public implementation.