Non-rigid Surface from Motion

Search IconIcon to open search

Notes:

Source: Kumar

  • The problem with dynamic or non-rigid scenes:
    if we project a scene point into a camera image plane, there will be several possible 3D configurations!
  • Allowing arbitrary deformations makes the 3D reconstruction an ill posed problem (underconstrained) –> need to make additional assumptions about the object or scene (make more constraints)!

Source: lamarca-2020 See also: nrsfm-in-defslam , sfm

NRSfM (non-rigid structure from motion)

  • batch processing of images to recover deformation
  • computationally demanding — slower than SfT

Orthographic NRSfM

  • usually fails with very large deformations
  • uses an orthographic camera projection/model
  • (weak approximation to the perspective camera) — a limitation, as many vision-related applications have a significant perspective effect
  • exploits
    • spatial constraints
    • temporal constraints
    • spatiotemporal constraints
  • usually ok for small deformations, but not for very large deformations

Perspective NRSfM

  • the perspective camera model is more accurate than the orthographic one
  • also uses the isometry assumption (as in SfT methods), which has produced good results in NRSfM
  • Parashar 2018 “Isometric NRSfM” [6] local method that handles occlusions and missing data well