Template substitution in DefSLAM

Parent:  Mapping step-by-step in DefSLAM

Source: lamarca-2020

• Tracking runs at frame-rate, and mapping at keyframe-rate
• Tracking processes Nm frames during a whole mapping run

Process

1. New keyframe $k$ is made. Now at time $t=k$
   • At this point, the template in the tracking is still based on the old shape-at-rest, S_(k-1)
   • Mapping thread starts
     • creates surface S_k which is aligned to prev. template T_(k-1)
     • k is set as the reference keyframe
     • from S_k, create template T_k and from now on use this template instead of the old one T_(k-1)
2. At time t=k+Nm, use data from the tracking thread
   • image points at t=k+Nm
   • deform the recently computed template T_k based on these images
     • use SfT but neglecting the temporal term (to allow large deformation, “as a lot might have happened in the time span of Nm”)
     • so now we get a T_k that is deformed (updated) to the most recent image points
   • we do this extra step instead of passing T_k (from step 1) to the tracker immediately because, due to the new points occurring at t=k+Nm, using the original T_k might lead to data association errors
   • mapper passes the new template T_k (t=k+Nm) to the tracker