Abstract
Deformable neural radiance fields (e.g. Nerfies , HyperNeRF, D-NeRF, NR-NeRF ) has been a notable technique to represent dynamic scenes and shows plausible space-time view synthesis results. However, the current implementations only warrant success on teleporting-like videos whose camera motions are significantly more rapid than object motions. Quality of their results significantly decrease on videos with more rapid object motions [4]. For example, in the "broom" example above, both Nefies and HyperNeRF show significant artifacts when trained only using a single camera (i.e. the left camera from the stereo camera rig), as oppose to teleporting between the left and right cameras in the original paper of Nerfies.
In this work, we conjecture the deficiency of these deformable NeRF-based methods is mainly due to lack of temporal regularization. The community has explored optical flow as an additional cue to help supervise the temporal transitions of other motion representations, such as scene flow fields and blend skinning fields. However, enforcing flow constraints with respect to a generic backward deformation field as in Nerfies is non-trivial. We propose an elegant solution to enforce flow constraints for deformable NeRFs, without inverting the backward deformation function. Our method produces plausible view synthesis result, as well as accurate depth map and motions.
