Treffer: SurgIPC: a convex image perspective correction method to boost surgical keypoint matching.
Lowe DG (2004) Distinctive image features from scale-invariant keypoints. In: IJCV.
Mikolajczyk K, CORDELIA S (2004) Scale and affine invariant interest point detectors. In: IJCV.
Morel J-M, Yu G (2009) Asift: a new framework for fully affine invariant image comparison. SIAM J Imaging Sci.
Toft C, Turmukhambetov D, Sattler T, Kahl F, Brostow GJ (2020) Single-image depth prediction makes feature matching easier. In: ECCV.
Rodriguez M, Facciolo G, Gioi RG, Muse P, Delon J (2020) Robust estimation of local affine maps and its applications to image matching. In: WACV.
DeTone D, Malisiewicz T, Rabinovich A (2018) Superpoint: Self-supervised interest point detection and description. In: CVPRW.
Chandelon K, Sharifian R, Marchand S, Khaddad A, Bourdel N, Mottet N, Bernhard J-C, Bartoli A (2023) Kidney tracking for live augmented reality in stereoscopic mini-invasive partial nephrectomy. Computer Methods in Biomechanics and Biomedical Engineering: Imaging and Visualization.
Kim J-H, Bartoli A, Collins T, Hartley R (2012) Tracking by detection for interactive image augmentation in laparoscopy. In: WBIR.
Zeisl B, Köser K, Pollefeys M (2012) Viewpoint invariant matching via developable surfaces. In: ECCV, Workshops and Demonstrations.
Wang S, kannala J, Pollefeys M, Barath D (2023) Guiding local feature matching with surface curvature. In: ICCV.
Lévy B, Petitjean S, Ray N, Maillot J (2002) Least squares conformal maps for automatic texture atlas generation. ACM Trans Graphics.
Gluckman J, Nayar SK (2001) Rectifying transformations that minimize resampling effects. In: CVPR, Kauai, Hawaii, USA.
Riemann B (1851) Grundlagen Für Eine Allgemeine Theorie der Functionen Einer Veränderlichen Complexen Grösse, Göttingen, Germany. PhD thesis.
Sarlin P-E, DeTone D, Malisiewicz T, Rabinovich A (2020) Superglue: Learning feature matching with graph neural networks. In: CVPR, Virtual.
Sun J, Shen Z, Wang Y, Bao H, Zhou X (2021) Loftr: Detector-free local feature matching with transformers. In: CVPR, Virtual.
Riba E, Mishkin D, Ponsa D, Rublee E, Bradski G (2020) Kornia: An open source differentiable computer vision library for pytorch. In: WACV.
Ranftl R, Lasinger K, Hafner D, Schindler K, Koltun V (2022) Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer. In: TPAMI.
Cui B, Islam M, Bai L, Wang A, Ren H (2024) Endodac: Efficient adapting foundation model for self-supervised depth estimation from any endoscopic camera. In: MICCAI.
Schonberger JL, Frahm J-M (2016) Structure-from-motion revisited. In: CVPR.
Schönberger JL, Zheng E, Frahm J-M, Pollefeys M (2016) Pixelwise view selection for unstructured multi-view stereo. In: ECCV.
Shao S, Pei Z, Chen W, Zhu W, Wu X, Sun D, Zhang B (2022) Self-supervised monocular depth and ego-motion estimation in endoscopy: appearance flow to the rescue. Med Image Anal 77:102338.
Budd C, Vercauteren T (2024) Transferring relative monocular depth to surgical vision with temporal consistency. In: MICCAI.
Weitere Informationen
Purpose: Keypoint detection and matching is a fundamental step in surgical image analysis. However, existing methods are not perspective invariant and thus degrade with increasing surgical camera motion amplitude. One approach to address this problem is by warping the image before keypoint detection. However, existing warping methods are inapplicable to surgical images, as they make unrealistic assumptions such as scene planarity.
Methods: We propose Surgical Image Perspective Correction (SurgIPC), a convex method, specifically a linear least-squares (LLS) one, overcoming the above limitations. Using a depth map, SurgIPC warps the image to deal with the perspective effect. The warp exploits the theory of conformal flattening: it attempts to preserve the angles measured on the depth map and after warping, while mitigating the effects of image resampling.
Results: We evaluate SurgIPC under controlled conditions using a liver phantom with ground-truth camera poses and with real surgical images. The results demonstrate a significant improvement in the number of correct correspondences when SurgIPC is applied. Furthermore, experiments on downstream tasks, including keyframe matching and 3D reconstruction using structure-from-motion (SfM), highlight significant performance gains.
Conclusion: SurgIPC improves keypoint matching. The use of LLS ensures efficient and reliable computations. SurgIPC can thus be easily included in existing computer-aided surgery systems.
(© 2025. CARS.)
Declarations. Conflict of interest: Rasoul Sharifian declares that there is no conflict of interest regarding the publication of this paper. Adrien Bartoli declares that there is no conflict of interest regarding the publication of this paper.