Multi-line-lasers and camera calibration using AR marker based on DNN for underwater dense 3D shape reconstruction using ROV
Recently, the importance of marine resources is noticed and demand for underwater 3D reconstruction is increasing. Since there are critical differences between in the air and underwater environment, it is hard to apply active 3D reconstruction methods using a camera and pattern projectors to the underwater scene.

In this sudty, we applied a laser-baesd 3D reconstruction method underwater and built a real system on an underwater ROV. Specifically, we used an approximation camera model and specially designed devices which make line laser projectors distortion-free as solution for the refreaction. And as the solution of the hardness during underwater calibration, we propose DNN based algorithm for robustly marker detection.

System overview
Figure 1. System configuration (left) and the real system (right)

AR-marker Detection by DNN

When in the air, it is possible to fix the position of calibration board and scanner device then capture laser-on/off images for calibration, whereas it is difficult for underwater environment due to the strong water flow. In the water, we can only capture laser-on images, and the laser pattern is overlapped on the AR-markers, severely interfering the marker detection.
As the solution, we made a new DNN to directly detect the AR-markers' positions and IDs from laser-on images. We also applied an auto-rescale method to deal with the large scale difference of AR-markers in different data sets.

As test, we used 5 datasets taken in the air and underwater in different condition such as a swimming pool and sea. By comparing the detected results by exsiting method (ARToolKit) and our DNN method, it is clearly confirmed that our method performed effectively.

Marker detection
Figure 2. Examples of AR-marker detection
(left: input image; center: DNN output mask; right: comparison of detected result of ARToolKit and DNN method, blue ones are detected results by ARToolKit, red ones are added markers detected by DNN method)

Underwater 3D Reconstrution Results

We also reconstructed objects in a pool by manually combining the results of several frames for demonstration. As shown in Figure 3, the surface of objects and the bottom of the pool are clearly separated, which demonstrated the effectiveness of our method.

Result
Figure 3. Reconstruction result of a table and a calibration board.

Publications
  • Hanbin Wang, Genki Nagamatsu, Takafumi Iwaguchi, Naoki Shirakura, Jun Takamatsu, Hiroshi Kawasaki, Multi line-lasers ROV for underwater dense 3D shape reconstruction using marker-based calibration by DNN, MIRU2021, 4 pages, 2021
  • Hanbin Wang, Takafumi Iwaguchi, Hiroshi Kawasaki, ROBUST CALIBRATION-MARKER AND LASER-LINE DETECTION FOR UNDERWATER 3D SHAPE RECONSTRUCTION BY DEEP NEURAL NETWORK, IEEE International Conference on Image Processing (ICIP), 5 pages, 2022
Kawasaki Laboratory