Japanese page >>

Research

Large scale scene modeling, understanding and synthesis
	MOTSLAM: MOT-assisted monocular dynamic SLAM using single-view depth estimation [2022]
	In this work, we present MOTSLAM, a dynamic visual SLAM system with the monocular configuration that tracks both poses and bounding boxes of dynamic objects.
	Attention based Accident Detection in Dashboard Cameras [2021]
	In this study, we propose a method for estimating accidents from video captured on a drive recorder using a deep learning with attention mechanism. For preprocess, self-annotation method for tracking of moving objects in video was also developed.
	Estimating Sudden Braking using Weather Estimation by a Deep Learning Framework from Drive Recorder Data [2020]
	To analyze the occurrence of sudden braking, we use deep learning to identify emergency braking data using weather estimated from drive recorder video.
	Flooded detection from drive recorder video using deep learning (GAN) [2018--2019]
	We use deep learning, specifically GAN, to extend the data in order to detect the unusual event like flooded scene from drive recorder image sequence.
	City name estimation by larning large image dataset using random forest [2014]
	There are many anonymous on-vehicle video uploaded on the Internet. Our technique can estimate the place where the video is captured by learning based approach using random forest. Since our technique uses not only single image, but also several frames, it is expected to improve accuracy than common techniques. In the experiments, we selected 10 cities and tested to show the effectiveness of our method.
	Efficient On-Vehicle Video Localization by Space-time Matching [2012--2014]
	On-vehicle video localization without GPS is necessary. In our research it is achieved by matching between real data and digital map. To increase the stability and accuracy, space-time feature is extracted from both video data and digital map.
	Photorealistic Wide-Area Real-time Rendering [2012--2013]
	We make photo-realistic wide-area rendering using billboard represents general shape of urban scene captured car-mounted camera. And also we apply PCA compression technique to texture, reconstruct texture on GPU as faster than faster, implement real-time rendering.
	Video synthesis with multiple cameras [2008--2009]
	Like photo-synthesis, 3D Reconstruction of urban scene can be realized by multiple videos captured by car-mounted video cameras.
	Real-time Rendering using Eigen Space Compression [2007--2008]
	We compress the sequence of omni directional image based on IBR(Image Based Rendering), and render a large scale scene in real-time. This operation is all linear, and because we run it in the fragment-shader of the graphics card, real-time rendering is achieved.
	Space-Time Image Analysis [2000--2007]
	This research is about generating and analyzing space-time images by fixing the image sequence (video) to a time base direction.
	Large scale scene modeling [2000--2009]
	Multiple video cameras are set up on the car, and take real world images in various directions. The panorama image is generated by integrating multiple images.
	Image based large scale scene rendering [2000--2005]
	Large scale scene rendering is done by IBR/MBR hybrid method using generated panorama images. Efficient image compression technique is also proposed for image based method, which used geometry.

Active 3D shape acquisition and analysis
	High-density and high-speed one-shot measurement system by U-Net interpolation [2021--2022]
	We propose a pattern interpolation technique to densify the 3D shape measurement by using U-net and an accuracy improvement technique based on MRF optimization for oneshot active scan.
	Underwater 3D mapping using light-sectioning method and V-SLAM [2019--2021]
	We developed a 3D mapping system using ROV attached with multiple cross-line-lasers. Self-calibration of multiple laser-lines are also proposed.
	Underwater Camera and Laser Calibration using AR marker for ROV [2019--2020]
	We applied a laser-based 3D reconstruction method underwater by AR-marker detection using DNN, and built a real system on an underwater ROV.
	Unified Underwater Structure-from-Motion [2018--2019]
	We enable 3D reconstruction on various scenarios by representing any geometry settings of underwater SfM with an unified model.
	Underwater projected pattern stereo using Multi-scale CNN [2018--2019]
	Accurate and dense dynamic shape reconstruction technique for underwater environment by projected pattern stereo and Multi-scale CNN.
	Temporal shape super-resolution by intra-frame motion encoding using high-fps structured light [2017]
	Research on a new 3D measurement method that super-resolves the 3D shape in the temporal direction from one image taking a fast moving object
	Active one-shot scan for wide depth range using a light field projector based on coded aperture [2012--2015]
	Three dimensional measurement in a wide depth range is realized by installing the coded aperture in the projector
	Calibration method for Projector [2009--2014]
	We are developing an efficient and simple calibration method for a projector and camera systems. Gray code version, which requires multiple captures, an extended version using checker pattern, and AR marker.
	3D shape recconstruction by using projection defocus [2013]
	By analyzing the blurring caused by lens, depth of the captured scene can be estimated.
	Dense High Speed 3D reconstruction method By One-shot technique [2009--2011]
	By using high-speed camera with our oneshot technique, 3D shape of objects of fast motion can be captured.
	One Shot Active Stereo Scanning Technique [2008]
	3D shape can be measured by simply capturing a scene by a single camera, where a fixed grid pattern is projected onto the target objects.
	3D shape acquisition system method based on uncalibrated stereo (3DFax) [2005--2006]
	3D acquisition system has being developed, which comprises one camera and projector.
	Active 3D shape acquisition entire shape acquisition system by using hand-heeld active 3D scaner (3DWand) [2003--2004]
	Representing the real world object in a PC is available by the proposed 3D acquisition system.

Medical imaging and sports analysis [Project page]
	Improved accuracy of PPG estimation using a camera by 3D tracking of the head [2020]
	To realize non-contact heart rate measurement that is robust to movement, we perform 3D tracking of the head using 3D information.
	Grid-pattern feature detection technique using FCN for 3D endoscope [2017--2018]
	Feature extraction for grid-based oneshot 3D scan is improved by a learning-based approach, such as CNN (U-Net).
	Grid-ICP: registration for multiple-line-based scan for endoscope [2016]
	Line scanning results are difficult to apply shape registration because of sparse shape reconstruction. We developed a new ICP method specialized for such line scan.
	Development of 3D endoscope system besed on active stereo [2014--2019]
	Development of 3D endoscope system using commercial products with our special optical-fiber based laser pattern projector.
	Non-Contact Biosignal Measurement by Active Stereo [2013--2019]
	By using our grid-based oneshot 3D scanning system, non-contact heartbeat / respiration measurements can be achieved.
	Active 3D shape acquisition of moving human body [2010--2011]
	By using multiple cameras and projectors, entire shape of moving human body can be captured densely and accurately without holes.

Texture analysis and synthesis
	Underwater image quality enhancement methods using NAS and Diffusion Model [2022-2023]
	We introduce the neural architecture search (NAS) to find an optimal and lightweight network for underwater image enhancement. The selectable transformer module can expand the search space and extract robust deep features. We further propose a Transformer-based architecture as the denoising network in the diffusion model. Besides, the non-uniform sampling strategy for the inference stage is proposed.
	Near-light photometric stereo using diffuser [2021-2022]
	We propose near light photometric stereo with a small number of measurements by optimizing patterns of a projecter projecting them onto a diffuser, utilizing them as distributed light sources.
	A Method for Adding Motion-Blur on Arbitrary Objects by using Auto-Segmentation and Color Compensation Techniques [2021]
	We propose a panning shot video synthesized by using a low-quality video frame without blur and a high-quality image with blur.
	High-frequency shape recovery from shading by CNN and domain adaptation [2021]
	Improving the accuracy of one-shot reconstruction with structured light by estimating high-frequency shapes from a single shading image using deep learning.
	Dense Pixel-wise Micro-motion Estimation of Object Surface by using Low Dimensional Embedding of Laser Spackle Pattern [2020]
	The speckle pattern generated when a laser beam is irradiated on a rough surface is used to estimate and visualize the micro-vibration.
	Independent character division projection method on multiple planes [2019]
	Different characters are independently projected to different depths simultaneously using multiple projectors. Those characters are unreadable other than predefined depth.
	Refraction-free image synthesis using light field camera [2019]
	Synthesize a refraction-free image from an underwater image captured by light field camera.
	Simultaneous multi-depth multi-image projection mapping [2016--2018]
	Different images are independently projected to different depths simultaneously using multiple projectors.
	Super-resolution method for blurred image captured by handheld camera [2010--2011]
	Image taken with handheld camera occurs blur and includes moving 3D objects. We are developing a Super-resolution method for these condition.
	Research on outdoor application of separation of reflection and penetration layer by space analysis of time [2007]
	The separation is applied to outdoor by using the speed correction etc. that consider the difference of the depth using the separation and the block match here.
	Reflection and Transparency Separation Method using EPI analysis [2006--2007]
	All pixels on the epipolar line that belong to the image behind glass will produce the same type of track in the EPI image. Any pixels that are due to reflection will produce a track at a different angle, depending upon the apparent distance to the reflected object. By looking at the color of inconsistent tracks, the reflection can be removed.
	Analysis and modeling of surface reflectance using active lighting [2006--2007]
	Reflectance parameters (albedo & specular) of real object are estimated from captured images and geometric model captured by projector and camera system.

Efficient utilization of real world data and robots
	Gesture generation method based on the semantic information of text using VAE [2023]
	Using TED data, we propose a method to generate gestures from text by contrastive learning considering the Attention of each word.
	Deep Gesture Generation for Social Robots Using Type-Specific Libraries [2022]
	To move avatars and robots autonomously, we collect TED Gesture-Type Dataset and propose a method for generating gestures from text.
	Calibration Method between Markerless Motion Capture and Optical Motion Capture [2021]
	We propose a calibration method between marker-less motion capture and optical motion capture utilizing the ICP algorithm.
	PoseRN: A 2D pose refinement network for bias-free multi-view 3D human pose estimation [2020--2021]
	We propose a new 2D pose refinement network that learns to predict the human bias in the estimated 2D pose using OpenPose.
	Representing a Partially Observed Non-Rigid 3D Human Using Eigen-Texture and Eigen-Deformation [2018--2019]
	Shape deformations of loose clothes caused by human action are efficiently represented by neural network based regression for sparse bases and their coefficients.
	Landmark navigation System [2015--2016]
	The main scenario of this navigation assumes the user to be a bicycler or pedestrian who do not have any smart devices. The system provides several landmarks, e.g., high tower, tall buildings, etc., to follow, leading to reach the final destination.
	Marker-less facial motion capture system [2013]
	The proposed method consist of three part: (1) Facial parts recognition, (2) Non-rigid deformation tracking and (3) Key point refinement.
	3D Animation Creation Support System [2013]
	This research provides system that an individual or small-scale organization can create 3D animation contents even without professional equipment and skill and control 3D animation with gesture operation on tablet device.
	A Route Navigation System based on Web Information and Visibility of Landscape [2007]
	We propose an efficient tourist route search system that not only recommends a path that simply connects several tourist spots, but also recommends the path with the most beautiful scenery. To achieve an automatic tourist spot recommendation dependent on each user's preferences, we propose a personalized tourist spot recommendation technique using web information.
	System for presentation of meta information on a 3D object according to the viewpoint [2007--2008]
	This research is about a system which shows the META information and the hyperlink added to a 3D object on a web page. The system is controlled interactively with a mouse and keyboard. This is a joint research project with Kyoto Sangyo University.