KinectやLeap motion, VRやARなどの技術を使ってやったことのメモとか

Diagnostic Support of Echography with Augmented Reality (AR) and Markerless Probe Tracking

 Echography(Ultrasonography) is a diagnostic imaging method to visualize a cross section of organ and bloodstream simply by holding an instrument called ultrasonic probe against a body. Thus, the method has been used for observing various organs due to its convenience. However, obtaining good echogram requires a training since it takes an image as grasping a positional location between a location/shape of organ and an image of cross section.

 In such a background, we have been conducting research & development for a system to visualize a location/angle of cross section at the time of image photography by a skilled doctor or a location/angle of cross section for a currently-operating probe by AR technology respectively, then also examined the usefulness as a support tool for image photography [1, 2].


 On the other hand, because a marker detection system by ARToolKit was adopted for tracking a location/angle of probe, it was necessary to install a marker to a probe but a lower operability problem still remained.

 Therefore, after applying a probe tracking with the development of object tracking algorithm by a totally-different image processing technique, it could verify a smooth movement more than we expected. In future, we are considering to improve the accuracy and stability by further modification of algorithm.

 Furthermore, since CG was superimposed on the image of positionally-fixed camera in the system respectively as previously described, we could not see the information from a viewpoint of probe operator. 

Thus, as a sequential approach for such situation, we could accomplish to observe a location of cross section from physician's viewpoint along with the correspondence of HoloLens. For this system, we are also considering to continuously enhance the functions as well.