My interest in Virtual/Augmented Reality technology began when I started working as a research assistant on a project called “ARWED” in ISU. The goal of the project was to see how the brain reacts by observing an action (in an immersing VR environment) instead of performing it. This project opened the door to the VR/AR world for me, a new technology where virtual environment and objects can expand the human perception of their world.
As a student working in the Robotics LAB, I became very interested in investigating VR/AR applications in Robotics industry and have made a VR application where the Robot Kinematics Synthesis can be done in.
Following my interest, I inquired and received an internship at House of Design, so that we could investigate the applications of how AR technology could be used with Robotics.
The leadership at HOD consistently works on keeping current with new technologies, so that they can utilize them for their robotic systems in order to enhance their efficiency and quality. In the first phase, we decided to make a mobile app that utilizes some Augmented Reality features. AR technology can give the support team of a company the ability of remotely assisting the client. Also, the maintenance team can get all the information of a system by detecting and tracking 2d images attached to the system. We can pull up the 3D model, data sheets and all the information of the industrial parts plus possible tutorials or disassembly instructions. After reviewing many SDKs available for AR, I found one suitable for our needs. One of the helpful features of the SDK is using SLAM (Simultaneous Localization And Mapping) which extracts feature points of the real world with only one RGB camera to do spatial mapping of the environment. As a result, we do not need any special hardware or cameras to do the 3D mapping. Moreover, the ability of this kit to detect 2D images pretty fast and precisely provides a good platform to identify a system and get its information. This SDK was implemented to the mobile application that I am making for HOD and I am currently in the process of adding several additional AR features to it.
We are looking to come up with new ideas such as exploiting AR to ease the debugging process for robot programmers. The purpose of this feature is to display the running code blocks and their outputs right in front of the system so that the programmer can see the results of the changes in their code faster.
My future and ultimate goal is to replace current tools that are being used to interact with ABB robots with an AR alternative. This will ease the design process, robot training and debugging for robot programmers. We hope we can do this project through a collaboration between HOD and ISU in the near future.