Bringing humans and robots closer through open source

I have been using open-source software for over 6 years now, ranging from an OS to programming language and specific applications. So recently, I have been yearning for opportunities to repay this huge debt. Last year in my junior year at college, I was working on research problems at the intersection of robotics and computer vision. This was the perfect time for me to explore open-source as it provided me a perfect platform to hone my domain skills and get acquainted with a large community with similar interests.

Robocomp

Robocomp is an open-source Robotics framework that provides the tools to create and modify software components that communicate through public interfaces. The organization works on several robotics-related tasks at the intersection of computer vision, natural language processing, and speech processing. I got the opportunity to work on the problem of ‘People Identification Component for the EBO educational bot’.

Face Identification System

The project aimed to build a facial identification system that can enable enriched interaction between robots and humans. There were two major challenges to this problem:

• It was a few shot learning problem where the training data for new samples was very less.
• The model should not be trained every time a new user was added to the database.

After reading numerous research papers and blogs, I came across the Triplet Loss function that helped me solve both of my problems. Triplet Loss function tries to optimize the learned embeddings by maximizing the facial similarity score in the compact Euclidean space. Next, to generate reliable and robust facial embeddings, I used features from FaceNet followed by a KNN classifier. For more details on implementation and experiments, please visit refer to this link. Further, I used this component to login to my system directly using a webcam instead of entering the password.

Mentor at GSoC 2020

My journey with Robocomp didn’t end there. This year, I volunteered to co-mentor two different projects at Robocomp:

• Hand gesture recognition for sign language recognition
• Human recognition (identification) using multi-modal perception system

Wrapping up

In the end, I would like to conclude that over the last 2 years, GSoC and Robocomp have given me a wonderful opportunity to contribute to open source and hone my skills in the Computer Vision domain. It has taught me the significance of writing proper documentation and effective communication. And also earned me the badge of Arctic Code Vault Contributor.