This is an application that combines emotion recognition with Spotify music playback to give the user music that fits the mood. The user’s PC camera provides a live video feed from which their face is detected and isolated. Then the program recognizes the emotion currently displayed on the user’s face and plays the appropriate public Spotify playlist. It accomplishes this through computer vision and machine learning tools such as OpenCV and Keras.
OpenCV is a computer vision library which I used to capture and detect the user’s face in real time. The data collected from the user’s face is then processed by Keras, a deep learning framework. Keras compares the processed data to a model of possible human emotions from which the emotion most likely displayed is chosen. This emotion is then displayed on top of the live camera feed along with a box encompassing the user’s face. Finally, when the user presses the “play music” button in the GUI, Spotify will process the request and return the appropriate playlist.
This is the final project I worked on, one that I conceptualized since the beginning of Summer 2021. Although it's been complete for a few weeks, it took me until now (Sept. 13th) to get everything up. This was because I’d been working on projects like this one all Summer and (at the time) with 2 weeks left before school, I just wanted to relax and enjoy the rest of my break. Nevertheless, it's here!
I left this project at the bottom of my to-do-list since I didn’t think I’d be capable of bringing it to life. Working on my other projects during the Summer showed me just how many great (and free) resources are available on the internet. It also showed me that as long as I’m determined I could bring my ideas to reality, I just had to try. So, just like with my Spotiplayer project, I searched the web for ways to have a machine recognize human emotion. And the web brought me to computer vision and machine learning.
Prior to this project, I had very little idea what either of those terms meant. I did know that I couldn’t just throw myself into building my program without having some basic knowledge on these topics, so I started with research. I spent the first week just learning about each topic and making sure I understood the common terms and their definitions. I also found frameworks, libraries, and classes associated with them. I learned about OpenCV and TensorFlow, the two most common AI tools when it came to image recognition.
Finally, it was time to put the project together (I’ll skip the details). The final product isn’t always consistent with recognizing emotion, however, I’m still very pleased with it. Just like with my other projects, this project has room for improvement and I will definitely try to improve it in the future (particularly the accuracy in emotion recognition).
Visit my GitHub repo for the project’s code and more information.