Commissioned Project

🙌 Posture Detection using PoseNet in Real-time - Interactive LED Image wall

Interactive LED wall with Processing and Posenet

🙌 Posture Detection using PoseNet in Real-time - Interactive LED Image wall

❓About

This is an interactive installation to turn audience poses with real-time pose detection techniques into the control of images and hashtags on the LED image wall. With the deep learning TensorFlow model PoseNet and RealSense depth camera, we have developed a program in Processing to perform real-time gesture control.

‍

It was co-developed by me and Chan Long Fung Lazarus as a creative technologist and a new media artist. It was a commissioned project in ARCH “Arts X Tech” Exhibition invited by the 13th Arts Ambassadors-in-School Scheme organised by the Hong Kong Arts Development Council (HKADC).

‍

This project was an extension of an AR artwork Life as usual by Natalie Lo Lai-lai. The screen recordings of the Instagram AR filter uploaded by the users are stored in our database, and they are converted to GIF animation to our Processing code as GIF assets remotely every day.

🔨Technologies

Processing (JAVA)
PoseNet
Deep Learning

🔗Link(s) to the project

GitHub

🌟Features

Posture Detection using PoseNet

The audiences' pose positions are mapped to the coordinates. The GIFs and Hashtag images position is controlled by the attractor class so that they are slowly attracted to the pose coordinates.

‍

What is PoseNet?

PoseNet is a deep learning TensorFlow model that allows you to estimate human pose by detecting body parts such as elbows, hips, wrists, knees, ankles, and form a skeleton structure of your pose by joining these points.

Posenet keypoints displaying human skeleton structure

Image from Posture Detection using PoseNet with Real-time Deep Learning project

👏Acknowledgements

📄License

CC BY-NC-ND