WORKSHOP Kinoteka, 14th April, 10AM-6PM

Machine Learning for Artists, Musicians, Gamers and Makers

by Phoenix Perry and Dr. Rebecca Fiebrink

Are you interested in creating live interactions with sensors, cameras, game controllers, or microphones? Machine learning can be a fast and empowering tool to connect these inputs to outputs such as animation, sounds, robots, game engines, or even custom-built physical computing systems. While it might sound like you need a degree in computer science to try out this technique, it is actually much easier than you’d think.

For example, imagine you are building an interaction using an Arduino with some sensors to control sound when a dancer moves around a space. Instead of writing a computer program to specify which sound to play based on the sensor values, you can simply use machine learning to record the data the sensors output when the dancer moves, and provide a set of example sounds to trigger when the dancer makes those gestures. A machine learning tool can then learn how you want the sound to change based on the movement over time.

In this way, machine learning makes it possible to quickly build expressive interactions that aren’t possible to create using only programming. Also, it makes it easy for non-programmers to design and customize interactive environments.

In this workshop, we’ll give you a hands-on introduction to using machine learning for creating interactive art, music, games, and other real-time systems. Using the Wekinator (, created by the course’s instructor Rebecca Fiebrink, we will cover an introduction to using machine learning for creative practice. This free and cross-platform software tool connects a wide variety of existing hardware and software (e.g., Arduino, Unity 3D, Max/MSP, PD, Ableton, openFrameworks, Processing, Kinect, Bitalino, …) using OpenSoundControl (OSC). Expect to learn the basics of a few standard techniques to help you get started integrating machine learning into your own projects.


Participants should bring a Windows, Mac, or Linux laptop with the latest version of Wekinator and its example pack installed (all free, from We also encourage participants to bring any hardware they want to work with, including sensors (e.g., Kinect, microbit, gaming controllers, biosensors, DIY sensing systems using Arduino…), and/or hardware they want to control (e.g., microcontroller-based robots).

Participants don’t need any background in machine learning or math. All the tools we’ll use have graphical user interfaces, so it’s not required for participants to have programming experience. That said, machine learning is most useful when you can connect it to something that you’re interested in, and programming helps a lot with this.