Mark Matamoros ITP Blog

ITP Courses

Fall 2019

Spring 2020

Fall 2020

The Neural Aesthetic

Midterm Project. November 3, 2020.

POSENET FOR SOUND CONTROL

My midterm project is an excercise in utilizing PoseNet as a controller for sound parameters within a Processing program. Specifically, a waveshaped signal is generated and thereafter manipulated by a variety of effects. It must be noted that all sound-related components are provided by the Minim library.

Regarding the control parameters, the following PoseNet-derived body part coordinates were utilized for sound manipulation:

Unfortunately, occasional sound "hiccups" pertaining to the derived PoseNet coordinates creates a small amount of random parameter control. Additionally, my computer appeared to have a difficult time handling my Processing program in conjunction with the Runway application. Due to these issues, I decided to implement two delay effects within the signal processing chain to "smooth" the final output. Furthermore, these delays provided a personal sonically gratifying effect.