Mark Matamoros ITP Blog

ITP Courses

Fall 2019

Spring 2020

Fall 2020

The Neural Aesthetic

Midterm Project. November 3, 2020.


My midterm project is an excercise in utilizing PoseNet as a controller for sound parameters within a Processing program. Specifically, a waveshaped signal is generated and thereafter manipulated by a variety of effects. It must be noted that all sound-related components are provided by the Minim library.

Regarding the control parameters, the following PoseNet-derived body part coordinates were utilized for sound manipulation:

Unfortunately, occasional sound "hiccups" pertaining to the derived PoseNet coordinates creates a small amount of random parameter control. Additionally, my computer appeared to have a difficult time handling my Processing program in conjunction with the Runway application. Due to these issues, I decided to implement two delay effects within the signal processing chain to "smooth" the final output. Furthermore, these delays provided a personal sonically gratifying effect.