hello_world: a new album by Adam Rokhsar free on Facebook — see the link below  Click for high-res photo

hello_world: a new album by Adam Rokhsar

free on Facebook — see the link below 

Ruler of the Universe
2011

materials:

Microsoft Kinect
Custom Software in MaxMSP

thanks to Jean-Marc Pelletier for jit.freenect.grab

The Sibling Dance
2011 

materials:

1 iMac
1 Microsoft Kinect
2 Siblings
Custom software built in MaxMSP

This video was made using the same system as my previous video titled “Body Language” (see below).  Once again, an artificial intelligence algorithm called a self-organizing map continuously attempts to learn common poses according to the position of our bodies relative to the Kinect.  When it finds a pose, it plays a note and colors the animation.  

Body Language
2011

The Microsoft Kinect for the Xbox 360, like a normal camera, gathers information about the color of everything it sees.  

What makes it unlike a camera is that is also can calculate the distance between it and everything it sees.

Using Jean-Marc Pelletier’s freenect extension to the programming language MaxMSP, I created this piece to explore the possibility of controlling sound with gesture.  The video is an animation in a virtual 3D space of my body.  I used an artificial intelligence algorithm called a self-organizing map to watch my movement and identify seven distinct poses.  Each of these seven poses are trigger a different note from the C major scale and a color with which to draw myself.

 The self-organizing map identifies gestures without any instruction — in the world of AI and machine learning, this is called an unsupervised learning task.  Because of this, I must move to uncover different notes and colors.

For those interested in learning about self-organizing maps (SOMs), there is an article available here by Dr. Teuvo Kohonen, the inventor of the algorithm: http://www.scholarpedia.org/article/Kohonen_network

For those interested in the technical details of how I set up the SOM, I have an iPad sending OSC messages which allow me to adjust the learning rate and neighborhood radius of the SOM in realtime.  At around 40 seconds into the video, I pull down the learning rate considerably and it is possible to observe the SOM switching very clearly between different gestures.  

The Word of God (II)
an audio/video installation, 2010

You are listening to a machine trying to understand The Bible. The red network of lines is a live video of the machine’s brain, built using a custom artificial intelligence algorithm.  Each line is like a neuron that fires whenever it sees a particular word.  As time goes by, the brain gets better at reading.  It adapts to recognize words, starting with the words it sees most frequently.

The text on the right is the original text, the text below is what the activated neuron thinks it’s reading.   The piece is run as a live installation; it takes approximately two days for the machine to read The Bible.

This video is a short excerpt of that process, taken as the machine is beginning to understand its first words.

3 Neural Networks

a multichannel video installation, 2010

The three pieces below comprise a multichannel video installation created using a kind of artificial intelligence called a self-organizing map.

As the machine watches each of the three videos, it tries to understand what it sees by building a series of memories.  At first, the machine’s memories are nothing but random noise.  As time goes by, the noise slowly changes to become more like a different aspect of the scenes played out before it.  Much like in the human brain, experience presses itself in distinctive ways on the artificial brain, changing not only what it remembers, but what it perceives.  In a self-organizing network, memories tug on each other, so that everything the machine sees effects everything it remembers.

Obama
2010 

Manhattan Bridge
2010 

The Mirror
2010

Jennifer I’m Filming You
2010 

This video was created using an artificial neural network called a Kohonen Map, which is meant to mimic in software a simplified version of what happens naturally in the human brain.

A video is fed into the neural network, which can be seen on the left side of the screen. Each line can be thought of as an unrolled video frame, in which each row of pixels is laid out end to end.  There are eight rows of these pixels, which start off as random grayscale values.  As the network watches the video, the row that is the closest match to the current frame slightly adjusts to match it better.  It is then output by the network as the video on the right side of the screen.  Over time, the network gets better at understanding what it’s seeing, but it must constantly adjust its memory to continue to construct the world.

This is called machine learning, and in some ways can be likened to the biological phenomenon of neural plasticity.