converting a processing sketch with audio and visuals to a video on ubuntu
I really like generative art, so I have been playing with processing
for a while. Processing is an open source language (on top of java), that gives the possibilty to create images, animations, with added interactivity.
First I created a simulation of raindrops
, and because I wanted it to be easily configurable I used processing.js
Next-up I wanted to create a music visualisation. Interact with the music. I had the perfect piece of music in my head: "Endless Season" by Ken Ishii.
Now processing.js does not interact with music. There is HTML5 audio, but it still is very experimental, and I did not find any API for processing music, reacting to and analysing music played. Processing (the java version) has an excellent library for this: Minim (actually more than one, but I ended up using that one).
When the sketch was finished, I wanted to share it, convert it to a video. What were the options.
Use Processing itself
There are two ways to convert a processing sketch to a movie, from within processing itself:
MovieMaker: this requires quicktime and unfortunately does not work on Ubuntu
- when each frame is drawn, do
saveFrame and then convert all frames to a movie afterwards. While in theory this should work, saving the images slowed down my sketch, and ultimately screwed up the sync with the audio. My frame-rate was not consistent enough. For straightforward stuff this does not matter, but I needed it to sync with the audio.
- There is a third option:
So I needed an alternative approach. If it runs correctly on my screen, couldn't I just record it on my screen?
GSVideo, but frankly, that seemed to damn hard for me, so I skipped that.
Use some sort of screenrecording/screencast software
To record my desktop, on ubuntu, including the sound from processing proved to have some issues:
The first proved to be the hardest.
- processing (i.e. java/JDK 6) does not use ALSA to create the sound, but address the hardware devices directly
- I do not want to record my entire desktop, but a specific part, of a specific size
- I want to share my video on vimeo, so it has to follow certain guidelines
Recording the system audio out together with the video
On ubuntu, I found one approach to work very well for me:
- use PulseAudio mixer, it will allow to take the sound output as input to record
- and record away! :)
But, unfortunately, since java does not ALSA but uses the hardware devices directly, PulseAudio was unable to capture the sounds. However, by accident I found out that if you export your processing sketch to an applet, and run the applet in the browser, it does use ALSA and can be recorded perfectly. Awesome.
Part one solved.
Recording a specific part of the screen
gtk-recordmydesktop allows to specify an area of the screen to record, but somewhat akwardly. Now, for exporting to vimeo, it had to follow certain fixed, optimal sizes. E.g. 640×480 for 4:3 SD video, 640×360 for 16:9 SD video, and 1280×720 or 1920×1080 for HD. And that is hard to do if you are trying to position the recording box manually.
But, as I found out here, when using recordmydesktop from the commandline, you can send those options along:
recordmydesktop -x=0 -y=0 --width=320 --height=240
So, if you open the
Advanced->Misc, and look for the
Extra options field, there you can fill in the same options, and when you press record gtk-recordmydesktop will show the bounding box that is recorded.
Preparing your video for uploading to vimeo
To upload your video to vimeo, you have make sure two things are correct:
- the screensize, which we discussed before
- the video format
gtk-recordmydesktop creates an Ogg Theora file, and unfortunately vimeo does not accept that format yet. Converting that to MP4 was hell, until I found Arista Transcoder.
Using Arista to create an MP4 is easy (but you have to know it):
This will create a file called
- open Arista Transcode
- create a new conversion
- select your file, by default called
out.ogv, as the source
Sony Playstation - PSP as the device
- press create!
out.mp4 which is just perfect for vimeo, including the sound.