This week my major work was around building an in-browser audio recording system. I’ve been reading about using new HTML5 APIs to get access to the user’s mic (and camera) for a while and I was finally able to sit down and start building something out. Because I feel the need to name everything I build, I had to name this as well. None of the bands I was in ever really practiced or recorded in a garage, they mostly worked out of basements. As a bit of a nod to (and departure from) Garageband, I decided to just call my recording stuff Basement.
The core technology for this whole process is getUserMedia(). This is what prompts the user to allow the page to access the mic (or camera or whatever) and then handle a LocalMediaStream object. From there you can work with whatever is being picked up by the mic.
Outputing the audio in real time was very easy (hey I built a monitor!) and after doing a bit of research I found a fantastic library to help with the recording aspect of this project: Recorderjs. It provides a nice Recorder object to work with and, best of all, it’s a VERY small, easy to understand library so I can go through every line of code and understand exactly what is being done. This is very important because all of this in-browser media work is very new and deep tutorials are rare. Plus I’m going to need to do a few things that haven’t been done perfectly yet.
Right now I’m working on taking the recorded audio, which exists as a Blob tied to the page it was created on, and sending it back to the server to be uploaded in a way that works nicely with our existing Paperclip system. I’m really digging into some of the new HTML5 APIs for all this work and it’s been really interesting.
The in-browser recording is the last new major feature we need to build before polishing up the RecLaboratory site (which I updated this week, too!) for a very early, very small beta. I’m really excited to get people using this tool soon!
Along with some other RecLab stuff (which is coming along nicely), this week I finally tackled a little project that I’ve been thinking about for a while. Unlike most of the RecLab work this was much more artistic than technical. I built the first first hand drawn icons for the site.
I knew that I wanted custom instrument, track, and song icons so I started out with a small set of 8 (with plans to make more as we need them). I did these icons the same way I do my comics: first drawing in pencil, then inking, then scanning into Photoshop, then cleaning up the lines and adding color in Illustrator. They had to look good at a wide range of sizes: 400px to 30px so I didn’t add too much detail.
To handle what I think will be the most commonly used instruments I started with (electric) guitar, (electric) bass, drums, keys, vocals, sax (perhaps my own bias), percussion, and general track.
So here they are!
Guitars are still hard to draw.
I used a small pallet of colors for all the icons in the hopes of creating an overall cohesive effect when they all get seen together on the site. Bobby doesn’t like the percussion one very much and he’s probably right. It’s my least favorite too, but this seems like a good start and really adds some nice visual variety to the site. Plus it’s always fun to draw things!
So as of last week I have the RecLab Rails site up and running on Heroku and the audio processing API, Roctopus, up and running on my own server. Amazingly they are able to talk to each other and work pretty much as expected.
There are still quite a few things to fix up, particularly with Rocktopus, but it’s nice to see the full application working together on remote servers. While the Rails/Heroku setup was pretty straight forward, Rocktopus was uncharted territory for me.
I had never “launched” a Flask application before so this was all a bit new. As a (temporary, I swear!) measure I moved the files to my server over SFTP and started up a Gunicorn server to run the application. I still need to put Nginx in front of all of this but for now it works as expected (though I can’t imagine it would handle much traffic). I’d also like to learn more about Fabric for Python deployments. I’ve never been much of a sysadmin, so this was all a bit new to me.
The only really new thing I did for the Rails/Heroku app was add Unicorn as a server. Heroku has a great article that held my hand through this setup.
Also, as part of this setup I’ve been playing around with Slack and it’s wonderful integrations for monitoring the app. So far I have it working with Heroku, PaperTrail, and Pingdom. Nothing’s gone wrong yet, but I suppose that is to be expected when there are no users…
As I mentioned, there are still quite a few rough edges to be worked out, a bunch of minor features, and at least one major feature that still needs to be built but this feels like a big step in getting RecLaboratory launched and letting people try it out. We have some people in mind for VERY early alpha testers, but please let me know if you’re interested in trying it out when it’s ready!
So, you may have noticed that I’ve missed the past few Thing a Week weeks. In my defense I did just get married last Saturday, so I’ve been pretty busy with wedding related travel and planning so I haven’t had much time for extra projects. The wedding was amazing and I now have a wife, which is fun. In the spirit of the wedding I wanted to share a project that I worked on a while ago but didn’t want to show until now.
For the wedding I wanted to build a ring bearer box out of an old book. After looking around some used bookstores I picked an old Tarzan book because 1) it seemed common enough that I didn’t feel bad using it as materials and 2) it had a cool elephant on the cover. I wanted to hollow out part the inside pages and create inlays to hold the rings (like a secret flask holder but for wedding stuff).
I started by covering the covers in plastic (to avoid getting glue on them) painting the sides of the pages with puzzle glue in order to stick all the pages together. I did about 5 coats of this. Once that glue was dried I measured out the inlay and started carving pieces out using an Excato knife. This took a lot longer than expected and when I was down about half an inch, I coded the new inner wall with glue to give it a nice solid feel.
After I had my basic inlay I needed to dig out the slots for the actual rings. This was even more tricky. I thought I would be able to make clean cuts, like I did with the inlay but I ended up basically using the knife to turn the paper into pulp and then just digging the pulp out of the the slots. After removing top page from the inlay, which had been pretty well torn up by all the digging I was left with a nice ring box.
The pages held the rings perfectly and I could even hold the box open and upside down without the rings falling out. The box was put to use by our ring bearer, Sui Chi, on Saturday and it looked great! This was a cool little project and it was fun to build something non-digital again. It really took a lot longer than I expected but the result was worth it.
So most of this week I was traveling so I wasn’t able to get very much work done on any projects. I did, however, attend an Indian wedding in Long Island which was really cool so the week was not a total loss.
The work I did get done earlier this week was mainly with the RecLaboratory interface to Rocktopus, our audio processing API. I built the interface to the bounce_song functionality, which will take all the individual tracks of a song and combine them into a single (probably .mp3) file.
I also set up a sort of hacky but effective, for now, “loading screen” to let users know that their audio is being uploaded/processes. It seems like a little thing but letting the page just hangs after clicking submit is really bad UI. This is a very early version of the product and we haven’t given UI the full focus it needs but this was one thing that would be too awkward to not address early. It was a bit of an awkward fix because we’re not actually doing the audio processing or uploading as a background process. Eventually I’ll build a queuing/callback system in Rocktopus that will work much nicer, but this gets us going now.
This next week is another busy week. It is the last week in SF before we fly back to Chicago for the wedding (which is less than 2 weeks away now…). If I get anything Thing A Week thing done this week it will probably be something wedding related.
Once again, another busy week divided between wedding stuff and side projects.
I spent this week adding more features to the RecLaboratory audio player and I feel like it’s in a pretty good spot for now. Obviously, there’s more I’d like it to do but it’s certainly at a point where it will work for basic track and song playback.
Here’s a list of new features that got added this week:
Visual indicators on the in page play buttons to show what is playing in the global player
Some basic layout polish and player styling
Jump forward or back in the audio by clicking on the progress bar
Play/pause using the space bar
Some basic (if clunky) error checking to make sure files loaded
I’m really happy that is “just works” that way, I was worried that I would have to go back and turn this whole project into a single page app eventually to get uninterrupted playback, so this is a nice win. I do need to spend some time testing things to make sure I understand WHY it works. It seems dangerous to accidentally get a nice feature like this.
Like I said, while Sauropod is still a little rough around the edges, it’s good enough to use for the site. I’m now moving on to fixing up the interface to Rocktopus for audio compression and song bouncing.
This week I’ll be traveling to New York for a friend’s wedding, so my Thing A Week time may be limited. Also I need to eventually finish writing the ceremony script for my wedding or else some people are going to be mad at me…
Another 7 days, another busy week. With the wedding growing ever closer I’ve been squeezing in time for other projects wherever I can. This week I managed to make some good progress on Sauropod by putting in some work before I go to work and on the plane while traveling to Chicago for the 4th.
I got Sauropod hooked up into the asset pipeline of the RecLab Rails site and started attaching it to the various audio controls on the site. While Howler.js handles loading the audio file and playing the sound in browser, I still needed to build all the hooks to control and monitor the audio.
At this point I am able to click a “play” button next to a track and have Sauropod load the song and update global play/pause button, display the name and owner of the track (with links), and display a progress bar and time progress. I also have a global mute/unmute working and started on global volume controls. Most of the controls and UI updates were just simple hooks into Howler and some jQuery for display manipulation, but the progress bar and timer were a little tricky. While a regular HTML5 audio player sends an onprogress event that you can listen for to control the progress display I had to write my own listener that starts an stops with the audio. It wasn’t very hard, I just used a setInterval() function around my own sauropod.updateProgress() function:
I still need to build all the hooks into playing a mutitrack song (though it should work based on how I’ve built the system) and then work on the track editing features (muting/level adjustments at a per-track level). I also need a bit more robust error handling and a cleaner UI, but all in all the audio player is coming along nicely.
Thing A Week Progress Update
So this week marks half a year of this “Thing A Week” project. It hasn’t been exactly what I planned, one discrete project finished every week, but it has really keep me focused on building things and I’ve really enjoyed writing about my progress on all the things I’ve been working on.
As seems to be the trend lately, last week was very busy. Between work stuff, wedding stuff, seeing a musical, and going to a show it was hard to find time to get extra things done.
Almost all of the weekend was devoted to knocking off extra wedding things, like designing take-home gifts for guests and buying a bunch of ties. There is a big list of things that we apparently need to do, but we’re making good progress on it.
We had a really productive RecLab meeting on Tuesday where we worked out some good layout stuff and got the feed stuff into a good place. With the basic feed working, I am now able to move on to the audio player component: Sauropod.
Sauropod is still in it’s very early stages, but I have a pretty good architecture planned out and I’m excited to see it develop.
This was another busy week but I manged to get some solid work done on RecLaboratory stuff.
First I took some pictures to use as a cover photo on our Twitter and Facebook pages. This ended up being a big project that involved moving a bunch of my music equipment around and finding a clear area in my small apartment. I am not a great photographer but I’m pleased with how it looks. Feel free to follow/friend us on either. We’re starting to post things.
Last Wednesday night I just couldn’t sleep so I stayed up until 2:30 and knocked off a few bugs around comments but the major work I got done this week was around the event feed. I knew I couldn’t just rig something together with a bunch of ugly table JOINs I ended up using the public_activity gem. It listens for controller actions and makes an entry in a separate table for every event I want to show up in the feed. Most of our events are just the result of object creation and public_activity handles this really well right out of the box.
Right now I just have basic events around Users, Comments, Tracks, and Songs and I need to clean up how I’m using the public_activity params hash to avoid some n+1 database call issues but so far it’s made it really easy to build feeds on User, Track, and Song pages. I’m working on building the main homepage feed now, which is proving a little trickier. Obviously I’d like to do this with a single call to the Activities model but because it’s tracking polymorphic objects, the SQL is taking some work.
I’m hoping to have the feed stuff done by next week and start setting up a non-local version of the Rails site + Rocktopus soon. Then I have BUNCH of WebAudio stuff to work on around the player/audio tools. Oh, and I should probably be planning my wedding at some point too…
I actually drew a real comic this week, one where I penciled and inked and everything. Check it out!
This weekend Eileen was out of the house for her bachelorette party so I tried to knock out a few outstanding wedding related projects (apparently that is coming up pretty quickly). The Bird Watcher project had been dragging on for a while so I thought this would be a good time to wrap it up, too.
For those who don’t remember, Bird Watcher is a system that listens for photos with certain #hashtags and displays them to a live page. I plan to use it to let people take pictures during our wedding and show them on a projector during the reception.
This project has spanned a few ThingAWeek posts and involved playing with the Twitter and Instagram APIs, as well as learning a bit more about Postgresql and the Sequel gem.
At this point most of the hard work had been done. I have Twitter and Instagram listeners (which both work in very different ways) that store info about photo posts in a database. I just needed to write something to pull them out and display them.
Originally I had written some nice code for prioritizing new photos and minimizing database calls but that ended up being a scoping pain in the ass in Sinatra. I would have had to build some sort of session object persistence system and honestly, the scale of this project just wasn’t worth it.
I ended up with a simple AJAX loop on a timer that pulls a random “photo card” with the image and some information about it, and displays it on a page that should work nicely full screened on a projector. I still need to run some tests to make sure I don’t get any weird image sizes but it’s working well right now.
I’m a little worried that I’ll spend part of my own wedding reception tweaking code or sshing into my server to restart some service, but I think everything should run pretty smoothly. This was a fun little project and made me play with some new technologies. Hopefully lots of people use it at the wedding.