obfuscurity.
hidden in plain sight

I recently found myself in the need for a way to run a one-off Ruby script at scheduled intervals. As this is a work project I didn't just want to run it on my laptop or some random server. Turns out there's an easy way to run this for free on the Heroku Cedar platform without having to piggyback it on a "real" application. Because there are no web processes running, we'll be able to limit our dyno usage to a single dyno (in other words, it's free).

The script itself handles garbage collection duties for removing expired hosts off our beta account with Boundary. Basically I just want it to run every hour and cull anything that hasn't reported to their collectors in a day. For the purpose of this article the contents of the script are inconsequential, although I intend to present it fully in a future post.

To run your own scheduled Ruby code, you'll first need to create Gemfile and Gemfile.lock files. These Bundler files are used by the Cedar platform to determine that you need the Ruby buildpack.

$ cd oscar

$ ls -l
total 16
-rw-r--r--  1 jdixon  jdixon   47 Feb 11 09:25 Gemfile
-rw-r--r--  1 jdixon  jdixon  189 Feb 11 09:25 Gemfile.lock

bin:
total 8
-rw-r--r--  1 jdixon  jdixon  954 Feb 11 09:25 purge_meters

You'll want to initialize your git repo (if you haven't already) and commit. Create a new Heroku app and add push it to the Heroku git server.

$ git init .
Initialized empty Git repository in /Projects/oscar/.git/

$ git add .

$ git ci -m "initial commit"
[master (root-commit) 41ec621] initial commit
 3 files changed, 56 insertions(+), 0 deletions(-)
 create mode 100644 Gemfile
 create mode 100644 Gemfile.lock
 create mode 100644 bin/purge_meters

$ heroku create -s cedar
Creating blooming-robot-7955... done, stack is cedar
http://blooming-robot-7955.herokuapp.com/ | [email protected]:blooming-robot-7955.git
Git remote heroku added

$ git push heroku master
Counting objects: 6, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (4/4), done.
Writing objects: 100% (6/6), 1002 bytes, done.
Total 6 (delta 0), reused 0 (delta 0)

-----> Heroku receiving push
-----> Ruby app detected
-----> Installing dependencies using Bundler version 1.1.rc.7
       Running: bundle install --without development:test --path vendor/bundle 
         --binstubs bin/ --deployment
       Fetching gem metadata from http://rubygems.org/..
       Installing json (1.6.4) with native extensions
       Installing mime-types (1.17.2)
       Installing rest-client (1.6.7)
       Using bundler (1.1.rc.7)
       Your bundle is complete! It was installed into ./vendor/bundle
       Cleaning up the bundler cache.
-----> Discovering process types
       Procfile declares types -> (none)
       Default types for Ruby  -> console, rake
-----> Compiled slug size is 1.5MB
-----> Launching... done, v4
       http://blooming-robot-7955.herokuapp.com deployed to Heroku

To [email protected]:blooming-robot-7955.git
 * [new branch]      master -> master

Next, add the Scheduler addon to your app.

$ heroku addons:add scheduler:standard
-----> Adding scheduler:standard to blooming-robot-7955... done, v5 (free)
       This add-on consumes dyno hours, which could impact your monthly bill.
       To learn more:
       http://devcenter.heroku.com/addons_with_dyno_hour_usage
       To manage scheduled jobs run:
       heroku addons:open scheduler

Now you'll need to specify the task you want to run (your Ruby script). Run the following command to open the Scheduler dashboard in your default web browser.

$ heroku addons:open scheduler
Opening scheduler:standard for blooming-robot-7955...

Click the Add Job... link to edit your task command. For my task I wanted to run the script with bundle exec ruby. Choose your Frequency, optionally select the Next Run time, and then Save. Once this is saved you can also click the Run button to run the job manually and look for any errors in the Task Output.

Voila, you have your own free cron job running through the wonders of THE CLOUD. Now go off and conquer the world!

This evening I asked for your suggestions on blocking online distractions, allowing me to focus on code for an extended period of time. I have a constant struggle with interruptions (read: shiny things) including online news, email and Twitter. There was a flood of responses in no time. Here are the more popular suggestions, along with my winners below.

• quit the offending apps
• block websites
• login with a different user
• work offline
• coffee
• music
• self-control (lol rite)

I suspect that all of these would have some benefit, perhaps except coffee, which has never given me much of a boost anyways. As a side note, I've quit Diet Coke since my surgery and am exclusively drinking water. So it's possible that coffee might give me an insanely productive hit, but I'm not willing to tread that path yet. Here are the specific steps I took which seemed to work quite well for my particular workflow.

1. quit Twitter app
2. quit Chrome (primary browser)
3. quit Firefox (used for HTML email, Facebook and banking)
4. quit Propane (Campfire app, work communications)
5. quit Adium
6. quit Skype
7. detached my remote screen session (mutt and irssi)
8. equipped Sony MDR-V6 headphones
9. launched Spotify radio (trance)
10. put iTerm2 in full-screen mode (used for psql, git and debugging development server)
11. put MacVim in full-screen mode
12. launched Safari for API docs and development site

I'm pleased with the recent experiment, even if it only lasted one hour. I'll continue to make adjustments and report any significant improvements I find.

I find myself using Sequel in conjunction with Sinatra these days to write more of my web applications. Never having been a fan of ORMs in general, and mostly comfortable with the ickier bits of SQL wizardry, it took me a while to warm up to the idea of using one for database migrations. But I've seen the possibilities with stuff like ActiveRecord. Being able to migrate my schema into a versioned state is "dee-lish".

Read the rest of this story...

One of my first projects at Heroku has been to modernize our shared PostgreSQL offering (working with @asenchi). As we get closer to internal testing of our new service, @markimbriaco asked for benchmarks looking for any bottlenecks in PostgreSQL 9.x when creating large quantities of small databases. We've seen instances where Pg 8.3 will start to choke after 2000 databases on the same server and we're hoping that 9.x alleviates this issue.

My initial test was overly simplistic but still revealed some interesting patterns. I started with createdb on the command-line, generating 8000 roles and empty databases, serially. The results were promising, with PostgreSQL 9.0.4 (Ubuntu 10.04) able to scale up without any noticeably increasing latency. Unfortunately, it's not a terribly useful benchmark given the absence of any workload. And yet, I couldn't help but notice a pattern in the scatter plot:

Notice the gap between 500 and 600 ms? I don't have an explanation for this but I suspected that Pg has an internal condition that triggers for actions that take 500ms or longer. Regardless, our primary expectations had been met. Whatever bottleneck 8.3 demonstrated when creating databases on a server with large quantities of existing small databases appears to be fixed in 9.0.

The next test was to run a similar sequence with our new application server. It offers an internal RESTful API using Sinatra and Sequel to provision and manage customer databases on shared servers. The results for this run were even more enlightening. Check out the stratification:

Not only is the initial gap (around 400ms) even more pronounced, but you can see a pattern of latency introduced at 200ms intervals after the initial 400ms delay. I have no explanation for this, but I wanted to publish these results and see if anyone else has a guess as to what might be causing these patterns.

UPDATE: To rule out any distortion caused by GNU time, I ran another test using Ruby's Time class to get a more accurate representation. In the most simple terms, we start the clock with Time.now, connect to the database (no caching), create a role, create the database and stop the clock. Output is logged and then imported into Excel for plotting. I think the results speak for themselves (measured in milliseconds):

I'm proud to announce that I'll be starting at Heroku in a couple of weeks. This is an exciting opportunity to work at a place that breathes DevOps and eats Infrastructure as Code. Whenever you hear someone describing "Platform as a Service", there's a good chance that Heroku will be the example they're talking about.

I first met Mark Imbriaco (@markimbriaco) when he was the Operations Manager at 37signals. Mark's a level-headed guy with a undeniable talent for Web Operations and an excellent track record for supporting his customers. It was no surprise to me when he took over as the Director of Cloud Operations at Heroku. Even after the acquisition by Salesforce.com last December, they've continued to innovate at a breakneck speed (proof here, here, here and here).

Heroku development and operations teams get to work on the sort of rapid scaling and engineering challenges that pique my interest. I'm doubly excited to be able to share the fact that I'll be joining up simultaneously with Curt Micol (@asenchi) as the newest Operations Engineers on Mark's team. It's an odd coincidence that we're both big fans of BSD. Hopefully nobody holds that against us. ;-).

Needless to say I'm thrilled about the whole thing and hope that it gives me more cool stuff to write about here. Stay tuned.