Getting New Relic and RDS to play nice

Mismatched plug and socket

Anyone who’s ever had to support server infrastructure of any kind knows the value of having a comprehensive, automated monitoring solution in place. With this in mind, we have begun to roll out the New Relic platform to monitor all our AWS based servers. New Relic comes with many great monitoring metrics straight out of the box, but still has the flexibility for software developers to create their own plugins for customized metrics on just about anything your users will care about.

Halo Vert.x – Building a Chatbot With Vert.x

A little while ago I did an internal talk at Shine describing the use cases and benefits of using asynchronous web technologies. As part of the talk I showcased a little demo I’d put together called Haloworld.

Haloworld is a simple web-based chat application with a chatbot called Hal (inspired by the infamous movie bot) who can answer various questions you throw at it.

At the time, I used Atmosphere on top of Jetty for the demo, in order to illustrate that these sort of  applications are now easy to build on top of the JVM. While it was relatively painless, it still seemed quite heavyweight compared to using something like Socket.io or SockJS with Node.js.

Not long afterwards, I came across Vert.x, a new JVM based framework for developing highly scalable networked applications. Vert.x seemed a much better fit for Haloworld, so I decided to port the demo across to it. You can find the results on our Github repository along with details on how to get up and running with the demo.

Converting Rails tests to 2.2 format

This is a short one, but we’ve had to do it a couple of times so I thought I’d put it up.

As you may be aware, Rails 2.2 introduced a new format for test names. Where you once might have had:

def test_should_do_stuff
  ...
end

You can now have:

test 'should do stuff' do
  ...
end

We’ve found this much easier to read and type – especially when your test names start to get big, or would be more readable if they contained characters that aren’t valid in Ruby method names.

However, if you’ve got lots of existing tests and want to shift them across to this format, it can be a pain to do it manually. Unless you’ve got a Ruby script, of course:

directory = 'unit'
Dir.mkdir("#{directory}.new")
Dir.new(directory).entries.each do |entry|
  original_filename = "#{directory}/#{entry}"
  File.open("#{directory}.new/#{entry}", 'w+') do |new_file|
    File.open(original_filename).readlines.each do |line|
      new_file.puts(line.gsub(/def test_*(.*)/) do |match|
          "test '#{$1.gsub('_', ' ')}' do"
        end)
    end
  end unless File.directory?(original_filename)
end

..which you’ll also find in this gist.

If you go into your ‘test’ folder and run this script, it’ll create a new directory called ‘unit.new’, that contains copies of all of your original unit tests, converted to the new format. Change ‘unit’ to ‘functional’, and it’ll do the same for your functional tests. Note that this will not do tests in subdirectories. YMMV. Feel free to steal and modify as you see fit.

How to bridge Ruby and AppleScript

Not that long ago I gave a demo in which I showed how it was possible to control iTunes from my laptop using native Ruby code.

This was all possible because of a great little gem called rb-appscript.

rb-appscript is a really neat Ruby to AppleScript bridge that gives you the power to effortlessly control any AppleScript aware applications from within your very own Ruby applications.

Launching iTunes and the DVD Player via irb

Assuming you have already installed the rb-appscript gem open up a Terminal shell and lauch irb. What we are going to do is in a few lines of code open up iTunes from ruby.
>> require 'appscript'
=> true (notice above we don't need to prefix the gem with rb-)
>> it = Appscript.app('iTunes')
>> it.run

Now how easy was that! iTunes just started up as soon as we ran it.run. Obviously if you already had iTunes running you would be disappointed because nothing would have happened.
You can use the above code for any application you could easily have just done
>> Appscript.app('Dvd Player').run
If you wanted to launch the dvd player.

Let’s tell iTunes what track we want to play

In the following example I am going to show you how you can retrieve a list of tracks from your Music library and tell iTunes to play our selection.
>> require 'appscript'
=> true (notice above we dont need to prefix the gem with rb-)
>> it = Appscript.app('iTunes')
>> it.run

Print out our tracks we have available to us
>> track_count = 0
>> it.playlists["Music"].tracks.get.each do | track |
?> puts "#{track_count += 1}. #{track.artist.get} - #{track.name.get}"
>> end

Ok play the 3rd track
t.playlists["Music"].tracks[3].play
Magic happens the song starts playing!

Script Editor + ASTranslate is your friend

Now the examples I have given you are very basic however if you want to do more then you should launch Script Editor and load the application dictionary that gives you access to all commands and properties you can change for that application.

ASTranslate is a great developer tool that allows you to write normal AppleScript commands and it will attempt to translate this into the rb-appscript equivalent!  So if you come across any cool AppleScripts out there you may be lucky and be able to translate them into ruby code!

I was amazed at how easy it was for me to command iTunes to do what I wanted it to do all from within my ruby application.

Useful references

http://appscript.sourceforge.net
http://www.apeth.com/rbappscript/10examples.html

Nesting URLs in RESTful resources

The Problem

A while back we had a Flex client that needed to be able to display search results received from a server. The server was designed RESTfully, returning XML results to the client. The Flex client would display these results nicely to the user, and when the user clicked on a result, their browser would be directed to a HTML page for editing the actual item.

The problem was that, when a client clicked on a result, we were manually constructing the resource URLs in the Flex client from the IDs buried in the search results. The Flex code looked a bit like this:


navigateToURL(new URLRequest('/item/' + result.id +"/edit"),"_self")

Furthermore, we had an impending need to support lots of different types of resources, so our logic for assembling URLs was about to become rather convoluted.

A Solution

Put the URLs into the search results. It’s blindingly simple (when you think about it, we do it all the time with pure HTML sites), but requires a subtle shift in mindset when you’re writing a fat client.

The solution was prompted by an article on REST Anti-Patterns I had read on InfoQ. – in particular, an anti-pattern called ‘Forgetting hypermedia’:

The first indicator of the “Forgetting hypermedia” anti-pattern is the absence of links in representations. There is often a recipe for constructing URIs on the client side, but the client never follows links because the server simply doesn’t send any.

It goes on to suggest that:

a client should have to know a single URI only; everything else…should be communicated via hypermedia, as links within resource representations.

You wouldn’t want to take this to an extreme, but it’s worth aiming for.

This approach helped simplify our code. We tweaked our server to write the resource URL into our XML, then had the Flex client simply navigate to the URL.

navigateToURL(new URLRequest(result.edit_path),"_self")

Furthermore, this solution was somewhat polymorphic – a link to any sort of resource could be passed back and the client would always be able to navigate to it.

Caveats

The only catch was that we had to tweak the server to manually write the paths into the XML. As our server was a Rails app, we found it easiest to just add a ‘.xml.erb’ file to our views directory:

xml.results do
  @results.each do |result|
    xml.result do
      xml.tag! :name, result.name
      xml.tag! :description, result.description
      xml.tag! :edit_path, edit_polymorphic_path(result)
    end
  end
end

Note the use of edit_polymorphic_path to ensure that the path to any sort of resource can be written out.

Providing User Feedback on Ajax calls in Rails

Following on from Tom’s post on Handling AJAX Errors using Prototype & Rails, I thought I’d take a look at how to ensure that all Ajax calls are subject to decent error handling – or any other sort of handling you want to do.

Towards the end of his post, Tom showed how he overrode form_remote_tag to include his error handling hooks. The only catch is that form_remote_tag isn’t the only way to trigger an Ajax call – there’s also form_remote_for, link_to_remote and a bunch other methods in ActionView::Helpers::PrototypeHelper.

You could override all of them if you wanted to, but it might just be easier to override the granddaddy of them all: remote_function. As far as I can tell, all of the PrototypeHelper methods use it, and you might even have to use it directly yourself for particularly sticky problems (that’s why it’s part of the public API). It’d look something like this in your helper class:

 def remote_function_with_user_feedback(options) # .. Do extra stuff... remote_function_without_user_feedback(options) end alias_method_chain :remote_function, :user_feedback 

Of course, that does beg the question: what exactly is the ‘extra stuff’?

Well, you could setup a callback that notifies the user if a server error occurs:

 options[:failure] = (options[:failure] || '') + 'alert("An error has occured on the server. Please try again later."); ' 

Or you could even try out the more sophisticated network-outage detection code in Tom’s original blog entry.

Alternately, have you ever clicked a button in an Ajax app and been left wondering why nothing seems to be happening? Wouldn’t it be good if something started spinning to tell you that ‘stuff’s happening’? Well here’s how you could do it:

 options[:before] = (options[:before] || '') + "$('spinner').src = '#{image_path "#active-spinner.gif"}';" options[:complete] = (options[:complete] || '') + "$('spinner').src = '#{image_path "#inactive-spinner.gif"}';" 

Note that in each case, it’s important not to overwrite any existing Javascript callbacks that may have been provided to remote_function – instead we just concatenate our own Javascript code to them. Thanks to Tom for pointing this out.

It’s also worth remembering that you’re extending a pretty important Rails method here so make sure you get it right…and by that I mean make sure you’ve got some tests 🙂

Handling AJAX Errors using Prototype & Rails

Ruby on Rails comes with a lot of nice helper methods for generating the JavaScript driving the AJAX calls to your controllers. Handling the responses from the HTTP server becomes a snap too, with Rails providing a few simple callbacks to handle the response from the server:

But what if the server doesn’t get a chance to respond at all? What if the user’s browser has been unexpectedly disconnected from the Internet? What if your server – god forbid – has crashed?

Most AJAX applications in the wild will simply sit there forever waiting for a response. Certainly, that’s what happened to us when we first tested such a scenario. This is unacceptable: if the browser is unable to communicate with the server, we need to let the user know somehow.

Internally, Rails uses a cross-platform Javascript library called Prototype to do the heavy lifting when making AJAX calls. When you call a helper method like link_to_remote, Rails generates code to instantiate a Prototype Ajax.Request object (or a Ajax.Updater object in some cases) to make the remote call when a certain event occurs. In the case of link_to_remote, this will be when a user clicks on the generated link.

Many of the helper methods accept a common set of options which provide callbacks for handling success or failure using the following callbacks:

  • :success will be executed if the server responds to the AJAX request with any sort of 200 HTTP OK code.
  • :failure will be executed if the server responds to the AJAX request with anything *but* a 200 code.
  • :complete will be executed after the AJAX request has finished, irrespective of the result.

These callbacks can be mapped to corresponding callbacks in Prototype on a one-to-one basis. For the above callbacks, the corresponding Prototype callbacks are called onSuccess, onFailure and onComplete, respectively. Unfortunately, only the “complete” callback will be triggered if some fatal error occurs when communicating the server – “failure” will only be triggered if we get a response back from the remote host indicating a server-side. We have no way of determining what will happen if the browser is unable to communicate with the server at all.

So how do we detect server outage? Well it just turns out that if your browser is unable to communicate with the server, an exception will be thrown. We can actually use this as a basis for detecting server outages and other classes of errors that might occur browser-side. For this purpose, Prototype provides onException which is triggered by your browser if any sort of exception is thrown while the AJAX call is in progress. If we can tap into this callback, we can ensure our AJAX methods will dutifully report any problems that might occur.

Unfortunately Rails doesn’t yet provide access to this callback out-of-the-box, so there is no “:exception” callback available to the Rails helpers. We don’t want to clutter our templates with ugly JavaScript: we’d prefer to keep with the existing Rails conventions if we can. So what can we do?

PrototypeHelper#build_callbacks is used by remote_function to generate the Javascript responsible for making an AJAX call in helper methods like link_to_remote, form_remote_tag and others. It turns out that this little method exposes virtually all the callbacks available to Prototype *but* onException.

This is the hook we’re looking for: here we override and adapt the original code in PrototypeHelper#build_callbacks to add the following method to our ApplicationHelper:


  include ActionView::Helpers::PrototypeHelper

  # ...

  def build_callbacks(options)
    callbacks = {}
    options.each do |callback, code|
      name = 'on' + callback.to_s.capitalize
      if CALLBACKS.include?(callback)
        callbacks[name] = "function(request){#{code}}"
      elsif callback == :exception
        callbacks[name] = "function(request,exception){#{code}}"
      end
    end
    callbacks
  end

This makes the :exception callback available to Rails using the familiar notation seen for the other types of callback. :exception is somewhat different in that it provides the exception object thrown internally available to your callbacks as the “exception” variable. Outside of that, it’s just like any other callback.

With this new callback available to us, we can then override the AJAX helper methods to include our error handling code in ApplicationHelper. As an example, here’s how we’d do it for form_remote_tag:


  alias old_form_remote_tag form_remote_tag

  def form_remote_tag(options={}, &block)
    # In the event of a network failure (or some other error), we want to display an error message
    options[:exception] = (options[:exception] || '') +
      'alert("Anerror occured while communicating with the server. Please try again later.");' +
      # Bubble the javascript exception up to prevent the caller from continuing execution (failure
      # to do this may result in multiple error messages being displayed!).
      'throw exception;'
    old_form_remote_tag(options, &block)
  end
end

With this code in place, any attempts to submit a form via AJAX will include our special error detection code, so we can rest at ease knowing our users will be notified if something goes horribly wrong at the network or language leve.

When using this technique to check, you have to be quite careful in your JavaScript code. Where any language- or browser-level exception that was thrown during the AJAX request was once ignored, it will be caught by the :exception callback. This means syntax errors in the response Javascript, bad content types (e.g. a content type of “text/javascript” with a body of HTML) or other errors will trigger the :exception callback. Unfortunately there doesn’t seem to be an obvious way to get more information about the nature of the error. For our purposes, however, this seemed to be enough if we were careful with the AJAX responses.

A further improvement to this approach might use a timeout mechanism to provide a hard limit for AJAX requests: if the server doesn’t respond with data within X seconds, abort the request and display an error. This would remove the reliance on the browser reporting errors via an exception, resulting in somewhat more robust code – the feedback may not be as immediate as if one were using exceptions, however.

Don’t leave your users out in the dark: keep them informed if things go wrong with an AJAX request.