Playing with Play Framework 2.3.x: REST, pipelines, and Scala

21 Apr 2015 Playing with Play Framework 2.3.x: REST, pipelines, and Scala

It’s an established trend in the modern software world that if you want to get something done, you’ll probably need to put together a web service to get do it. People expect data and services to be available everywhere, in a mobile world. With the plethora of frameworks and technologies available to go about implementing a web service, it becomes a chore to try using anything beyond what’s already familiar. But every now and then it’s an enjoyable experience to dive into something new and distinctly unfamiliar.

Enter the Play! Framework, which I’ve been—excuse the pun—playing with on-and-off for a bit over a year now. Whilst Node, and particularly Express, are excellent platforms when targeted at the small-scale—particularly so, as everyone and their grandmother could pick up Javascript—it’s in the larger projects that technologies like Rails, Spring, and the Play Framework, truly shine (more puns).

A forewarning though; as I learned Scala from scratch side-by-side with Play, most of the code samples below will be presented in Scala. While directly equivalent analogues are available in Java, due to personal preference and word counts, here thar be Scala.

A Quick Introduction

Play! Is a full-stack framework designed for scalability and reactivity. With a Java and Scala API available, and running mostly on the JVM, it’s also not too unfamiliar to most developers. Thanks to the Netty network framework, Play is designed to run as an Evented Server, so it uses one thread per CPU core and non-blocking I/O (also familiar to node, akka, and others). This is great when we’re spending a lot of time for data to come back from other services, e.g. the disk, a database, or some Big Data job, compared to Threaded Servers like Rails, PHP, Spring, etc.

Also in Play is an in-built JSON library; a Web Services library; multiple content parsers; asynchronous event helpers; database connectors, ORMs, and ODMs; WebSocket support; template engines; and so on. Rather than examining any specific database plugin or html template system, which are a dime-a-dozen, I decided for now I’d focus on what Play excels at: web services, JSON data pipelining, and performant asynchronous I/O.

Playing with Play, Pt I: Getting Started

Getting started turns out to be pretty simple. All that’s needed is a Java SDK and Play itself, which is provided by the ‘activator‘ toolset (available from https://www.playframework.com/ ). Activator can be run from the command line or as a visual server tool in the browser. As an alternative, IntelliJ and Eclipse offer Play support with Scala language support as well via easy-to-install plugins. All these tools document their usage. Play uses Ivy (via the SBT build tool) for dependency management, so Maven and Ivy are both available and easy to use.

Projects in Play look like this:

Axiom #1 – Controllers, Actions, and Routes are your everything.

Play uses Controllers to handle the bulk of its work; Controllers provide Actions, such as “index”, and Actions provide Results, such as a “200 OK”.

object MyController extends Controller {
  def index = Action {
    Ok(views.html.index("Your new application is ready."))
  }
}

Routing URLs to Actions in Play is also straightforward, and is defined in the /conf/routes file. A standard syntax makes routing paths to controller actions simple, e.g. /api/foo/bar

GET  /api/foo  controllers.Foo.list()

Or dynamic path variables, e.g. /api/foo/7, which are implicitly cast to the specified type

GET  /api/foo/:id  controllers.Foo.show(id: Int)

Query parameters, e.g. /api/foo?page=7, are remarkably easy to anticipate and pass as method variables as well, via the inbuilt ‘Option‘ class wrapper

GET  /api/foo  controllers.Foo.list(page: Option[Int])

And dynamic parts can even specify fixed or default parameter values as a fallback if the param is not supplied

GET  /api/foo  controllers.Foo.list(page: Option[Int] ?= Some(1))

Finally, all the standard HTTP Verbs are available, with the same syntactic stylings

PUT     /api/foo/:id  controllers.Foo.update(id: Int)
POST    /api/foo      controllers.Foo.add()
DELETE  /api/foo/:id  controllers.Foo.delete(id: Int)

Outside of the /conf/routes file, Play also offers reverse-routing in order to navigate to a URL via calling an Action. E.g.

def redirectToIndex = {
  Redirect(routes.Foo.list(Some(3)) // Navigates to: /api/foo?page=3
}

Neat.

Playing with Play, Pt II: Actions and Action Composition

We’ve seen in brief what an Action is in Play; in greater detail, an Action is actually defined as an object built as an instance of a Trait/Interface called ActionBuilder.

object Action extends ActionBuilder[Request] {
  def invokeBlock[A](request: Request[A], block: (Request[A]) => Future[Result]) = block(request)
}

ActionBuilder defines a single method which takes a Request object and a function block that maps Request to a Result. The code above in redirectToIndex is an example of such a function block.

Hence, custom ActionBuilder implementations can define reusable action stacks, which can then be used to build Actions. In English, this means we have a really neat way of dealing with boilerplate.

For example, Actions can be made to log some additional information whenever they occur. Instead of doing this in every Action…

def loggingAction = Action {
  Logger.info("Logging some things")
  Ok("Hooray.")
}

… the logic can be automated with a custom ActionBuilder. Essentially we just define a type of Action that logs something before it executes the function block. Like so:

object LoggingAction extends ActionBuilder[Request]{
  def invokeBlock[A](request: Request[A], block: (Request[A]) => Future[Result]) = {
    Logger.info("Logging some things")
    block(request)
  }
}

def loggingAction = LoggingAction{ request =>
  Ok("Hooray")
}

With a little extra tomfoolery, Play lets Actions be composed together in a chain, so that they can be written as a sequence of events like “Do X, then Y, and finally Z”:

case class Logging[A](action: Action[A]) extends Action[A] {
  def apply (request: Request[A]): Future[Result] = {
    Logger.info("Logging some things")
    action(request)
  }
}

def composedAction = Logging{
  Action{ request =>
    Ok("Hooray")
  }
}

Axiom #2 – Action Composition is to boilerplate what a hot knife is to butter.

Play provides additional helper traits to simplify the Action Composition chain into more atomic links. These are:

• ActionTransformers, which can modify the request object itself
• ActionFilters, which can selectively interrupt requests to produce error results
• ActionRefiners, which is the general case of each of the above
• ActionBuilders, which take requests and construct actions

All of these traits inherit from the base trait ActionFunction, which can be used for defining arbitrary action functionality that does not “fit” with the above helpers.

A common case for this is authentication and authorisation—for every request made to the web service, we’ll use Action Composition to extract a username and password from the Request and make it available to the Action’s function block.

This is as simple as making a descendent of Request, called AuthenticatedRequest, that provides additional username and password fields, and then creating an ActionBuilder that retrieves the user credentials from the request and constructs the descendent Request class before passing it to the function block.

class AuthenticatedRequest[A] (val username: String, val password: String, val request: Request[A]) extends WrappedRequest[A](request)

// An ActionBuilder that constructs an AuthenticatedRequest from a Request and passes it to the Action
object AuthenticatedAction extends ActionBuilder[AuthenticatedRequest] {
  def invokeBlock[A] (request: Request[A], block: (AuthenticatedRequest[A]) => Future[Result]) = {
    val optionalCredentialsTuple = parseUserCredentials(request)
    optionalCredentialsTuple.map { credentials =>
      block(new AuthenticatedRequest(credentials._1, credentials._2, request))
    } getOrElse Future.successful(Forbidden("Invalid User Credentials"))
  }
}
// Helper method retrieves the credentials from the request url params
def parseUserCredentials(request: RequestHeader): Option[(String, String)] = {
  val query = request.queryString.map { case(k, v) => k -> v.mkString }
  for {
    user <- query.get("username")
    pass <- query.get("password")
  } yield(user, pass)
}

Then, using the AuthenticatedAction builder within a controller Action, the username and password fields are made available with no extra work:

def authAction = AuthenticatedAction{ request =>
  Ok("Hello" + request.username)
}

Playing with Play, Pt III: Asynchronous I/O

Getting asynchronous logic functional in a web service quickly and reliably, is a major selling point of any framework. Play does this though Promises in Java, and Futures in Scala, which are a cinch to get going.

As the name implies, these classes are just wrappers that promise to eventually contain whatever object they are wrapping at some point in the future. These wrapper can perform operations to modify and transform their contents that will be executed only once they have received whatever data they are wrapping. For example:

val futureFoo = LoadFoo() // Returns a Future[Foo] object
val futureBar = futureFoo.map(foo => TransformFooToBar(foo)) // Returns a Future[Bar] object

Here, within the scope of the map operation, foo is an instance of Foo. Outside of this scope however, the object is still a Future.

Axiom #3 – Intuitive async workloads and pipelines saves lives. It’s as easy as saying “when the mail arrives, then I’ll do something with it.”

In Play there is a convenient sub-class of Result called an AsyncResult, built for returning Future[Result] objects to the client only once it is available. Combined with Play’s Web Services library, we can make asynchronous calls to some online web service—which returns a Future response—then transform that response to a result for our own client.

To test, I’ve generated and uploaded an array of sample user objects as JSON via http://www.json-generator.com/ , where each user contains a username and password. Putting everything together looks like this:

def asyncAction = Action.async{ request =>
  val futureJsUserArray = WS.url("http://www.json-generator.com/api/json/get/cfLxEnRoAy?indent=2").get()
  futureJsUserArray.map{jsResponse => Ok(jsResponse.body).as("application/json")}
}

 Sequentially nested asynchronous tasks or parallel tasks are only slightly more difficult to deal with thanks to some other helpful tricks. Consider the following:

val stringArr = "I am a string".split(" ") // Returns Array(I, am, a, string)
val nestedStringArr = stringArr.map( str => str.toCharArr.toList ) // Returns Array(List(I), List(a,m), List(a), List(s,t,r,i,n,g))
val flatStringArr = stringArr.flatMap( str => str.toCharArr.toList ) // Returns Array(I,a,m,a,s,t,r,i,n,g)

 Now, in much the same way:

val futureResponse = WS.url("http://www.example.com").get() // Returns a Future[WSResponse]
val flattenedFutureResponse = futureResponse.flatMap( response => WS.url(response).get() ) // Returns a Future[WSResponse]

Just like a nested collection, flatMap can also be used on Futures to deal with nested callbacks. Multiple parallel jobs are even easier, as they can be comprehended into a single Future and then mapped to a result only once all the jobs have finished:

val futureA = WS.url("http://www.example.com").get()
val futureB = WS.url("http://www.foo.com").get()
val futureC = WS.url("http://www.example.com").get()

Future.sequence(Seq(futureA, futureB, futureC)).map{ futureSequence => 
  // Concatenate the response bodies
  Ok(futureSequence.map(_.body).reduce(_+_)).as("text/html")
}

Finally, we can do parallel and sequential async tasks thanks to for-comprehensions in Scala, instead of deep-nesting or sequence collections.

 val allMyFutures = for {
  a <- WS.url("http://www.json-generator.com/api/json/get/bUSnjMfGWa?indent=2").get()
  b <- WS.url(a.body.substring(1, a.body.length - 2)).get() // returns "http://www.example.com"
  c <- WS.url(b.body.substring(1, a.body.length - 2)).get() // returns content of http://www.example.com
  d <- WS.url("http://www.example.com").get()
} yield (a.body + b.body + c.body + d.body)

allMyFutures.map(Ok(_).as("text/html"))

Much easier than nested maps and flatMaps. And any day spent without a dozen nested Javascript callbacks is a good day indeed.

Playing with Play, Pt IV: Coast-to-Coast Data Pipelines

Play’s Actions allow for taking an optional body parser argument when they are defined. The body parser allows for the incoming request from the client to have its body parsed into some particular format, such as into an incoming file or JSON stream.

def parsingAction = Action(parse.json) { request =>
  Ok(request.body)
}

Axiom #4 – Native JSON object support is kind of excellent.

Play’s JSON library is perhaps the best implementation I’ve seen of native JSON traversal without the overhead of deserialisation into concrete class instances. JSON branches can be picked and traversed using the “branch” operator, ‘\’, or recursively searched using ‘\\’. Transforms to prune, modify, or crop branches can also be applied, which all form the basis of what Play calls its “coast-to-coast” JSON pipeline.

Of course, it also provides de/serialisation support either by explicit serialisation definitions, or by implicit compile-time code injection thanks to Scala macros which were added in Scala 2.11.

So, trying it out in practice, we try putting two and two together from what we’ve covered so far. From the collection of user data earlier, there is a user, Christensen Haley, with username field “chaley” and password field “Haley”. When a user attempts to access the user data resource, we’ll authenticate them and restrict them to only viewing their own profile data.

def asyncAuthAction = AuthenticatedAction.async{ request =>
  val credentials = (request.username, request.password)
  val futureJsUserArray = WS.url("http://www.json-generator.com/api/json/get/cfLxEnRoAy?indent=2").get()
  futureJsUserArray.map{ jsResponse =>
    val jsonArr = Json.parse(jsResponse.body).as[List[JsObject]]
    val userEntry = jsonArr.find(jsObj => (jsObj \ "username").as[String] == credentials._1 && (jsObj \ "password").as[String] == credentials._2)
    Ok(userEntry).as("application/json")
  }
}

Here, the web service response is parsed to a collection of JSON objects. Then the collection can be iterated over and the username and password branches picked from each object, to compare their value to the AuthenticatedRequest‘s credentials. Once the matching user’s branch is found, the JSON object can be returned.

Playing with Play, Pt the Last: Putting it all Together

Unlike other platforms such as Node and Express, which are very forgiving, I’ve found Play (especially with Scala) to be the opposite. It’s easy to use, but it’s difficult to use well knowing the tricks. Here’s a few more I’ve picked up.

Performing repetitive asynchronous tasks, e.g. looking up database items, authorising users, etc, is all within the realms of Action Composition. For example, looking up and authorising a user can be done simply with an ActionFilter composed with the AuthenticatedAction builder created earlier:

// ActionFilter returns an Option[Result], where Some indicates a filtered result, and None will continue with composition

object AuthorisedAction extends ActionFilter[AuthenticatedRequest] {
  def filter[A] (request: Request[A]) = Future.successful {
    if (UserIsAuthorised(request.username, request.password)) 
      None
    else
      Some(Forbidden)
  }
}

Composing this action results in the action execution stalling until the user is authorised against whatever service UserIsAuthorised uses. If this method performs an asynchronous lookup, then the thread can be put on to handle other requests while the result is awaited, and the client waits a little longer for one async task to complete, then the other.

For some tasks, such as simple reads, it may be more efficient to execute the read in parallel with the authorisation, and defer the response to the client until both operations have completed. For this, use an ActionFunction to kick off the action and the auth calls, then await both before responding.

object AsyncAuthorisedAction extends ActionFunction[AuthenticatedRequest, AuthenticatedRequest] {
  override def invokeBlock[A](request: AuthenticatedRequest[A], block: (AuthenticatedRequest[A] => Future[Result])): Future[Result] = {
    val isAuthorised = UserIsAuthorised(request.username, request.password) // Returns Future[Result]
    val result = block(request) // Returns Future[Result]
    isAuthorised.flatMap{ 
      case true => result
      case false => Future.successful(Forbidden)
    }
  }
}

It may also be desirable for some actions to define own variables for use during the composition phase; e.g. we may wish to define that an action that requires the user to have read access on a resource. First, define a new WrappedRequest that builds off an AuthenticatedRequest, which takes an optional list of permissions.

class PermittedRequest[A](val perms: Option[List[String]], val request: AuthenticatedRequest[A]) extends WrappedRequest[A](request) {
  def username = request.username
  def password = request.password
}

Then, use an Action Transformer to construct it.

def PermittedAction(perms: Option[List[String]]) = new ActionTransformer[AuthenticatedRequest, AuthorisedRequest] {
  def transform[A](request: AuthenticatedRequest[A]) = Future.successful {
    new AuthorisedRequest(perms,request)
  }
}

If the Authorised Actions above are hence modified to take a PermittedRequest, then Play can compose all the above into an action like so:

def permittedAction = (AuthenticatedAction andThen PermittedAction(Some(List("ADMIN_ACCESS","READ_ACCESS"))) andThen AsyncAuthorisedAction) { request => 
  Ok("Hooray, the user has permission to be here")
}

Which will either complete the request and return successfully, or else will fail authentication or authorisation and return a fail result.

Dealing with errors is another straightforward job, simply by using the recover method—mostly analogous to catching exceptions, but with Scala pattern-matching syntactic sugar to boot.

def throwableAction = Action { request =>
  WS.url("Not a valid URL at all").get().map(result => Ok(result.body).as("text/plain"))
  .recover {
    case t: Exception => InternalServerError("Uh oh:" + t.getLocalizedMessage)
    case _ => InternalServerError("Uh oh")
  }
  Ok("Hooray")
}

Recovering can be done with as much or as little specificity as desired. Pattern matching against the _ wildcard will match against literally anything that goes wrong, while matching against particular object types offers more specificity. When providing a result dependent on multiple service calls for example, responses could be wrapped in an Option by using map and recover, then stitch the results together to provide partial responses—sometimes 2 out of 3 successful service calls is still a success, after all.

Wrapping Up

There’s a thousand other features and libraries to explore within Play! It’s a tremendous learning curve, especially as Scala is a first-party citizen in Play yet generally doesn’t earn as much of the spotlight in the development community. Play itself is massive, but in architecting massive web services, it’s got all the tools to get the job done quickly, cleanly, and reactively. Plus it’s a decent bit of fun, which wins it some extra points in its favour.

For anyone interested, I’ve made a demo project of all the source (and then some) available on Github for use: https://github.com/sampsonjoliver/PlayTest/