Building a REST service in Scala with Akka HTTP, Akka Streams and reactive mongo

At the end of last year I wrote a couple of articles that showed how you can use Spray.io to create a Scala based REST service () and how to create a websocket server with Scala, Akka and reactivemongo (). I wanted to explore the REST server part a bit more, but found out that at the end of 2013 Spray.io was acquired by typesafe, and would be integrated with the Akka stack. So in this article we’ll look at how you can use the Akka HTTP functionality to create a simple web server, and in a follow up we’ll look at how the routing from Spray.io was ported to Akka.

In this article we’ll take the following steps:

• Get some dummy data into mongoDB for testing.
• Create a server using Akka Http that uses a simple asynchronous handler to process requests.
• Create a server which uses a custom flow graph to process incoming requests.
• Test both of these servers with a http client also created with Akka-Http.

So lets start with some preparation work and get some data into mongoDB for us to work with.

Loading data into mongoDB

For this example we use some stock related information which you can download from here (http://jsonstudio.com/wp-content/uploads/2014/02/stocks.zip). You can easily do this by executing the following steps:

First get the data:

    wget http://jsonstudio.com/wp-content/uploads/2014/02/stocks.zip

Start mongodb in a different terminal

    mongod --dbpath ./data/

And finally use mongoimport to import the data

     unzip -c stocks.zip | mongoimport --db akka --collection stocks

And as a quick check run a query to see if everything works:

jos@Joss-MacBook-Pro.local:~$ mongo akka      
MongoDB shell version: 2.4.8
connecting to: akka
> db.stocks.findOne({},{Company: 1, Country: 1, Ticker:1 } )
{
        "_id" : ObjectId("52853800bb1177ca391c17ff"),
        "Ticker" : "A",
        "Country" : "USA",
        "Company" : "Agilent Technologies Inc."
}
> 

At this point we have our test data and can look at the code required to run a server.

Create a server which uses a simple asynchronous handler to process requests

To work with Akka Http and access the data in mongo we’re going to need some additional libraries. So before we do anything else, lets first look at the sbt build file we’ve used for this article:

When you look through the dependencies you’ll see the usual suspects:

• akka-http-core-experimental contains all the http server and client stuff we're going to use. This library depends on akka-stream so we'll also get that library on our class path.
• reactiemongo allows us to connect to mongo in a reactive way.
• I've also included play2-reactivemongo and play-json which makes converting the BSON returned from mongo to JSON a lot easier.
• Finally, for logging we add logback.

Now before we look at the code required to run the server, lets quickly look at how we’ll query mongo. For this we’ve created a simple helper object creatively named Database:

Not that much to explain. The most important thing to notice here is that both the find functions return a future, so calls to these functions won’t block. Now that we’ve got the basics out of the way, lets look at the code for the first http server which uses an asynchronous handler.

In this piece of code we create a http server which listens on port 8091. We process each connection that is made with an asyncHandler. This handler should return a Future[HttpResponse]. Lets look at this handler next:

As you can see from this code the handler code is pretty straightforward. We use pattern matching to match a specific url and use the Database object we saw earlier to query mongo. Note the calls to convertToString. These are a couple of helper methods that convert BSON to JSON using the play libraries we included earlier:

When we start this server and open the adres in the browser, we’ll see something like this:

Easy right? Now lets look at a bit more advanced scenario.

Create a server which uses a custom flow graph to process incoming requests.

Akka-http internally uses akka-streams to handle http connections. This means that we can use akka-streams to easily handle http requests in a reactive manner. For a linear flow we can use the standard flow api provided by akka. For more advanced graphs akka-streams provides it’s own DSL, with which you can very easily create more complex graphs where stream events are processed in parallel.

Lets create a new serverbinding that listens on port 8090:

This serverbinding is created in the same manner as we did earlier. The main difference is that this this time we don’t pass the processing of the request onto a handler, but we specify an instance of a flow with the name broadCastMergeFlow. This broadcast merge flow looks like this:

The most important part are the last couple of lines in this code fragment. Here we draw a graph that defines how a message is handled when it is processed by the server. In this case we first broadcast the incoming http request to three parallel streams. In each stream we next make a call to our database to get a ticket. Next we merge the results back together (a merge takes the first available upstream even) and create a response. So depending which of the steps is the fastest we return a ticker either for GOOG, AAPL or MSFT. To see the result better we added a sleep to the getTickerHandler:

Neat right! Akka-streams provides a number of basic building blocks you can use to create these flows (for more info see their documentation: http://doc.akka.io/docs/akka-stream-and-http-experimental/1.0-M2/scala/stream-graphs.html). For instance if we want to zip the responses of the steps together we could create a flow like this:

I really like how this works and how easy it is to visualize data flowing through the different steps. If we use the merge approach you’ll see a result which looks something like this (when called 10 times):

{"_id":{"$oid":"52853804bb1177ca391c2221"},"Ticker":"GOOG","Profit Margin":0.217
{"_id":{"$oid":"52853804bb1177ca391c2221"},"Ticker":"GOOG","Profit Margin":0.217
{"_id":{"$oid":"52853800bb1177ca391c1809"},"Ticker":"AAPL","Profit Margin":0.217
{"_id":{"$oid":"52853807bb1177ca391c2781"},"Ticker":"MSFT","Profit Margin":0.282
{"_id":{"$oid":"52853804bb1177ca391c2221"},"Ticker":"GOOG","Profit Margin":0.217
{"_id":{"$oid":"52853800bb1177ca391c1809"},"Ticker":"AAPL","Profit Margin":0.217
{"_id":{"$oid":"52853807bb1177ca391c2781"},"Ticker":"MSFT","Profit Margin":0.282
{"_id":{"$oid":"52853804bb1177ca391c2221"},"Ticker":"GOOG","Profit Margin":0.217
{"_id":{"$oid":"52853800bb1177ca391c1809"},"Ticker":"AAPL","Profit Margin":0.217
{"_id":{"$oid":"52853807bb1177ca391c2781"},"Ticker":"MSFT","Profit Margin":0.282

The final part I’d want to show it how you can also use the same approach when you create a http client with akka-http.

Test both of these servers with a http client also created with Akka-Http

Akka-http also provides functionality to easy setup a http client that also uses a stream/flow based approach of message processing. The following listing shows the complete running client:

I won’t go into detail here, since the code follows the same process as for the HTTP server. That’s it for this article and an introduction into akka-stream and akka-http. I really like their approach to message processing and creating readable, reactive code. In a future article we’ll look at some other aspects of akka-http (routes for instance).