With the arrival of Scala 2.10 came two interesting new features: the reflection API and macros. The possibilities for macros are many, and one that I want to explore is how you can use a macro to create a Play Framework REST API from a data model represented as a set of case classes. In this and subsequent articles I'll be using a simple book store type application with case classes for Book, Author and Category in the model package.
Normally in the Play Framework you'd declare your rest API in the routes file, something like:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
GET /api/category/:id controllers.Categories.get |
However, as you can't use custom Scala in routes (since Play 2.0, anyway), we'll be designing a macro for use in the onRouteRequest function of the Global object that extends the GlobalSettings trait. This function takes a request and returns an Action. The selection of what Action to return will be based on the URL, so we'll have to match the path to the right case class.
In this first article, we'll be working out how to do that matching based on an unknown number of case classes in the application's model.
If we weren't using a macro, we'd probably do something like:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
def caseStmt(s : String) : Option[String] = { | |
val bookRegex = "/api/book/(.+)".r | |
val authorRegex = "/api/author/(.+)".r | |
val categoryRegex = "/api/category/(.+)".r | |
s match { | |
case bookRegex(id) => Some("Book with ID " + id) | |
case authorRegex(id) => Some("Author with ID " + id) | |
case categoryRegex(id) => Some("Category with ID " + id) | |
case _ => None | |
} | |
} |
Our macro will need to generate the same function by iterating over all the found case classes. Firstly we need to declare the macro, and then complete the implementation, as follows:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
def macroCase(s : String) : Option[String] = macro macroCaseImpl | |
def macroCaseImpl(c : Context)(s : c.Expr[String]) : c.Expr[Option[String]] = { | |
import c.universe._ | |
reify(Some("TODO")) | |
} |
What we've done here is declare a macro as described on the documentation linked above. The macro context's universe is imported - it is important to make sure you import the universe exactly as above, as if you use an import like c.{universe => u} then for some reason erasure stops them working properly. We've then used the universe's reify function to return an expression for Some("TODO") - the reify function basically turns the enclosed scala into an AST tree expression.
Finally, we need a unit test to check we've implemented the method correctly:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import org.junit.Test | |
import org.junit.Assert._ | |
class MacroTest { | |
def caseUsingMacro(s : String) = Macro.macroCase(s) | |
@Test def testCase = { | |
assertEquals("Book with ID 123", caseUsingMacro("/api/book/123").get) | |
assertEquals("Author with ID 456", caseUsingMacro("/api/author/456").get) | |
assertEquals("Category with ID 789", caseUsingMacro("/api/category/789").get) | |
assertEquals(None, caseUsingMacro("/api/comic/321")) | |
} | |
} |
The caseUsingMacro function will have its body replaced with the result of macro call - i.e. it is here that the function is going to be generated, replacing Macro.macroCase(s).
Now we're ready to start implementing the macroCaseImpl function from above - but what do we need to return, and how can we debug our macro's output to see how we're doing. Fortunately, there are two very useful functions in the universe - show and showRaw. Given an AST tree, the show function will return a pretty-formatted string of the equivalent scala code. The showRaw function will return a string of the AST making up the tree. As what we're trying to implement is something that for our 3 case classes will look like the caseStmt function above, we can get a good hint for what we need to end up with to look at its equivalent AST, which we can get by doing:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
println(showRaw(reify{ | |
val bookRegex = "/api/book/(.+)".r | |
val authorRegex = "/api/author/(.+)".r | |
val categoryRegex = "/api/category/(.+)".r | |
s match { | |
case bookRegex(id) => Some("Book with ID " + id) | |
case authorRegex(id) => Some("Author with ID " + id) | |
case categoryRegex(id) => Some("Category with ID " + id) | |
case _ => None | |
} | |
}.tree)) |
The results of this println will be output to the console for the compilation [1]. We get something like this:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Block(List(ValDef(Modifiers(), newTermName("bookRegex"), TypeTree(), Select(Apply(Select(Ident(scala.Predef), newTermName("augmentString")), List(Literal(Constant("/api/book/(.+)")))), newTermName("r"))), ValDef(Modifiers(), newTermName("authorRegex"), TypeTree(), Select(Apply(Select(Ident(scala.Predef), newTermName("augmentString")), List(Literal(Constant("/api/author/(.+)")))), newTermName("r"))), ValDef(Modifiers(), newTermName("categoryRegex"), TypeTree(), Select(Apply(Select(Ident(scala.Predef), newTermName("augmentString")), List(Literal(Constant("/api/category/(.+)")))), newTermName("r")))), Match(Ident(newTermName("s")), List(CaseDef(Apply(Ident(newTermName("bookRegex")), List(Bind(newTermName("id"), Ident(nme.WILDCARD)))), EmptyTree, Apply(Select(Ident(scala.Some), newTermName("apply")), List(Apply(Select(Literal(Constant("Book with ID ")), newTermName("$plus")), List(Ident(newTermName("id"))))))), CaseDef(Apply(Ident(newTermName("authorRegex")), List(Bind(newTermName("id"), Ident(nme.WILDCARD)))), EmptyTree, Apply(Select(Ident(scala.Some), newTermName("apply")), List(Apply(Select(Literal(Constant("Author with ID ")), newTermName("$plus")), List(Ident(newTermName("id"))))))), CaseDef(Apply(Ident(newTermName("categoryRegex")), List(Bind(newTermName("id"), Ident(nme.WILDCARD)))), EmptyTree, Apply(Select(Ident(scala.Some), newTermName("apply")), List(Apply(Select(Literal(Constant("Category with ID ")), newTermName("$plus")), List(Ident(newTermName("id"))))))), CaseDef(Ident(nme.WILDCARD), EmptyTree, Ident(scala.None))))) |
We'll break this apart into useful fragments as we get to using them.
So, first we need to get a list of all the case classes in the model package. To do this, we can iterate over the context's enclosingRun.units property, matching on elements of the AST tree for each compilation unit to find out the package name, and then checking that we've got a case class. The rough code we need for this is:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
c.enclosingRun.units.filter(unit => unit.body.find(packageFinder(_, "model")).isDefined && unit.body.find(classFinder).isDefined) |
We then need to implement the packageFinder and classFinder functions. To check that the compilation unit is in the desired package, the first of these needs to return true when the Tree instance is a PackageDef with a name equal to, or starting with, the second argument, so:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
def packageFinder(t : Tree, pkg : String) = t match { | |
case PackageDef(id, _) => id.name.decoded.equals(pkg) || id.name.decoded.startsWith(pkg+".") | |
case _ => false | |
} |
Then to check that the compilation unit contains a case class, the second function needs to return true if the Tree instance is a ClassDef with a CASE modifier:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
def classFinder(t : Tree) = t match { | |
case ClassDef(mods, _, _, _) => mods.hasFlag(Flag.CASE) | |
case _ => false | |
} |
So now we've got a list (well, in fact it's an Iterator) of the case class compilation units in the package we need to turn them into regular expressions to match against the URL, and into case statements to do the matching.
Firstly, the regular expressions. If we take the AST that we generated above, we can see that the value definition for one regex is:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ValDef(Modifiers(), newTermName("bookRegex"), TypeTree(), Select(Apply(Select(Ident(scala.Predef), newTermName("augmentString")), List(Literal(Constant("/api/book/(.+)")))), newTermName("r"))) |
Of the four parameters to the ValDef creator, the first, Modifiers(), and third, TypeTree(), we can take straight into our generated AST. The second parameter we can generate using a context function to create a new unique val name, c.fresh("regex$"). The last parameter is using the implicit augmentString function on a String so that we can then call the r function to turn it into a Regex. However, we can do this more simply using the reify function combined with a string expression for the regex we want to use, which we can construct in the macro using the class name:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
val regexExpr = c.Expr[String](Literal(Constant("^/api/"+className.toLowerCase+"/(.+)"))) | |
val valueDefinition = ValDef(Modifiers(), valName, TypeTree(), reify(regexExpr.splice.r).tree) |
Here we've created an Expr object of a constant that is the regex string, and then used the splice function, which is a special function that allows an Expr object to be grafted into the middle of the block being reified. We then just call the r function inside the reify block for it to be turned into a Regex.
The other bit of AST that we need to generate for each class name is the case statement of the match block. Again, from the above, we can see that this should look something like:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
CaseDef(Apply(Ident(newTermName("bookRegex")), List(Bind(newTermName("id"), Ident(nme.WILDCARD)))), EmptyTree, Apply(Select(Ident(scala.Some), newTermName("apply")), List(Apply(Select(Literal(Constant("Book with ID ")), newTermName("$plus")), List(Ident(newTermName("id"))))))) |
Reformatting, this becomes a little easier to understand:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
CaseDef( | |
Apply(Ident(valName), List(Bind(newTermName("id"), Ident(nme.WILDCARD)))), | |
EmptyTree, | |
Apply(Select(Ident("Some"), newTermName("apply")), List(Apply(Select(Literal(Constant("Book with ID ")), newTermName("$plus")), List(Ident(newTermName("id")))))) | |
) |
Notice that again we've swapped the newTermName("bookRegex") for valName which is the fresh name we got using the "regex$" base name, the name of the defined value that is the Regex.
Now, we need to collect up all the val definitions and case statements so that they can be assembled into a block of scala code. We also need to add the default case:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
CaseDef(Ident(nme.WILDCARD), EmptyTree, Ident("None")) |
Putting all this together, and getting the class name from the AST tree for the compilation unit, we get:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
val regexes = ListBuffer[ValDef]() | |
val cases = ListBuffer[CaseDef]() | |
modelClasses.foreach(unit => { | |
val valName = c.fresh("regex$") | |
val className = unit.body.find(classFinder).get.asInstanceOf[ClassDef].name.decoded | |
Ident(valName) | |
val regexExpr = c.Expr[String](Literal(Constant("^/api/"+className+"/(.+)"))) | |
regexes += ValDef(Modifiers(), valName, TypeTree(), reify(regexExpr.splice.r).tree) | |
cases += CaseDef( | |
Apply(Ident(valName), List(Bind(newTermName("id"), Ident(nme.WILDCARD)))), | |
EmptyTree, | |
Apply(Select(Ident("Some"), newTermName("apply")), List(Apply(Select(Literal(Constant(className + " with ID ")), newTermName("$plus")), List(Ident(newTermName("id")))))) | |
) | |
}) | |
cases += CaseDef(Ident(nme.WILDCARD), EmptyTree, Ident("None")) |
So, we assemble the cases and regexes into a block. The return of the block is going to be the result of the match statement, and the regexes are declared before that final statement, so (remember, s is the original parameter to the macroCaseImpl function):
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
val block = Block(regexes.toList, Match(s.tree, cases.toList)) |
To check that this AST tree really is the Scala code we hope it is, we can use the show function I mentioned before:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
println(show(block)) |
And finally, we return the Expr that contains our block of scala:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
def macroCaseImpl(c: Context)(s: c.Expr[String]): c.Expr[Option[String]] = { | |
import c.universe._ | |
import c.{ universe => u } | |
def packageFinder(t: Tree, pkg: String) = t match { | |
case PackageDef(id, _) => id.name.decoded.equals(pkg) || id.name.decoded.startsWith(pkg + ".") | |
case _ => false | |
} | |
def classFinder(t: Tree) = t match { | |
case ClassDef(mods, _, _, _) => mods.hasFlag(Flag.CASE) | |
case _ => false | |
} | |
val modelClasses = c.enclosingRun.units.filter(unit => unit.body.find(packageFinder(_, "model")).isDefined && unit.body.find(classFinder).isDefined) | |
val regexes = ListBuffer[ValDef]() | |
val cases = ListBuffer[CaseDef]() | |
modelClasses.foreach(unit => { | |
val valName = c.fresh("regex$") | |
val className = unit.body.find(classFinder).get.asInstanceOf[ClassDef].name.decoded | |
Ident(valName) | |
val regexExpr = c.Expr[String](Literal(Constant("^/api/" + className.toLowerCase + "/(.+)"))) | |
regexes += ValDef(Modifiers(), valName, TypeTree(), reify(regexExpr.splice.r).tree) | |
cases += CaseDef( | |
Apply(Ident(valName), List(Bind(newTermName("id"), Ident(nme.WILDCARD)))), | |
EmptyTree, | |
Apply(Select(Ident("Some"), newTermName("apply")), List(Apply(Select(Literal(Constant(className+" with ID ")), newTermName("$plus")), List(Ident(newTermName("id"))))))) | |
}) | |
cases += CaseDef(Ident(nme.WILDCARD), EmptyTree, Ident("None")) | |
val block = Block(regexes.toList, Match(s.tree, cases.toList)) | |
println(show(block)) | |
c.Expr[Option[String]](block) | |
} |
And when we compile, we see that our Block tree is equivalent to:
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
{ | |
val regex$1 = Predef.augmentString("^/api/author/(.+)").r; | |
val regex$2 = Predef.augmentString("^/api/book/(.+)").r; | |
val regex$3 = Predef.augmentString("^/api/category/(.+)").r; | |
s match { | |
case regex$1((id @ _)) => Some.apply("Author with ID ".$plus(id)) | |
case regex$2((id @ _)) => Some.apply("Book with ID ".$plus(id)) | |
case regex$3((id @ _)) => Some.apply("Category with ID ".$plus(id)) | |
case _ => None | |
} | |
} |
Here we can see the generated val names and the familiar regexes and match-case block returning the Option values that are expected. On running the unit test, we find it now passes.
Phew!
In the next stage of this adventure, I'll be extending this to use persistence of case classes as declared in their companion object, and deducing what REST operations are available based on what data access functions are exposed.
[1] If you're using the Scala IDE it might take you a while to find the compilation output - what you need to do is turn up the logging to DEBUG in Window -> Preferences, Scala -> Logging. Your console output will then appear in [workspace]/.metadata/.plugins/org.scala-ide.sdt.core/scala-ide.log