ScaLearning 6 – DSL Exploration & Operator Interpretation

facebooktwitterdiggdzonestumbleuponredditdelicious


Like many developers who make the journey from Java to Scala, I often find myself amazed at how much easier it is to do some things, or how much easier it is to express myself in Scala.

“ScaLearning” will be a series of short blog-posts just documenting little tidbits I find interesting, confusing, amusing, or otherwise worthy of talking about.
 
 

DSL Exploration: Our Tale So Far

For those interested in the whole DSL Exploration “mini-series”, we began with DSL Exploration & Operator Notation.

To quickly summarize our tale so far, I decided to explore the creation of a DSL to expose some of Scala’s nuances. My project of choice was codifying the LOLCode language as a Scala DSL. So far we have created a syntax for outputting data to the screen.

Working syntax so far:

U SEEZ "Hello World" !
U SEEZ "Hello World" !!

 
 

Learned Lesson 3: Operation Interpretation

So far we’ve been doing all of our work in the Scala REPL. While extremely powerful, the REPL does have a few known limitations – it sometimes interprets a line differently than it would have been interpreted from a source file. Today we’ll discover one such situation, and explore the reason.

Let us start with a Scala source-file which will act as our basis:

class Printable(output:String) {
  var bang = false;
  def ! = { print(output); }
  def !! = { println(output); }
}
class OutputStream {
  def SEEZ(output: Printable) : Unit = println(output.toString)
}
val U = new OutputStream

U SEEZ "Hello"!
U SEEZ " World"!!
U SEEZ "Goodbye"!
U SEEZ " World"!!

The code above is equivalent to what we were writing into the REPL in previous posts, but let’s store that in a file ScaLol.scala and run it:

tcalder:bin tcalder $ scala ScaLol.scala
/.../ScaLol.scala:10: error: value ! is not a member of java.lang.String
U SEEZ "Hello"!

Clearly not what the REPL was telling us. At first glance, it now appears as if it’s completely changed its operational precedence, and that’s true… but only sort of.

After trying various combinations of the code (none of which helped), I eventually gave up and started reading the Scala Reference again.

Let’s review what we’ve said so far:

  • Methods with one argument can be used as an infix operator
  • Methods with no arguments can be used as a postfix operator
  • Unary operators for !, +, -, and ~ can be used if equivalent “unary_*” methods are defined
  • Postfix operators always have lower precedence than infix operators
  • Infix operators have an order of precedence which includes letters being lower than special characters

Reasoning briefly about our code, the REPL clearly evaluates the “!” as a postfix operator, which gives it the lowest precedence and results in:

U.SEEZ("Hello").!()

Yet in ScaLol.scala we appear to have attempted to run “!” on String. Since Postfix have low precedence, we just be attempting to evaluate “!” as an Infix operator. This would make sense since special characters have a higher operational precedence:

U.SEEZ( "Hello World".!(.....?) )

Now if you’re like me, you’re wondering what on earth it could possibly be trying to fill in as the argument to this strange new Infix “!”. The answer is, it’s trying to use the next line as a continuation on the current:

U.SEEZ("Hello".!(U)).SEEZ("World"... // Or some rough equivalent

From the ScalaReference:

Scala is a line-oriented language where statements may be terminated by semicolons or newlines. A newline in a Scala source text is treated as the special token
“nl” if the three following criteria are satisfied:

  1. The token immediately preceding the newline can terminate a statement.
  2. The token immediately following the newline can begin a statement.
  3. The token appears in a region where newlines are enabled.

“U” can start a statement, and “!” can terminate one, since the REPL clearly didn’t get upset at either of these. Newlines are enabled in our code – there’s a lot of logic in the Reference that explains why if you’d like to explore it.

Based on this, we can conclude that our “\n” after the “!” is, in fact, a “[nl]”.

The second piece of the puzzle that we need is the following (summarized) syntactic definitions, expressed in the Scala Ref using Extended Backus–Naur Form:

PostfixExpr ::= InfixExpr [id [nl]]
InfixExpr ::= PrefixExpr
                     | InfixExpr id [nl] InfixExpr
PrefixExpr ::= [‘-’ | ‘+’ | ‘~’ | ‘!’] SimpleExpr

In specific, draw your attention to the similarity between the definition of “PostfixExpr” and the second definition of “InfixExpr”, and pay special attention to our friend “nl”.

The following is a (rough) breakdown of what we’ve actually attempted:

U SEEZ "Hello World" !
U SEEZ "Goodbye World" !

SimpleExpr id SimpleExpr id nl SimpleExpr id SimpleExpr id

PrefixExpr id PrefixExpr id nl PrefixExpr id PrefixExpr id

InfixExpr id InfixExpr id nl InfixExpr id InfixExpr id

Bingo, there’s our culprit. The second “id” in the expression is our “!” operator. Based on the formal specification, “InfixExpr id nl InfixExpr” can be simplified as a single “InfixExpr”, and since “!” has higher precedence than “SEEZ”, it will clearly be evaluated first.

As a final proof, try saving the following code as “ScaLol2.scala” and running it. Note how we use two new-lines in-between “SEEZ” calls. This results in two “nl” entities instead of one, which prevents the compiler from interpreting an InfixExpr where we didn’t intend one:

class Printable(output:String) {
  def ! = { print(output); }
  def !! = { println(output); }
}
class OutputStream {
  def SEEZ(output: String) : Printable = new Printable(output)
}
val U = new OutputStream

U SEEZ "Hello"!

U SEEZ " World"!!

U SEEZ "Goodbye"!

U SEEZ " World"!!

The result on my machine:

tcalder:bin tcalder $ scala ScaLol2.scala
Hello World
Goodbye World

That’s all for this week! I hope today’s post proved interesting, it was far deeper than I ever expected to go, which was exactly what I was hoping for.


facebooktwitterdiggdzonestumbleuponredditdelicious

Advertisements

ScaLearning 5 – DSL Exploration & Operation Precedence

facebookdiggdzonestumbleuponredditdelicious


Like many developers who make the journey from Java to Scala, I often find myself amazed at how much easier it is to do some things, or how much easier it is to express myself in Scala.

“ScaLearning” will be a series of short blog-posts just documenting little tidbits I find interesting, confusing, amusing, or otherwise worthy of talking about.
 
 

DSL Exploration: Our Tale So Far

For those interested in the whole DSL Exploration “mini-series”, we began with DSL Exploration & Operator Notation.

To quickly summarize our tale so far, I decided to explore the creation of a DSL to expose some of Scala’s nuances. My project of choice was codifying the LOLCode language as a Scala DSL. So far we have created a syntax for outputting data to the screen.

Working syntax so far:

U SEEZ "Hello World"

 
 

Learned Lesson 2: Operation Precedence

Now that we’ve successfully defined “U SEEZ”, the LOLCode spec allows us to add a bang at the end of a sentence to “cancel” the newline. While this feels a little backwards, we can attempt it.

Our goal:

U SEEZ "Hello World" // Includes newline
U SEEZ "Goodbye"! // No newline

My first plan of attack was to make “SEEZ” return an object that has the “!” method, which would do a “print” instead of a “println”. Fortunately, before attempting this solution I realized that if I were to do so there would be no method call to trigger the non-bang “println”.

Attempt two, maybe we can allow SEEZ to accept a different object. That object can have a “!” method. Here’s what I’m aiming for:

U.SEEZ( "Hello World".!() )

 
We’ll also need an implicit to turn Strings into this special new object. Here goes.

scala> class Printable(val text : String, val bang : Boolean) {
     | override def toString() = if (bang) {text} else {text + "\n"}
     | def ! = new Printable(text, !bang)                           
     | }                                                            
defined class Printable

scala> class OutputStream {
     | def SEEZ(p : Printable) = print(p.toString())
     | }
defined class OutputStream

scala> val U = new OutputStream
U: OutputStream = OutputStream@e208506

scala> implicit def stringToPrintable(s : String) : Printable = new Printable(s, false)
stringToPrintable: (s: String)Printable

scala> U SEEZ "Hello World"
Hello World

That’s good news, we’ve managed to maintain the old behaviour, so let’s give our new “!” method a try:

scala> U SEEZ "Hello World"!
:10: error: value ! is not a member of Unit

Foiled again! So what happened here? As “println” returns type “Unit”, which is the Scala equivalent to “void”, we can venture a guess that the “!” method was actually run on the result of “println”:

U.SEEZ("Hello World").!()

 
As I’m sure you’ve guessed, the issue at hand is operator precedence. Let’s experiment a little:

scala> class MyObj(val v : String) {                                                  
     | def in(m : MyObj) = {println("[" + v + ".in(" + m.v + ")]"); new MyObj(v + m.v) }
     | def post() = {println("[" + v + ".post()]"); this}
     | def unary_+ = {println("[" + v + ".pre()]"); this}
     | } 
defined class MyObj
   
scala> implicit def stringToMyObj(s : String) : MyObj = new MyObj(s);
stringToMyObj: (s: String)MyObj

scala> +"a"
[a.pre()]
res2: MyObj = MyObj@7987b796

scala> + "a" in "b"
[a.pre()]
[a.in(b)]
res3: MyObj = MyObj@3cd41115

scala> + "a" in "b" post
[a.pre()]
[a.in(b)]
[ab.post()]
res4: MyObj = MyObj@6dcc55fb

scala> "a" in "b" in "c"
[a.in(b)]
[ab.in(c)]
res5: MyObj = MyObj@69cad977

scala> "a" in "b" post
[a.in(b)]
[ab.post()]
res6: MyObj = MyObj@37b24706

scala> "a" post
[a.post()]
res7: MyObj = MyObj@7ea4b9da

scala> "a".post in "b"
[a.post()]
[a.in(b)]
res8: MyObj = MyObj@7e28388b

scala> "a" post in "b"
:1: error: ';' expected but string literal found.
       "a" post in "b"
                   ^

 
 
We seem to be allowed to omit all the dots and brackets when we chain infix operators, but we’re unable to chain an infix operator to the result of a postfix operator unless the postfix used a dot to call. The postfix call at the end of a chain also seems to apply to the result of the previous infix call.

Note: By using the ‘dot’ in [“a”.post in “b”] we are no longer using an ‘operator’. Explicit method calls are referred to as “function applications” in the reference

According to the Scala Reference:

  • If there are several infix operations, operators with higher precedence bind more closely than operators with lower precedence
  • If there are consecutive infix operations with operators of the same precedence, all operators must have the same associativity
  • Postfix operators always have lower precedence than infix operators

While prefix operators aren’t mentioned, they appear to have a higher precedence than infix operators via experimentation:

scala> class MyObj(val v : String) {
     | def +(m : MyObj)  = {println("[" + v + ".in(" + m.v + ")]"); new MyObj(v + m.v) }
     | def unary_+ = {println("[" + v + ".pre()]"); this}
     | }
defined class MyObj
   
scala> implicit def stringToMyObj(s : String) : MyObj = new MyObj(s);
stringToMyObj: (s: String)MyObj

scala> +"a" + +"b"
[a.pre()]
[b.pre()]
[a.in(b)]
res21: MyObj = MyObj@144f1ada

The end result of this discussion is another compromise. After experimenting with various notations, I gave up and returned to my original plan of a “!” method on an object returned by “SEEZ”:

scala> class Printable(output:String) {
     | def ! = { print(output); }
     | def !! = { println(output); }
     | }
defined class Printable

scala> class OutputStream {
     | def SEEZ(output: String) : Printable = new Printable(output)
     | }
defined class Printable

scala> val U = new OutputStream
U: OutputStream = OutputStream@608d116e

scala> U SEEZ "Hello World"!
Hello World
scala> U SEEZ "Hello World"!!
Hello World

scala>

Stay tuned, next time we’ll attempt to put the pieces together and run HelloLol.scala


facebookdiggdzonestumbleuponredditdelicious

ScaLearning 4 – DSL Exploration & Operator Notation

facebooktwitterdiggdzonestumbleuponredditdelicious


Like many developers who make the journey from Java to Scala, I often find myself amazed at how much easier it is to do some things, or how much easier it is to express myself in Scala.
 
“ScaLearning” will be a series of short blog-posts just documenting little tidbits I find interesting, confusing, amusing, or otherwise worthy of talking about.
 
 

Why make a DSL?

I’ve heard it said that the real meat of learning Scala is done by people who want to expose a DSL. So far I’ve found learning Scala fairly easy. Sometimes I’m a little unclear on rules, like when I can leave out brackets and when I can’t, but when that happens I can just re-add the brackets and carry on.
 
The same flexibility isn’t quite so true for people who wish to create a good DSL. A great deal of effort goes into carefully constructing the syntax. Scala is such an incredibly extensible language that, with a little patience, you can create code that doesn’t look at all like “Scala”, and yet still compiles.
 
In order to get a taste of this learning, I decided I was going to attempt a DSL.
 
 

Starting into DSL Exploration

I decided my DSL of choice was going to be LOLCode.
 
Based on the (priceless) LolCats meme, LolCode presented itself as the perfect project to build a DSL around:

  • Some rigidity in sticking close to the community consensus on syntax
  • Enough flexibility to allow me to explore a few different approaches
  • Different enough from Scala to present a challenge in implementation
  • There’s a specific “feel” for the DSL we can aim for
  •  
     

    Learned Lesson 1: Operator Notation

    I decided to start where everyone starts, “Hello World”. System output is one of the most basic pieces of functionality for a language. The spec lists “VISIBLE” as the keyword for println:
    VISIBLE "Hello World" // Prints hello world with a newline
    VISIBLE "Goodbye"! // Prints goodbye, but with no newline

     
    This seemed easy enough. We’ll start by ignoring the bang-syntax:
    scala> def VISIBLE(text : String) = {println(text)}
    VISIBLE: (text: String)Unit
     
    scala> VISIBLE "Hello World"
    :1: error: ';' expected but string literal found.
           VISIBLE "Hello World"

     
    Uh-oh.
     
    Just starting in, we’ve already ran into a lesson on operator syntax. There are three kinds of Operator Notation:

    • Unary (prefix) Notation
      • Only works for +, -, !, and ~
      • Method must be no-args
      • Must omit the dot – ie: !.true would not work, but !true would
    • Postfix Notation
      • Method must be no-args
      • May omit the dot and brackets if desired, but not required.
    • Infix Notation
      • Method must be single-argument
      • May omit the dot and brackets if desired, but not required. ie: “a” + “b” or “a”.+(“b”)

     
    In my case, “VISIBLE” would be a good Unary operator on some string-like object, but it is not a valid unary operator. For it to be an infix operator, I need some other “object” for it to be Infix of. Looks like I’ll have to compromise on this one.
     
    Reading around the site, I discovered that while the spec listed “VISIBLE” as the “println” equivalent, “U SEEZ” was the community’s preference.
     
    U SEEZ "Hello World"
    U SEEZ "Hello World"!

     
    This is convenient, as commenters on LolCode pointed out, as “U” can be seen as the I/O similar to Java’s “System.out”, and “SEEZ” is simply a “print” method on any output stream.
     
    So let’s wrap up today’s post with one last crack at the non-bang notation:
    scala> class OutputStream {
         | def SEEZ(output: String) = println(output);
         | }
    defined class OutputStream
     
    scala> val U = new OutputStream
    U: OutputStream = OutputStream@4be03c55
     
    scala> U SEEZ "Hello World"
    Hello World

     
    Worked like a charm. Stay tuned, as I continue to explore the process behind creating a DSL, and what it reveals about the language.


    facebooktwitterdiggdzonestumbleuponredditdelicious