Diversity Creates Diversity

A few weeks ago I listened to Elizabeth Sampat talk about Women in the video game industry at GDC. Her talk, Women Don’t Want to Work in Games (And Other Myths), was very good. I found myself listening along and nodding as she spoke… Until Myth #3.

I didn’t disagree with what she was saying – actually, that would be impossible. All she was doing was stating what women in the industry felt according to her survey. Not her opinion but measured fact, which is pretty tough to argue with.

(And have anyone take you seriously, anyway.)

So what was it about what she said in Myth #3 that didn’t sit well with me?

Simple. I like “crushing some code” from time to time and describing it that way. Sometimes I have to design interactions between system components, and I feel almost artistic when I get elegant results. Other times I have some tasks that I need to get done and I breeze through them. When my boss asks me how the day went, I crushed it.

That’s how I feel. I’m not a “brogrammer”, but it’s the right word. If I had to I could say I “did amazing today”, but something about the emotional punctuation of “crushing” it feels right to me.

Eureka, a Solution!

Today I read a post by Doug Bradbury titled, Gender Equality Is Not Enough. What he wrote helped me realize something.

The talk didn’t sit right with me because Elizabeth was taking away a piece of my identity. I could see how the aggressiveness of that word wouldn’t suit everyone, but it suits me some days, so why couldn’t she let me have it?

After reading Doug’s post, I realized the answer. Our differences are why diversity is important.

She can let me have my word!
I can choose to use my word, and she can choose not to, and we could work together.
She could “do amazing today”, and I could “crush it”, and we can laugh and both go home.

Honestly, that sounds nice. I think I would enjoy that. I should say so, maybe she would agree?

And then I watched again

And I realized… she never tried to take my word.

When I listen closely a second time, with Doug’s post fresh in mind, I realized I was guilty of something I’ve criticized others for recently. I was being defensive, and I hadn’t even realized it.

When she talked, she said the women wanted:

• A family friendly, family welcoming atmosphere
• Flexibility, with ideas like “core hours” to accommodate schedules
• An environment with the ability to feel ownership
• Working hand in hand with peers who understand what each person brings to the table
• Communication with professionalism and respect
• “I want a company that prides itself on being well-rounded.”

That… actually sounds amazing. I mean, I knew it at the time, but I also thought she was trying to say that my identity wasn’t welcome to work with her.

Had I honestly missed what that bolded point meant?

My reaction to Elizabeth’s talk was to think she was asking me to be the same as her, to “soften” myself beyond recognition into homogeneity to make room for her. She wasn’t. She wants well-rounded teams full of diversity.

Sure, she did ask that the industry be gentler. It should be. No, it has to be, because it’s not welcoming right now.

It’s not okay for to judge someone for not “crushing” it today. It’s not okay to say she’s “not a good fit” because a talented woman doesn’t also like drinking beers and playing Call of Duty with the team after-hours. Honestly, I never would have made that claim anyway, but some people would.

No, as an industry we need to open up to diversity. We need to accept the differences in every programmer, male and female, and see that our teams get better the more we embrace diversity.

To answer Doug’s question:

Why?

Diversity creates diversity.

Because a team of clones will all design the same solution. A team of all “hackers” or all “brogrammers” or men or women – the more homogenous the team, the less likely we are to come up with diverse solutions.

Diverse solutions can be combined into ideas that no single individual would have thought of.

Women bring different perspectives, ones I can’t have. They bring different approaches and values that I don’t.

Diversity creates better solutions to harder problems, not because we’re “equal” in the sense that we’re somehow the “same”, but because we’re different and we choose to place ourselves willingly onto an equal playing field and work together respectfully and professionally.

*Note: I work in the commercial software industry, not video games. Hopefully it doesn’t need to be said, but our industries are very similar and suffer many of the same problems. If you reacted similar to me when you first listened to her talk, I highly recommend listening again with fresh perspective.

Double Entry Programming

 

In high school I remember my first encounter with double entry accounting. It was tedious extra work with no real benefit. On my first assignment I “lost” nearly half my money.

If a professional accountant makes a similar mistake in the name of efficiency, the losses could reach into or past the millions.

In a software program, even a simple bug transcribing data between two variables can cause a catastrophic failure. The losses can range from lost time to lost customers, from hundreds of dollars into or past millions of dollars.

If a programmer could use a practice similar to double entry accounting to prevent such an error, would it be professional of them to choose not to because it feels too “tedious”?

Keep that in mind the next time you decide that simple function isn’t worth testing.

My Experience with Play! Scala

A few months ago I experimented with the Play! framework, which at the time was only available in Java. I had originally intended to do a blog post on Play! but never got around to it.
 
In summary, it wasn’t good. The Java version of Play! actually uses an internal compiler which adds magic to the mix. What looked like Java was actually a close relative to Java using nearly-Java syntax with a few special properties. While I had other issues as well, that property alone was an instant deal-breaker. Use Java or don’t, doing it half-way is a no go.
 
So when I heard that Play! Scala was gaining popularity, I was intrigued.
 
 

What was I Looking For?

 
My exploration of Play! Scala was actually prompted by the Product Owner I am currently working with. He recently took up Ruby on Rails with the intent to be able to do rapid prototyping of concepts for customers. The prototypes help him determine the customer’s real wants/needs so that he can communicate them more easily to his teams, and get a better end product. So far the concept works quite well.
 
So why Rails? Well, our code-base is primarily Java, so non-Java code inhibits the temptation to keep a prototype. Rails also allows very fast prototyping. You can be doing basic CRUD moments after the rubber hits the road, allowing prototypes to even have limited functionality to more fully test user’s expectations.
 
However, as I am a strong proponent of Scala’s extensibility and power, I was curious; why Rails?
 
Is it possible for Scala to provide the same ability to build prototypes extremely quickly?
 
 

What I Found

 
Play! Scala has some real strengths:

• Development lifecycle very similar to a dynamic language – edit, and refresh
• Type-safe code without having to declare types often (type inference)
• Type-safe templates based on the Scala language
• Very fast to go from nothing to running

 
Unfortunately, it’s not ready:

• “Automated Testing” is done by clicking a button in the browser. I want CI.
• Had a hard time getting CRUD running for Scala
• Documentation could use some work 😦

 
Interestingly, CRUD likely would have worked, however I had already tore out the ScalaQuery JDBC implementation in favor of my preferred Squeryl.
 
A more thorough tutorial/example would have likely got my CRUD working before I attempted something like Squeryl.
 
In fact, a good start-to-finish tutorial application (like a basic Blog), complete with CRUD and authentication, could probably mature the basic “rails” (pun totally intended) of Play! Scala to the point that it could directly compete with Ruby on Rails for rapid prototypes.
 
Of course, for scalability Scala has many other options that are almost as fast; I’ll take Lift any day, but I’ve also heard amazing things about Scalatra, Spray, and a few other frameworks.
 
 

Final Thoughts

More/Better Tutorials.
 
Maybe integrate into SBT instead of having a separate executable if Play! Scala is “true” Scala.
 
Deliverable WARs instead of Play Apps to take advantage of a more well-known deployment lifecycle.
 
Easier hooks into testing, with the intent to enable automation for CI.
 
Lots of potential here. Given a little more maturity, Play! Scala may become a real option in my mind. For now, it’s a very interesting project that just has a little further to go.

Resisting Progress

Summarized in an Image

 
 

Why fly when a car will do?

Don’t get me wrong, I drive regularly. Occasionally I even take long trips by the pavement when the tarmac is too pricey. However, there’s a limit. Flying is faster, safer, and more convenient. When cost (both price and time-overhead) allows, I’d prefer to be in the air.
 
In a way, that’s where the comparison breaks down. While Maven is generally faster, safer, and far more convenient than Ant, nobody is charging you exorbitant prices to use it. Both are free, it’s up to you to pick which.
 
Unlike real air travel, Maven also avoids long up-front overhead costs. To fly for real I have to head to the airport early, get through security, sit on my hands for an hour, then jump on the plane and wait another twenty minutes before takeoff. Maven doesn’t have that. I’m running in about a half-dozen lines, and a couple directories.
 
Amusingly enough, I’m still talking about Maven and Ant here. I’ve personally moved on to SBT and I’m loving it. One quick command-line call, and I’ve got my project rolling. I can customize where I want to whatever extent I want because I get to use real code to do so. For me, it’s like moving from the incredible velocity of a jet plane, and strapping myself inside a rocket bound for wherever I want to go.
 
Still, I’m baffled every now and then when I look back down on earth to see someone driving their car because “it’s the way it’s always been done”.

And then I realized… Java is Frustrating!

---

Lately I’ve been slowly learning the Scala language in my spare time. It’s an interesting experience, and it’s incredibly exciting to learn while hearing buzz such as the rumour that Quora is moving to Scala. With other major players using the language, such as Twitter, LinkedIn, and Foursquare (just to repeat @yaaang’s non-exhaustive list), its keeps learning relevant and exciting.

Unfortunately, it has a few drawbacks.

I was introduced to Scala through Mike Nash, a coworker of mine who dove into the language first. While learning, I would regularly poke fun at his occasional Scala-isms while he tried to write Java code. For example:

public class ServiceTest {
    @Test
    public void testSummation() {
        List<Integer> intsToSum = List(1, 2, 3);  // Uh... that's not Java mike...
        int sum = service.sum(intsToSum);
        assertEquals(6, sum);

        // At least he didn't intsToSum.reduceLeft(_+_) ;P
    }
}

Even as I slowly slipped into the same habit, I would still tease, but today I realized that the real frustration isn’t the occasional Scala-ism. The biggest pain-point I’ve found so far in learning Scala is having Scala in my personal toolkit, but not having it in our team’s toolkit.

Scala is a JVM language. It would compile into the WAR-file and be indistinguishable to our customer. It’s statically typed (which I like), but still uses type-inference to avoid the verbosity of most static-type languages. It’s extremely expressive, complete with the power of the functional and the object oriented paradigms. It’s all those things, and more, and I can’t use it.

case class ProductFamilyCategoryWithChildren(category : ProductFamilyCategory, families : List[ProductFamily]);

public class ProductFamilyCategoryWithChildren {
    private ProductFamilyCategory category;
    private List<ProductFamily> families;

    public ProductFamilyCategoryWithChildren(ProductFamilyCategory category, List<ProductFamily> families) {
        this.category = category;
        this.families = families;
    }

    public ProductFamilyCategory getCategory() { return category; }
    public List<ProductFamily> getFamilies() { return families; }
}

That’s 1 line of Scala to 12 of Java. There are 111 characters of Scala to 456 of Java. I stare at both, swallow my pride a little, delete the Scala version, and move on…

---

ScaLearning 7 – Distributing Concurrent Tests

---

Like many developers who make the journey from Java to Scala, I often find myself amazed at how much easier it is to do some things, or how much easier it is to express myself in Scala.

“ScaLearning” will be a series of short blog-posts just documenting little tidbits I find interesting, confusing, amusing, or otherwise worthy of talking about.
 
 

Motivation

Recently in order to gain confidence in our web application, our team decided it pragmatic to run a series of tests over a deployed version of the application complete with production-like database. A simple suite of non-destructive tests we could run in any environment was quickly put together.
 
Unfortunately one of our simplest tests quickly began causing us trouble. The test emulated a search spider, crawling every link it found on the entire site in an exhaustive graph traversal complete with cycle-detection. Unfortunately, the test ran for over 24 hours without completing.
 
While we’ve made other optimizations to improve performance of the test (such as excluding sufficiently similar pages), the topic of today’s post is the concurrency we introduced in order to help take the edge off test time.
 
 

Distributing Concurrent Tests

Our goal was relatively simple. We wished to run a very simple test across many thousands of URLs:

def testSingleUrl(nextUrl : String) = {
      client.open(nextUrl)
      client.statusCode should equal(200)
}

 
 
If either of the calls inside this test fail, an exception would be thrown. This was an acceptable way of detecting test failure. However, the test needed to do a bit more – traverse the graph:

  def testUrl(nextUrl: String): Unit = {
      if (wasVisited(nextUrl)) return

      val client = new HtmlUnitRunner
      client.open(nextUrl)
      client.statusCode should equal(200)

      registerVisitedUrl(nextUrl)
      
      JListWrapper(client.currentPage.getAnchors())
              .map(_.getAttribute("href"))
              .filter(isLinkValid(_))
              .foreach((a: String) => {
                      markForTesting(nextUrl)
              })
  }

Of course, this was called by a method which pushed and popped from a stack, and “markForTesting” pushed a new link onto that stack. This code worked great sequentially, but we wanted it to operate concurrently in order to minimize testing time. For this, we employed Akka’s actors:

class ConcurrentTest extends Actor {
  def receive = {
    case name: (() => Unit) =>
      try {
        name()
        self reply(true)
      } catch {
        case t: Throwable =>
          t.printStackTrace()
          self reply(false)
      }
    case _ =>
      println("Actor received an unrecognized message")
      self reply(false)
  }
}

ConcurrentTest, as you can see, is the driver behind an individual run of the test method:

val testRunner = actorOf[ConcurrentTest].start()
val result = testRunner !!! (() => testUrl("/index.html")
// Other code can go here
result.get

 
 
Assuming the methods called within testUrl are thread-safe (which we also did using Actors), this will run a single test using a second Thread, and allow us to continue on with our business. However, since there’s only a single Actor here, we only have one Thread with which to process URLs. This means that we’re still effectively opening each link sequentially.

We need a Thread pool, and Akka is glad to provide one:

class Master(nofWorkers: Int) extends Actor {
  val workers = List.fill(nofWorkers)(actorOf[ConcurrentTest].start())
  val router = Routing.loadBalancerActor(new CyclicIterator(workers)).start()
  val answers = ListBuffer[Future[Boolean]]()

  def receive = {
    case name: (() => Unit) => answers += router !!! name
    case "result" => {
      Futures.awaitAll(answers.toList)
      self reply(answers.foldLeft(true)(_ && _.result.get))
      workers.foreach(_.stop())
      router.stop()
      self stop()
    }
  }
}

 
 
So far everything I’ve presented as code came very naturally. In fact, minor modifications for the purpose of blogging notwithstanding, we used the code I’ve presented so far to test several links across our site very successfully, in a fashion very similar to the following:

val masterRunner = actorOf(new Master(concurrentActors)).start()
val productList = fetchAllProductsFromDatabase

productList.map(productToUrl(_))
        .foreach(url => masterRunner ! (() => testSingleUrl(url)))

((masterRunner !! "result").getOrElse(false).asInstanceOf[Boolean]) should equal(true)

However, when we attempted to apply the same methodology to our crawl test, things didn’t work as well as we’d hoped:

    val masterRunner = actorOf(new Master(concurrentActors)).start()

    // def testUrl as seen above

    def markForTesting(nextUrl : String) {
            masterRunner ! (() => testUrl(nextUrl))
    }

   def fullTest() = {
            val baseUrl = "/index.html"
            masterRunner ! testUrl("baseUrl")

             ((masterRunner !! "result").getOrElse(false).asInstanceOf[Boolean]) should equal(true)
    }

 
 
The theory was that “result” would wait for all of the answers to come back before returning. Unfortunately, that’s not quite the sequence of events the Actor sees. In reality, after digging, we figured out what messages the actor received:

1. (() => testUrl(“/index.html”)) occurs, which is quickly sent to a ConcurrentTest runner
2. “result” comes next, as it takes a second or two for the runner to open the test
3. (() => testUrl(_)) is received for several other URLs as links are scraped off the first page

“result” doesn’t actually wait for all the answers to come back, as it has no way of knowing how many answers are actually required. For that matter, we aren’t sure of that number either, as the test is meant to be dynamic. Instead, “result” simply compiles the answers it has so far, and then shuts down all of the actors. This means we get a “yes” or “no” about “/index.html”, but all of the other URLs are still sitting in the mailbox of Master when it’s shut down. Uh-oh!

So how do we know when we’re done? Mailbox sizes. We added a new match case to Master which would calculate if it believed the tests to be done yet:

class Master(nofWorkers: Int) extends Actor {
  ...
  def receive = {
    ...
    case "done" => { // Added this case, the rest of "Master" remained unchanged
        Futures.awaitAll(answers.toList)
        val overallSize = workers.map(worker => worker.mailboxSize).foldLeft(0)(_ + _) + router.mailboxSize
        self.reply(overallSize == 0 && self.mailboxSize == 0)
    }
  }
}

 
 
This code is different than “result” in that it actually attempts to detect if the tests are done by:

1. Waiting for all currently outstanding test-methods to complete
2. Counting any pending messages in the router and worker mailboxes.
   (This should always be zero, as we’ve waited for all answers to return, but it’s still safer to be sure)
3. Counting any pending messages on the master
4. Return “true” if the total pending messages is 0, otherwise “false” as more tests have to run

This algorithm will work for us because when we run Futures.awaitAll it runs every outstanding test to completion. Any URLs found on the pages to be tested are checked against previously-visited URLs, and added to Master’s queue if they are new. Since Master is still processing “done” those tests will stay on the queue and “mailboxSize” will return a positive non-zero number. If, however, no new links are encountered, then there will be no tests waiting on the Master queue, and our “done” operation will detect 0 pending tests.

In use:

val masterRunner = actorOf(new Master(concurrentActors)).start()
def markForTesting(nextUrl : String) = masterRunner ! (() => testUrl(nextUrl))
// def testUrl as seen above

def fullTest() = {
    val baseUrl = "/index.html"
    masterRunner ! testUrl("baseUrl")
    while((masterRunner !! "done").getOrElse(false).asInstanceOf[Boolean] == false) {
        Thread.sleep(1000);
    }
    ((masterRunner !! "result").getOrElse(false).asInstanceOf[Boolean]) should equal(true)
}

Now we sleep our thread, asking the master if it’s completed its job once every cycle, until the master claims all of its workers have completed their work and no new work is pending for the master to distribute.

Feedback as to other potential approaches is very welcome, I find the entire topic of concurrency and job distribution very interesting.

---

ScaLearning 6 – DSL Exploration & Operator Interpretation

---

Like many developers who make the journey from Java to Scala, I often find myself amazed at how much easier it is to do some things, or how much easier it is to express myself in Scala.

“ScaLearning” will be a series of short blog-posts just documenting little tidbits I find interesting, confusing, amusing, or otherwise worthy of talking about.
 
 

DSL Exploration: Our Tale So Far

For those interested in the whole DSL Exploration “mini-series”, we began with DSL Exploration & Operator Notation.

To quickly summarize our tale so far, I decided to explore the creation of a DSL to expose some of Scala’s nuances. My project of choice was codifying the LOLCode language as a Scala DSL. So far we have created a syntax for outputting data to the screen.

Working syntax so far:

U SEEZ "Hello World" !
U SEEZ "Hello World" !!

 
 

Learned Lesson 3: Operation Interpretation

So far we’ve been doing all of our work in the Scala REPL. While extremely powerful, the REPL does have a few known limitations – it sometimes interprets a line differently than it would have been interpreted from a source file. Today we’ll discover one such situation, and explore the reason.

Let us start with a Scala source-file which will act as our basis:

class Printable(output:String) {
  var bang = false;
  def ! = { print(output); }
  def !! = { println(output); }
}
class OutputStream {
  def SEEZ(output: Printable) : Unit = println(output.toString)
}
val U = new OutputStream

U SEEZ "Hello"!
U SEEZ " World"!!
U SEEZ "Goodbye"!
U SEEZ " World"!!

The code above is equivalent to what we were writing into the REPL in previous posts, but let’s store that in a file ScaLol.scala and run it:

tcalder:bin tcalder $ scala ScaLol.scala
/.../ScaLol.scala:10: error: value ! is not a member of java.lang.String
U SEEZ "Hello"!

Clearly not what the REPL was telling us. At first glance, it now appears as if it’s completely changed its operational precedence, and that’s true… but only sort of.

After trying various combinations of the code (none of which helped), I eventually gave up and started reading the Scala Reference again.

Let’s review what we’ve said so far:

• Methods with one argument can be used as an infix operator
• Methods with no arguments can be used as a postfix operator
• Unary operators for !, +, -, and ~ can be used if equivalent “unary_*” methods are defined
• Postfix operators always have lower precedence than infix operators
• Infix operators have an order of precedence which includes letters being lower than special characters

Reasoning briefly about our code, the REPL clearly evaluates the “!” as a postfix operator, which gives it the lowest precedence and results in:

U.SEEZ("Hello").!()

Yet in ScaLol.scala we appear to have attempted to run “!” on String. Since Postfix have low precedence, we just be attempting to evaluate “!” as an Infix operator. This would make sense since special characters have a higher operational precedence:

U.SEEZ( "Hello World".!(.....?) )

Now if you’re like me, you’re wondering what on earth it could possibly be trying to fill in as the argument to this strange new Infix “!”. The answer is, it’s trying to use the next line as a continuation on the current:

U.SEEZ("Hello".!(U)).SEEZ("World"... // Or some rough equivalent

From the ScalaReference:

Scala is a line-oriented language where statements may be terminated by semicolons or newlines. A newline in a Scala source text is treated as the special token
“nl” if the three following criteria are satisfied:

1. The token immediately preceding the newline can terminate a statement.
2. The token immediately following the newline can begin a statement.
3. The token appears in a region where newlines are enabled.

“U” can start a statement, and “!” can terminate one, since the REPL clearly didn’t get upset at either of these. Newlines are enabled in our code – there’s a lot of logic in the Reference that explains why if you’d like to explore it.

Based on this, we can conclude that our “\n” after the “!” is, in fact, a “[nl]”.

The second piece of the puzzle that we need is the following (summarized) syntactic definitions, expressed in the Scala Ref using Extended Backus–Naur Form:

PostfixExpr ::= InfixExpr [id [nl]]
InfixExpr ::= PrefixExpr
                     | InfixExpr id [nl] InfixExpr
PrefixExpr ::= [‘-’ | ‘+’ | ‘~’ | ‘!’] SimpleExpr

In specific, draw your attention to the similarity between the definition of “PostfixExpr” and the second definition of “InfixExpr”, and pay special attention to our friend “nl”.

The following is a (rough) breakdown of what we’ve actually attempted:

U SEEZ "Hello World" !
U SEEZ "Goodbye World" !

SimpleExpr id SimpleExpr id nl SimpleExpr id SimpleExpr id

PrefixExpr id PrefixExpr id nl PrefixExpr id PrefixExpr id

InfixExpr id InfixExpr id nl InfixExpr id InfixExpr id

Bingo, there’s our culprit. The second “id” in the expression is our “!” operator. Based on the formal specification, “InfixExpr id nl InfixExpr” can be simplified as a single “InfixExpr”, and since “!” has higher precedence than “SEEZ”, it will clearly be evaluated first.

As a final proof, try saving the following code as “ScaLol2.scala” and running it. Note how we use two new-lines in-between “SEEZ” calls. This results in two “nl” entities instead of one, which prevents the compiler from interpreting an InfixExpr where we didn’t intend one:

class Printable(output:String) {
  def ! = { print(output); }
  def !! = { println(output); }
}
class OutputStream {
  def SEEZ(output: String) : Printable = new Printable(output)
}
val U = new OutputStream

U SEEZ "Hello"!

U SEEZ " World"!!

U SEEZ "Goodbye"!

U SEEZ " World"!!

The result on my machine:

tcalder:bin tcalder $ scala ScaLol2.scala
Hello World
Goodbye World

That’s all for this week! I hope today’s post proved interesting, it was far deeper than I ever expected to go, which was exactly what I was hoping for.

---

ScaLearning 5 – DSL Exploration & Operation Precedence

---

Like many developers who make the journey from Java to Scala, I often find myself amazed at how much easier it is to do some things, or how much easier it is to express myself in Scala.

“ScaLearning” will be a series of short blog-posts just documenting little tidbits I find interesting, confusing, amusing, or otherwise worthy of talking about.
 
 

DSL Exploration: Our Tale So Far

For those interested in the whole DSL Exploration “mini-series”, we began with DSL Exploration & Operator Notation.

To quickly summarize our tale so far, I decided to explore the creation of a DSL to expose some of Scala’s nuances. My project of choice was codifying the LOLCode language as a Scala DSL. So far we have created a syntax for outputting data to the screen.

Working syntax so far:

U SEEZ "Hello World"

 
 

Learned Lesson 2: Operation Precedence

Now that we’ve successfully defined “U SEEZ”, the LOLCode spec allows us to add a bang at the end of a sentence to “cancel” the newline. While this feels a little backwards, we can attempt it.

Our goal:

U SEEZ "Hello World" // Includes newline
U SEEZ "Goodbye"! // No newline

My first plan of attack was to make “SEEZ” return an object that has the “!” method, which would do a “print” instead of a “println”. Fortunately, before attempting this solution I realized that if I were to do so there would be no method call to trigger the non-bang “println”.

Attempt two, maybe we can allow SEEZ to accept a different object. That object can have a “!” method. Here’s what I’m aiming for:

U.SEEZ( "Hello World".!() )

 
We’ll also need an implicit to turn Strings into this special new object. Here goes.

scala> class Printable(val text : String, val bang : Boolean) {
     | override def toString() = if (bang) {text} else {text + "\n"}
     | def ! = new Printable(text, !bang)                           
     | }                                                            
defined class Printable

scala> class OutputStream {
     | def SEEZ(p : Printable) = print(p.toString())
     | }
defined class OutputStream

scala> val U = new OutputStream
U: OutputStream = OutputStream@e208506

scala> implicit def stringToPrintable(s : String) : Printable = new Printable(s, false)
stringToPrintable: (s: String)Printable

scala> U SEEZ "Hello World"
Hello World

That’s good news, we’ve managed to maintain the old behaviour, so let’s give our new “!” method a try:

scala> U SEEZ "Hello World"!
:10: error: value ! is not a member of Unit

Foiled again! So what happened here? As “println” returns type “Unit”, which is the Scala equivalent to “void”, we can venture a guess that the “!” method was actually run on the result of “println”:

U.SEEZ("Hello World").!()

 
As I’m sure you’ve guessed, the issue at hand is operator precedence. Let’s experiment a little:

scala> class MyObj(val v : String) {                                                  
     | def in(m : MyObj) = {println("[" + v + ".in(" + m.v + ")]"); new MyObj(v + m.v) }
     | def post() = {println("[" + v + ".post()]"); this}
     | def unary_+ = {println("[" + v + ".pre()]"); this}
     | } 
defined class MyObj
   
scala> implicit def stringToMyObj(s : String) : MyObj = new MyObj(s);
stringToMyObj: (s: String)MyObj

scala> +"a"
[a.pre()]
res2: MyObj = MyObj@7987b796

scala> + "a" in "b"
[a.pre()]
[a.in(b)]
res3: MyObj = MyObj@3cd41115

scala> + "a" in "b" post
[a.pre()]
[a.in(b)]
[ab.post()]
res4: MyObj = MyObj@6dcc55fb

scala> "a" in "b" in "c"
[a.in(b)]
[ab.in(c)]
res5: MyObj = MyObj@69cad977

scala> "a" in "b" post
[a.in(b)]
[ab.post()]
res6: MyObj = MyObj@37b24706

scala> "a" post
[a.post()]
res7: MyObj = MyObj@7ea4b9da

scala> "a".post in "b"
[a.post()]
[a.in(b)]
res8: MyObj = MyObj@7e28388b

scala> "a" post in "b"
:1: error: ';' expected but string literal found.
       "a" post in "b"
                   ^

 
 
We seem to be allowed to omit all the dots and brackets when we chain infix operators, but we’re unable to chain an infix operator to the result of a postfix operator unless the postfix used a dot to call. The postfix call at the end of a chain also seems to apply to the result of the previous infix call.

Note: By using the ‘dot’ in [“a”.post in “b”] we are no longer using an ‘operator’. Explicit method calls are referred to as “function applications” in the reference

According to the Scala Reference:

• If there are several infix operations, operators with higher precedence bind more closely than operators with lower precedence
• If there are consecutive infix operations with operators of the same precedence, all operators must have the same associativity
• Postfix operators always have lower precedence than infix operators

While prefix operators aren’t mentioned, they appear to have a higher precedence than infix operators via experimentation:

scala> class MyObj(val v : String) {
     | def +(m : MyObj)  = {println("[" + v + ".in(" + m.v + ")]"); new MyObj(v + m.v) }
     | def unary_+ = {println("[" + v + ".pre()]"); this}
     | }
defined class MyObj
   
scala> implicit def stringToMyObj(s : String) : MyObj = new MyObj(s);
stringToMyObj: (s: String)MyObj

scala> +"a" + +"b"
[a.pre()]
[b.pre()]
[a.in(b)]
res21: MyObj = MyObj@144f1ada

The end result of this discussion is another compromise. After experimenting with various notations, I gave up and returned to my original plan of a “!” method on an object returned by “SEEZ”:

scala> class Printable(output:String) {
     | def ! = { print(output); }
     | def !! = { println(output); }
     | }
defined class Printable

scala> class OutputStream {
     | def SEEZ(output: String) : Printable = new Printable(output)
     | }
defined class Printable

scala> val U = new OutputStream
U: OutputStream = OutputStream@608d116e

scala> U SEEZ "Hello World"!
Hello World
scala> U SEEZ "Hello World"!!
Hello World

scala>

Stay tuned, next time we’ll attempt to put the pieces together and run HelloLol.scala

---

ScaLearning 4 – DSL Exploration & Operator Notation

---

Like many developers who make the journey from Java to Scala, I often find myself amazed at how much easier it is to do some things, or how much easier it is to express myself in Scala.
 
“ScaLearning” will be a series of short blog-posts just documenting little tidbits I find interesting, confusing, amusing, or otherwise worthy of talking about.
 
 

Why make a DSL?

I’ve heard it said that the real meat of learning Scala is done by people who want to expose a DSL. So far I’ve found learning Scala fairly easy. Sometimes I’m a little unclear on rules, like when I can leave out brackets and when I can’t, but when that happens I can just re-add the brackets and carry on.
 
The same flexibility isn’t quite so true for people who wish to create a good DSL. A great deal of effort goes into carefully constructing the syntax. Scala is such an incredibly extensible language that, with a little patience, you can create code that doesn’t look at all like “Scala”, and yet still compiles.
 
In order to get a taste of this learning, I decided I was going to attempt a DSL.
 
 

Starting into DSL Exploration

I decided my DSL of choice was going to be LOLCode.
 
Based on the (priceless) LolCats meme, LolCode presented itself as the perfect project to build a DSL around:

• Some rigidity in sticking close to the community consensus on syntax
• Enough flexibility to allow me to explore a few different approaches
• Different enough from Scala to present a challenge in implementation
• There’s a specific “feel” for the DSL we can aim for

 
 

Learned Lesson 1: Operator Notation

I decided to start where everyone starts, “Hello World”. System output is one of the most basic pieces of functionality for a language. The spec lists “VISIBLE” as the keyword for println:
VISIBLE "Hello World" // Prints hello world with a newline
VISIBLE "Goodbye"! // Prints goodbye, but with no newline

 
This seemed easy enough. We’ll start by ignoring the bang-syntax:
scala> def VISIBLE(text : String) = {println(text)}
VISIBLE: (text: String)Unit
 
scala> VISIBLE "Hello World"
:1: error: ';' expected but string literal found.
       VISIBLE "Hello World"

 
Uh-oh.
 
Just starting in, we’ve already ran into a lesson on operator syntax. There are three kinds of Operator Notation:

• Unary (prefix) Notation
  • Only works for +, -, !, and ~
  • Method must be no-args
  • Must omit the dot – ie: !.true would not work, but !true would
• Postfix Notation
  • Method must be no-args
  • May omit the dot and brackets if desired, but not required.
• Infix Notation
  • Method must be single-argument
  • May omit the dot and brackets if desired, but not required. ie: “a” + “b” or “a”.+(“b”)

 
In my case, “VISIBLE” would be a good Unary operator on some string-like object, but it is not a valid unary operator. For it to be an infix operator, I need some other “object” for it to be Infix of. Looks like I’ll have to compromise on this one.
 
Reading around the site, I discovered that while the spec listed “VISIBLE” as the “println” equivalent, “U SEEZ” was the community’s preference.
 
U SEEZ "Hello World"
U SEEZ "Hello World"!

 
This is convenient, as commenters on LolCode pointed out, as “U” can be seen as the I/O similar to Java’s “System.out”, and “SEEZ” is simply a “print” method on any output stream.
 
So let’s wrap up today’s post with one last crack at the non-bang notation:
scala> class OutputStream {
     | def SEEZ(output: String) = println(output);
     | }
defined class OutputStream
 
scala> val U = new OutputStream
U: OutputStream = OutputStream@4be03c55
 
scala> U SEEZ "Hello World"
Hello World

 
Worked like a charm. Stay tuned, as I continue to explore the process behind creating a DSL, and what it reveals about the language.

---

The Fallibility of Tests

---

Recently I posted about Test Flow and Method Contracts. The key takeaway was that we can use tests to prove the contracts our system lives up to.

We all create a logical representation of our software in our minds in order to reason about it. Tests (either manual, or automatic) allow us to prove the rules of that representation to be true.

Today I’ll talk about what it means when there are mistakes in those tests. For the purposes of this discussion, “tests” may be manual or automatic, but examples are given in code for clarity.
 
 

Test Fallacy vs Test Absence

The first distinction I want to make is between incorrect tests, and missing tests.

Test Fallacy is when a test exists, but creates a logical fallacy within our reasoning system.

For example:

// {new Cache} r = cache.has(x) {r == false}
@Test
public void has_shouldReturnFalse_whenItemHasNotBeenAddedToCache() {
    //setup
    Cache c = new Cache();
 
    //execute
    boolean b = c.has("random");
 
    //assert
    assertTrue(b);
}


Clearly the code disagrees with the test name. Since the name of the test is more likely to correspond to the rule I have in my head that I believe the system follows, this test could be extremely destructive to my system.
 
 
Test Absence is when a test is absent, but we assume behaviour anyways. This is a logical mistake on our part, but it leads to bugs in the system.

I gave an example of this in my post Test Flow and Method Contracts where we “forgot” to define the behaviour of “has” when the cache is empty.

The problem with test absence is that we often assume the behaviour exists, and is “reasonable” by our own subjective measure. That creates a problem – what is “reasonable”?  
 

The Danger of Test Fallibility

While tests are fallible, either through a mistake or through absence, it’s useful to keep in mind why that is dangerous.

A professor for my logic class said something one class that stuck with me:

Logical fallacies don’t just create problems in logical systems, they destroy them. If you consider any false statement to be true, anything can be proven.

Let’s say, for example, that 2 = 3. I’ll prove I’m the Queen of England.

How? Well 3 – 2 = 1, but 3 – 3 = 0. Since 2 and 3 are equal, that implies 1 and 0 are equal.

Furthermore, 2 – 1 = 1, but 2 – 0 = 2. Since 1 and 0 are equal, so too are 1 and 2.

Now the Queen and I are, in fact, two distinct people. But 2 = 1, which means the Queen and I are one. So you see, I am in fact the Queen of England.
(Paraphrased from memory)

Honestly, I probably would have chosen to prove I was Metallica (the entire band), but if one admits that 2 = 3, the rest of the logic is difficult to argue against. With one false “fact”, we destroyed our entire reasoning system.

I’m sure many of us can relate this back to software. Try to recall a time when you received a bug complaint from a user, and after tracking down the bug you stared in awe at your computer screen. “That shouldn’t even be possible…”

The real problem? These “glitches” in our reasoning systems are inevitable. We’re all humans, we will make a mistake. One way or another, you will eventually have a bug in a test, or assume behaviour exists that really doesn’t.
 
 

How do we Address Test Fallibility?

This brings me back around to the original topic – how do we avoid this fallibility?

We automate our test suite.

It seems strange to say automation will “cure” all of our problems reasoning about systems, and that’s because it won’t. It’s simply the best way to combat the problem.

Automation addresses Test Absence by:

• being cheaper (time) than manual testing
• eliminating any reasons to skip tests
• growing as a regression suite over time as we think of new test cases

If you forget a test, simply add it and it will never be forgotten again.

Automation addresses Test Fallacies by:

• being repeatable
• being correctable (you can’t prevent humans from re-making mistakes)
• being easy to read, examine, and reason about

If you make a mistake in a test, simply fix it and the mistake will not reoccur.

In summary, automating your tests doesn’t guarantee that your tests will be good, but it allows you to confidently improve your tests over time. Good test allow you to reason correctly about your system.

---