• Home
  • Events
  • About
  • Slimmed Down Software – A Lean, Groovy Approach Part 3 – Create Knowledge

    August 12th, 2010

    This article originally appeared in the June 2010 edition of GroovyMag, the Groovy and Grails magazine. Parts 4 and 5 are currently available for download from the magazine’s site, and more will come each month. Enjoy!

    The Groovy Programming Language advertises itself as an “agile and dynamic Language for the JVM”, but what does this mean exactly? This series of articles explains Lean Software Development, and shows how your choice of programming language can make your entire process remain nimble and adaptive. Each months article will cover one of the seven Lean Software Development principles and explain how Groovy and the associated ecosystem help eliminate waste, defer commitment, and build quality into your product.

    About this Series
    Groovy is an agile programming language. In order to explain what this means, these articles are structured around the seven principles of Lean Software Development. The previous installments included a short introduction to Lean, explained how an expressive language can eliminate waste in unit testing, and showed how the easyb Behavior Driven Development framework and Groovy Builders can build quality into your life-cycle. In later months we’ll explore how a dynamic language lets you minimize coupling and defer commitment, and how metaprogamming allows you to deliver fast. You’ll see how Groovy’s Grapes module system, alongside modules like Groovy Web Services, streamline interactions between development and other groups like operations and QA (an important aspect of respecting people). Finally, we’ll look at optimizing the whole by applying Groovy to domains that cross department boundaries: how Gradle improves the build process and easyb serves as a collaboration mechanism between different organization roles. The series will end with a discussion on the best practices of mixing Java and Groovy.

    Some of the code examples are basic while others are advanced. The articles explore why the features of Groovy are important rather than the mechanics of any one Groovy feature. Hopefully you’ll leave with some new ideas about how to use Groovy, how to convince your team that Groovy is worthwhile, or most importantly, how to increase your productivity.

    Lean Revisited
    Part 1 of this series contained a more indepth explanation of Lean, and I won’t repeat it here. But as a quick refresher here are the 7 lean principles along with a summary for those principles covered previously:
    1. Eliminate Waste – Reduce work in progress and half done work
    2. Build Quality In – Instead of settling for finding defects, optimize your process to create fewer defects
    3. Create Knowledge
    4. Defer commitment
    5. Deliver Fast
    6. Respect People
    7. Optimize the Whole

    Principle 3 – Create Knowledge

    I prefer the term “Software Engineer” to “Programmer” because it evokes a sense of professionalism, adherence to standards, and attention to quality that sometimes seems lacking in our industry. But “Engineer” also carries negative connotations when applied to software. As my co-worker Mike Mannion says, “Software development has less to do with conveyor-belt-fabrication (the image unfortunately most likely to be to be conjured up when we use the word engineering) than knowledge acquisition.” Indeed, Software development is a knowledge creating process. The mostbasic thing we do in our jobs is to discover what needs to be done in order to delight our customers. In fact, your product will benefit by finding ways to speed up this knowledge acquisition. Specifically, Lean provides two practices for this: short, frequent learning cycles and delayed commitment. Delayed commitment, postponing decisions until the last responsible moment, is covered in next month’s article. Today we’ll focus on shorting learning cycles.

    So where does all this created knowledge reside? Large requirements and specification documents are enticing because they provide a wonderful illusion of certainty. They can be agreed upon in meetings, verified with checklists, cross referenced with traceability matrices, and then enforced by lawyers and courts when projects fail. Great software has as much to do with rock solid documentation as a great marriage has to do with a rock solid prenuptial agreement. No thanks, I say.

    The software industry has clearly turned towards executable solutions for knowledge creation and retention, any plausible description for a marriage related “executable solution” leaves me feeling quite uneasy. Executable acceptance test frameworks abound: Fit, Fitnesse, and easyb, and new unit test frameworks appear every few weeks. The code you write, and the tests that accompany it are the most basic persistent form of system knowledge, and it is important that this knowledge be verified in an automated way.

    The problem with traditional unit tests is that the test framework imposes so many particularities that the important knowledge of how your system must behave gets lost in a noise of subclassing, type definitions, and annotations. Wiki and spreadsheet based solutions like Fitnesse are a nice idea in theory, but I have not yet found customers willing to edit test data directly. Instead it is left to the developer to do it. To make traditional unit testing more friendly, some developers try to cut through this noise by using strange naming conventions and commenting their tests with standard comments, as in Listing 1.

    I’ve found word separating underscores in method names to bring no real value. You’re going to have to press the shift key one way or another, whether it be with underscores or camel case. The practice I do like is the arrange-act-assert pattern. This pattern originated with William Wake as a way to add clarifying structure to unit tests. The idea is to codify and standardize the way tests are written: First arrange all necessary preconditions and inputs, then act on the object or method under test, and finally assert that the expected results have occurred. Tests are easier to read because the setup and verification phase is clearly separated from the block exercising the system under test, and test smells like a too busy act phase or missing assert blocks rise quickly to the surface.

    Encoding your tests with comments is an interesting idea, but in the end comments are not executed, grow stale, and become misleading. A better choice than using comments is to pick a testing framework that eliminates all this noise, one that natively supports the structure of unit tests. A testing framework like Spock.

    Spock is an innovative framework, taking new directions on fixture setup, mocking, and data driven tests. The goal of Spock is to allow the developer to focus solely on the input and output of tests and then get out of the way. One type of test available in Spock is the data driven test, seen in Listing 2.

    Did you know that question marks and new line characters in methods names, like in Listing 2, are completely legal on the JVM? The Java language just doesn’t allow you to use them! While this is far from the main productivity boost from Spock, it does lead to some very nice looking JUnit reports. Figure 1 is our actual JUnit report from my current project. Seeing a space separated test name sure beats deciphering a camel case or underscore laden test name.

    Spock Test Report

    The value of Spock being built on top of JUnit should be obvious: your reports, builds, IDEs, and other tools continue to work as before. To the tools, Spock is just another unit test.

    There are more interesting questions to ask about Listing 2, however. Reading the listing from top to bottom may be confusing. Are those “==” statements assertions, and how can they be made before the variables have been initialized? What does this test do exactly?

    Spock shows the great power available behind Groovy. Those goto labels (setup, expect, and where) are some of the meaningful tags you are allowed to use within Spock, and these tags are enforced within the compiler. They are not goto labels at all but a vital part of a Spock test specification. Just as test fixtures can have setup and cleanup phases, so can individual test methods. Spock supports this idea natively at the compiler level. In-line comments and funny method names are replaced by first class support for testing.

    Within the expect: phase, the statements including == are indeed assertions. You may write out the assert keyword longhand if you desire, but the point of an expect: block is to run assertions, so Spock lets you drop out this bit of ceremony from your tests.

    Another surprising feature is that the expect block will be executed three times in this test: one time for each row in the dataset. So the first execution is with data user1, John, Doe, and 123456, the second execution receives the second “column” of the dataset and so on. Once you are used to Spock, the data tends to be read in vertical columns, since each column is input to the expect block. For testers weaned on JUnit the test might seem backwards, raising questions like “Why is the data the last piece of the test, shouldn’t that come first?” The short answer is, not if you want to create knowledge about your system. The important part of the system is how it behaves, how the UserService retrieves users. This information is front and center communicating to any future programmers on the project. The data that ensures this behavior comes later, and is usually less important than the assertions. The Arrange, Act, Assert pattern is required in imperative, sequential programming. Luckily, a strong language and an innovative framework let you focus more on knowledge creation than satisfying the semantics of your chosen compiler.

    Data driven tests are just one of the options in Spock specifications. As you explore Spock you’ll find many of the BDD concepts available to use, as well as a unique solution to working effectively with mocking frameworks. Spock is built on JUnit, so it works without effort for popular build tools and IDEs, and the online documentation is superb (how often can you say that about an open source framework?). If you want to take tests as a form of knowledge creation, then Spock is definitely worth investigating.

    The new 0.4 release of Spock contains another option for data driven tests as well: the data table. Previously, Listing 2 showed how lists can be used within Spock so that assertion blocks are automatically called several times with different data. The downside is that I sometimes find myself tilting my head as I try to read the vertical columns embedded in the lists. Similar to the data tables features in Scala’s Specs framework, Spock now allows you to embed pipe delimited data tables in your tests. Listing 3 is the same test as in Listing 2 but uses data tables instead of the Spock data
    driven format.

    The result is a much more readable dataset, neatly divided into rows and columns.

    The test driven community has a great interest in improving the framework landscape. Unit tests have been heralded in the past as a way to capture information about the system being developed, but the JUnit and Java syntax combine to under-deliver on this promise. Finally, with both the Groovy language and frameworks like Spock, we are seeing unit tests come into their own in their ability to capture, express, and execute the workings of the system.

    Sidebar: Expressive vs. Terse
    I’ve been very careful to call Groovy expressive instead of terse, which are two different things. Terse means showing less. Terse can mean anything from an abstraction to an abbreviation. Court stenography or writing shorthand is terse. All of the information is captured, but in as small amount of space as possible. And practically no one except the original author can make sense of it. Perl and regular expressions are terse.

    Expressive is also about showing less, but expressive specifically means showing less of the inconsequential and more of the consequential. If something is important, then an expressive language shows it to you. If something isn’t important, then an expressive language should hide that detail.

    Being expressive is an important part of staying DRY (Do Not Repeat Yourself). Everyone knows that DRY mandates that you do not say things twice: there should be a single source of knowledge for any fact. Hiding information, and terseness, often violates this. An expressive language helps you stay DRY by making sure knowledge is embedded in the system once without a lot of extra noise. Simple terseness and hiding information might leave you with zero sources of knowledge! Not a good place to be. Strive for expressive, and be wary of terse.

    Next Steps
    An important part of creating knowledge is maintaining a culture of constant improvement: every team can benefit from a feedback cycle where you experiment with a new practice, reflect and learn from the experience, and then accept or reject the practice as part of your team culture. The annual, year end reviews common in many companies are the antithesis of this. They are self-improvement theater, where feedback comes months after events occur and the details of how to improve are lost behind closed door meetings, never to be heard from again. As an alternative, think short term. Treat each two week iteration as a chance to conduct an experiment. Try Spock for two weeks and then discuss the results as part of your iteration close. If the team doesn’t like it then stop using it and try easyb for two weeks.

    If that doesn’t work then try something else. A two week commitment is much easier to sell to your teammates than a long term framework decision, and two weeks shouldn’t be long enough for even the worst decisions to irreparably harm a project. Week by week your team will improve as some of the experiments work and others fail. Just remember, an experiment or new practice should try to solve a specific problem for your team.

    Next Month: Defer Commitment
    Deferring commitment is about your keeping options open. It may seem wise to make decisions early, and it does lead to the illusion of progress. But decisions can be costly to unmake, and oftentimes new information will become available that affects the choices you’re willing to make. Next month’s article explores how Groovy and dynamic languages allow you to make decisions when you are ready, investigating ways in which you can use Groovy’s optional typing to your advantage.

    Learn More
    Spock is available through Maven and Google Code at http://code.google.com/p/spock/. The web has some nice Spock tutorials, the project lead has a blog at http://pniederw.wordpress.com/ and Parleys.com has a great 20 minute presentation from the 2009 Devoxx conference available to subscribers. For more information on the very cool mock objects available then you may wish to read my own post on the topic: http://canoo.com/blog/2010/04/20/spock-and-test-spies-a-logical-choice/ For books on agility, I still contend these three cannot be beat:

  • Implementing Lean Software Development – Poppendieck
  • The Pragmatic Programmer – Hunt and Thomas
  • Extreme Programming Explained, 2nd Edition – Beck and Anders
  • Share and Enjoy: These icons link to social bookmarking sites where readers can share and discover new web pages.
    • email
    • Print
    • Twitter
    • LinkedIn
    • XING
    • Facebook
    • Google Bookmarks

    Something Wicked Comes this Way

    August 10th, 2010


    Don’t doubt it for a second: A paradigm-shifting idea is taking shape on Johnny Anonymous’s laptop as I write these words. Johnny’s idea is a radical as it is ingenious. It will change the way you communicate, work, think… breath. It may bring you that one love you’ve been longing for all your life; bring your kids opportunities and joy beyond their wildest dreams; resolve international crises; lead to cures for cancer; cut CO2 emissions in half; make quantum mechanics comprehensible to the lay-person; cause Dan Brown to write a decent book (okay, maybe not). Hell, it may even help us to make contact with aliens – how cool is that?!

    Will Johnny’s idea ever see the light of day? As long as the Internet remains a level playing field, he/it has good a chance as any. Yet if the likes of Veroogle (Goozon?) have their way, then Mr. CEO will quash Johnny’s idea quicker than you can say the words “big fat severance package”.

    Should Big Corporate really be permitted to dictate the accessibility and the quality of service that Johnny’s idea receives when he attempts to reveal it to the world? Let me put it this way: Can we trust corporations to govern themselves in such a way that they never,ever take active steps to diminish the chances of a potential competitor’s idea from taking hold? Don’t worry – it was a rhetorical question.

    Is the world – virtual or otherwise – really so oblivious to the messy, awesome beauty that is the Egalitarian Internet; so apathetic that it will not lift a finger to rescue one of humankind’s finest achievements? Will the masses revolt? Furthermore, who will speak for us? Where are you Tim Berners-Lee?

    So until the masses mobilize – and they surely must – here’s a tiny, yet powerful step that Joe Public – read: you – can take RIGHT THIS INSTANT: Bing.

    Share and Enjoy: These icons link to social bookmarking sites where readers can share and discover new web pages.
    • email
    • Print
    • Twitter
    • LinkedIn
    • XING
    • Facebook
    • Google Bookmarks