This article originally appeared in the September 2010 edition of GroovyMag, the Groovy and Grails magazine. Part 7 is currently available for download from the magazine’s site, and more will come each month. Previous articles in this series are on the Canoo website: Part 1: Eliminate Waste, Part 2: Build Quality In, Part 3: Create Knowledge, Part 4: Defer Commitment, and Part 5: Deliver Fast. Lastly, if you like this, you may want to check out some of my older blog posts from my personal site under the ‘craft’ category. Enjoy!
The Groovy Programming Language advertises itself as an “agile and dynamic Language for the JVM”, but what does this mean exactly? This series of articles explains Lean Software Development, and shows how your choice of programming language can make your entire process remain nimble and adaptive. Each month’s article will cover one of the seven Lean Software Development principles and explain how Groovy and the associated ecosystem help eliminate waste, defer commitment, and build quality into your product.
About this series
Groovy is an agile programming language. In order to explain what this means, these articles are structured around the seven principles of Lean Software Development. The previous installments included a short introduction to Lean, explained how an expressive language can eliminate waste in unit testing, showed how the easyb Behavior Driven Development framework and Groovy Builders can build quality into your life-cycle, demonstrated how the Spock Testing Framework can create project knowledge, and discussed how tools like Real Options and dynamic languages defer commitment. This month we’ll see how Groovy’s Grapes module system, alongside modules like Groovy Web Services, streamline interactions between development and other groups like operations and QA (an important aspect of respecting people). Finally, we’ll look at optimizing the whole by applying Groovy to domains that cross department boundaries: how Gradle improves the build process and easyb serves as a collaboration mechanism between different organization roles. The series will end with a discussion on the best practices of mixing Java and Groovy.
Some of the code examples are basic while others are advanced. The articles explore why the features of Groovy are important rather than the mechanics of any one Groovy feature. Hopefully you’ll leave with some new ideas about how to use Groovy, how to convince your team that Groovy is worthwhile, or most importantly, how to increase your productivity.
Part 1 of this series contained a more in-depth explanation of Lean, and I won’t repeat it here. But as a quick refresher here are the 7 lean principles along with a summary for those principles covered previously:
- Eliminate Waste – Reduce work in progress and half done work
- Build Quality In – Instead of settling for finding defects, optimize your process to create fewer defects
- Create Knowledge – Create fast feedback cycle and encode project information in executable form
- Defer commitment – Delay decisions until the last responsible moment
- Deliver Fast – Increase velocity by working in small batches and removing bottlenecks
- Respect People
- Optimize the Whole
Principle 6 – Respect People
Respecting people means creating an environment where the human element drives innovation and value. Creative problem solving and spontaneity are our most valuable assets, and great companies arrange their development process to let every single employee be able to exercise these traits.
My experience with a big Capability Maturity Model (CMM) rollout was exactly the antithesis of this: we brought in several outside consultants who proceeded to observe the way we worked, then created a disciplined process we were required to follow. In their hearts, the consultants believed they were respecting people because the process they documented and codified was the one that we ourselves, the workers, had created ad hoc over the last several years. But our ability to improvise and make changes to the process greatly diminished. Trying something new meant filling out paper work and appearing before a process change group, so most people stopped bothering. The consultants never understood why we refused to display any sort of ownership over the new process. Well, it’s hard to think you own something when you aren’t allowed to change it. A development process must be malleable and open to experimentation, not locked down. There is nothing wrong with bringing in some outside help, but be careful who you choose. Consultants leave you with a some new documents or artifacts; coaches leave you with new skills. Seek that latter.
I’ve seen many development-led initiatives fail to respect people as well. At a previous company, we had a distinct pattern for making big technical decisions: call all the best people into a room to discuss the possibilities then assign a small group to deliver the changes. This ensured that the best developers solved the major issues, but everyone not in the council had neither input nor visibility into changes that affected their work. It was incredibly demoralizing to be a junior developer and know a big, new web framework was being developed behind closed doors, with no obvious way to influence its outcome. If you’re having a meeting with all the best people in your department, then there is only one thing you should be doing: firing everyone else. Seriously, what’s the value proposition of tolerating mediocrity? Why employ someone only to ignore their most valuable asset, their creative problem solving process?
The open source community, and the Groovy community in particular, is an excellent case study in how to respect people. The entire Groovy product development process is open and transparent. If you want to know what is going on, then subscribe to the Groovy-Developer mailing list and the Groovy JIRA ticketing system on The Codehaus. Both resources are open to the public and almost nothing happens that is not documented in one of these two places. If you want to provide input to the future direction of Groovy then just get involved in the email discussion and start commenting on the issues. Of course, the real beauty of Groovy is how the language grows and accepts changes. More than other languages I have seen, Groovy relies on their actual users for improvements. If several users ask for the same utility method then, more than likely, that method will show up in the next release. Methods like the new tr(), stripMargin(), stripIndent, expand(), and unexpand() were all added for the forthcoming 1.8 release. Instead of relying on research papers and doctoral theses for inspiration on how to grow the language, Groovy looks directly to you, the users. Next time you need to copy and paste utility methods between projects, or somehow else share that StringUtils you have, why not just create a new JIRA ticket and attach the source? The community will thank you and you’ll feel good about yourself.
The big question now is how you can get your company to start respecting people. Pitching an open source style work-flow isn’t going to get you very far. Most likely, the changes you need to make to start respecting people means changing a deep and long-standing business culture. Nathaniel Schutta summed up this common scenario perfectly: “Changing culture is hard, but for many organizations, it’s the critical first step towards better software.” Regardless of how open to new ideas your company is, the best approach to introducing change is often leading by example. The easiest way to convince a skeptical developer or operations team member of the value of Groovy is use it to make someone’s life easier. Repetitive environment configuration steps? Find a way to script it. Lengthy troubleshooting cycles? Ship operations some helpful .groovy files.
Sidebar: Agile Stage 0
Be careful promoting agility within your workplace: your company simply may not be ready. Andy Hunt of the Pragmatic Programmers describes an unwilling culture as “Agile Stage 0”, meaning it is not ready for agile. Symptoms include: behaving territorially, being inflexible, growing uncomfortable with uncertainty, treating developers like a commodity, and believing development is a linear process. If this describes your workplace, then expect a slow pace of adoption and find ways to enjoy the small victories.
A sustainable pace for programmers requires that we are allowed to work with technologies we love. Great techies crave new technology, and good companies crave great techies. In general, which job description do you think appeals to the best programmers: one listing Java and JUnit or one listing Java and Groovy? Fostering pride in workmanship is a key part of Respecting People, and the cultural change of allowing new ideas and languages like Groovy to flourish in your workplace can improve morale, attract new employees, and retain the ones you have. I’ve seen this first hand on a past project. For companies that aren’t early adopters, that have lots of legacy code, using Groovy outside of production source code, in testing, scripting, and builds, can be a great way to energize and respect the team.
Part of Respecting People is moving responsibility and decisions to the lowest level possible. The person most capable of making a decision about how to perform a job is the person doing that job. Creating knowledge of Groovy within your team gives everyone more and better choices for completing tasks. Consider the scenario of needing to create an ad-hoc web service client, perhaps for the QA or operations team to consume some of your web services. The GroovyWS module provides an easy way to do this, as seen in Listing 1.
@Grab(group='org.codehaus.groovy.modules', module='groovyws', version='0.5.1')
wsdl = 'http://www.webservicex.net/CurrencyConvertor.asmx?WSDL'
proxy = new groovyx.net.ws.WSClient(wsdl, this.class.classLoader)
rate = proxy.ConversionRate('EUR', 'CHF')
println "Euro to Swiss Franc conversion rate is: $rate"
Listing 1: Deployable SOAP Web Services Script
This script “Grabs” the GroovyWS module, contacts a web service endpoint, and dynamically creates a web service client capable of calling operations on the service. This is the entire application, nothing is hidden, including a deployment module! You can email this script to QA or Ops without a Jar and without an accompanying cryptic command line invocation. The @Grab Groovy annotation is a way to retrieve dependencies from repositories. Running this Groovy script will download the GroovyWS module and put it in the script’s classpath. The user only needs Groovy installed, one small change to the user’s HOME directory so that the Maven repositories can be found, and they will never again have to receive large Jar files or confusing scripts to run. And, unlike many Java web service solutions, there are no generated Java files that you’ll need to step through later trying to find out what is broken. The Grapes module system is a boon for anyone needing to help out other teams (which is pretty much all of us).
Sidebar: Configuring Grapes
Grapes needs to be told once where to find the Maven repositories. You do this in the ~/.groovy/grapeConfig.xml. On Unix systems, ~ is the user’s home directory, or $USERHOME. On Windows, this is something like “c:/Documents and Settings/username”. The GroovyWS website  has clear instructions on the XML to copy into this file.
It is easy advice to tell someone they should learn a scripting language. Having Groovy available to write scripts like these, that leverage the Groovy module system and are easily run by other teams, has clear value when compared to trying to write the Java equivalent. But a problem arises when developers are allowed to use dynamic scripting languages only for scripting. How do you learn a language effectively when you can only use it for one task a month? You don’t. Using a scripting language only for scripting is a mistake. It ensures that only the most self-motivated developers will take the time to write a script, and the majority of developers will have no clue how to maintain it should anything ever break. Effective scripting will only happen after a language has been adopted, it is not a means to language adoption. Groovy testing is a better gateway towards making these other opportunities viable.
W. Edwards Deming was an American statistician and professor who played a significant role in creating Japan’s reputation for high quality products. Deming emphasized a whole-systems, multi-dimensional view of the manufacturing process called the System of Profound Knowledge. Deming summarized his views in a short list of 14 points, one of which contained this: Eliminate annual performance ratings for salaried workers. I wholeheartedly agree. Annual performance reviews feed rivalry and office politics, producing an environment of fear around the first of the year. And for peer-reviews, I have never seen these amount to anything more than a popularity contest. The number one action to take to ensure a good year-end peer review is to remember to bring in treats and pastries once a month. So much for focusing on business value. Annual reviews do nothing to improve good teams and simply waste time for poor teams.
Without reviews how will team members know how to improve? How will we know who to promote? How much do we give in raises? One of my favorite agile thinkers, Esther Derby, recently republished several articles on her website addressing just these issues , , . Ongoing, face-to-face coaching sessions need to replace a year end evaluations. And if someone isn’t doing their job, then why wait for the year end cycle to fire the person? Likewise with those who are over-performing. If you don’t find ways to reward top performers more often than at year end, then you’ll find many of them move on in the middle of the year! Profit sharing is a much fairer compensation plan than raises, but most of us aren’t doing it for the money anyway. Before disagreeing too quickly, watch the YouTube video called “Drive: The surprising truth about what motivates us” . You’ll realize reward is a complex subject; but reward definitely doesn’t come from a year end review process. A good step to building an honest, trust based, and high performing workplace is to throw out those cursed year end reviews.
Next Month: Optimize the Whole
Improving your business is too often an exercise in sub-optimization: you may spend all your effort on improving the unit testing experience without realizing it’s your interactions with operations that’s derailing the project, or you may perfect your own team’s delivery process while the rest of the company continues behaving wastefully. In either case, it pays to step back and examine the situation as a whole. If you’re optimizing your unit testing with Groovy, then step back and ask how you can improve the entire project and not just testing. If you’re working with your team on a better software life-cycle, then step back and ask how you could improve the life-cycle of all the teams. Next month we’ll discuss how to “measure up” and optimize the whole, looking specifically at how to use easyb to facilitate cross-department interactions and Gradle for full project automation.
I’ll be presenting a session on Lean Groovy in Denver at the Uberconf conference in July. I hope to see you there! Otherwise, several references were used to create this article, and all are enjoyable, informative reads in their own right:
 Lean Software Development – Poppendieck
 Implementing Lean Software Development – Poppendieck
Hamlet D’Arcy Canoo Engineering AG
Hamlet D’Arcy has been writing software for over a decade, and has spent considerable time coding in C++, Java, and Groovy. He’s passionate about learning new languages and different ways to think about problems, and recently he’s been discovering the joys of both F# and Scheme. He’s a committer on several projects including Groovy, CodeNarc and JConch, and is a contributor on a few open source projects (including Griffon and the IDEA Groovy Plugin). He blogs regularly at http://hamletdarcy.blogspot.com and http://canoo.com/blog.