Tuesday 30 December 2008

End of year burp..

I've been getting plenty end of year roundups in my reader and thought I might add something forward looking to the blog to encourage the team to use 2009 to investigate some interesting new things which we could all benefit from.

For Grails I would suggest that anyone working on new project (ahem, you know who you are) using Liquibase (or planning to), should seriously consider using Autobase. Its DSL for Liquibase, so keeps things more Groovy and less XML (we loved stepping off the spring.xml train, so now we're in Liquibase XML fun... why?).

The HttpBuilder DSL was announced without too much fanfare, which I know we use a lot. Again, another handy tool to jump on.

If you haven't heard of Robert Fischer by now then you've been sleeping beside your pair, but his new book is sure to be a boon to us all: Grails Persistence with GORM and GSQL. Get your pre-orders in, its out on the 26th of Jan.

Another interesting looking book is ScrumBan by Corey Ladas. We're always talking about improving how we work, and the Lean folks are challenging Scrum'mers to think a little more like them.

Which brings me to my last and more unusual mention which is that David Anderson managed to convince the people at Agile 2009 to setup the Agile Frontier Stage. David Anderson gave the singularly most captivating presentation at Agile Vancouver 2008 and stands out to me to challenge (in a very healthy and practical way) a lot of the orthodoxy which Scrum has become entrenched with. He's good people, see him present!

Thats its from me for this year

Happy New Years all :)

Invalidating Single Cached Items

Should have RTFM on this one but it was stuck on the board for so long and even F5 support couldn't figure out so I thought I'd post it here.

Sometimes we need to remove content from our CMS which means our app servers will serve up 404s for those requests. But we found that our F5 kept serving content from its stand-in cache for another 24 hours after the max-age had expired. We thought the stand-in cache was only there for when all the nodes in your pool had died so it looks like it's doing a little more than that.

Anyway, sometimes when we remove stuff from 'live' we really want it gone so we searched around and found that you can remove a single item from the F5 cache using a SOAP request to the management interface. The bad news is that you have to provide usernames and passwords in the clear but there's a nice way to get the same effect using invalidation triggers. At the back of Chapter 11 in the Policy Management Guide there's a straightforward example of what to do.

Thursday 11 December 2008

Handy Hints in BASH

Ever wanted to diff the output from 2 commands but didn't want to have to write the output to temporary files first? Well bash subshells come to the rescue.

Let's say you want to compare the list of files in 2 different directories. Try this:

diff <(ls /path/to/dir1) <(ls /path/to/dir2)

The bits in brackets are run in a subshell and then we redirect the output of that subshell to the stdin of diff. (On a personal note I prefer to run diff with the -y switch to get a side-by-side diff, but that's me).

Of course you can get really tricky with these things. How about making sure the file permissions are right as well?

diff -y <(ls -l /path/to/dir1 | awk '{print $1, $NF}')
<(ls -l /path/to/dir2 | awk '{print $1, $NF}')

(All that should be on one line. This is my first post so be gentle on me if it comes out looking like an uncombed monkey.)

So, have fun with subshells. And be careful out there.


Caching Grails Services

I've been plugging away for a few days on adding caching to Grails services. At first I tried to go down the road of decorating the services' metaClasses but since Grails uses an AOP proxy to add transactional behaviour to services this wouldn't work - the metaClass I got from the injected service object was not the same one my plugin had added stuff to. Also annotations on the service class were unavailable from the proxy.

The Spring Modules project has a nice caching implementation that uses annotations to decorate methods with caching and flushing behaviour. Getting this to work with Grails, was reasonably straightforward. As it turns out one additional config line made the difference between the spring-modules example config and something that worked in Grails.

I based the config on the annotation example from the Spring Modules documentation, converting the Spring XML to Grails BeanBuilder format:

    import org.springframework.aop.framework.autoproxy.*
import org.springmodules.cache.annotations.*
import org.springmodules.cache.interceptor.caching.*
import org.springmodules.cache.interceptor.flush.*

def doWithSpring = {

// declaration of cacheManager and cacheProviderFacade omitted - implementation specific

autoproxy(DefaultAdvisorAutoProxyCreator) {
proxyTargetClass = true
}

cachingAttributeSource(AnnotationCachingAttributeSource)

cachingInterceptor(MetadataCachingInterceptor) {
cacheProviderFacade = ref("cacheProviderFacade")
cachingAttributeSource = ref("cachingAttributeSource")
def props = new Properties()
props.myCachingModel = 'cacheName=MY_CACHE_NAME'
cachingModels = props
}

cachingAttributeSourceAdvisor(CachingAttributeSourceAdvisor, ref("cachingInterceptor"))

flushingAttributeSource(AnnotationFlushingAttributeSource)

flushingInterceptor(MetadataFlushingInterceptor) {
cacheProviderFacade = ref("cacheProviderFacade")
flushingAttributeSource = ref("flushingAttributeSource")
def props = new Properties()
props.myFlushingModel = 'cacheNames=MY_CACHE_NAME'
flushingModels = props
}

flushingAttributeSourceAdvisor(FlushingAttributeSourceAdvisor, ref("flushingInterceptor"))
}
The vital bit is the proxyTargetClass = true on the autoproxy bean. I won't pretend to understand what that's doing - I just noticed that's how ServicesGrailsPlugin was getting the transactional proxies to work.

After that it's a simple case of annotating service methods (in fact methods on any Spring-managed bean but services are the obvious use case).

    import org.springmodules.cache.annotations.*

@Cacheable(modelId = "myCachingModel")
def getSomethingInAnExpensiveWay(param1, param2) {
// ...
}

@CacheFlush(modelId = "myFlushingModel")
def updateSomething(param1, param2) {
// ...
}
This works fine on transactional and non-transactional Grails services. You can see the cache operations by setting log4j.logger.org.springmodules.cache = "trace".

The actual caching implementation used can be anything - spring-modules supports ehcache, oscache, JBoss cache, JCS, etc. All that's required is to wire in the cacheManager and cacheProviderFacade beans as described in the Spring Modules documentation.

I've bundled the results up as a Grails plugin (grails install-plugin springcache). It uses a ConcurrentHashMap backed simple cache implementation by default but wiring in ehcache or oscache is easy.

Caching and flushing models are configured in Config.groovy, e.g.:

    springcache {
cachingModels {
cachingModel1 = 'cacheName=CACHE_1'
cachingModel2 = 'cacheName=CACHE_2'
}
flushingModels {
flushingModel1 = 'cacheNames=CACHE_1,CACHE_2'
}
}
Particular caching implementations may have further options or require additional external config (such as ehcache's ehcache.xml file).

To disable the plugin you can set springcache.disabled=true. For example, it may be desirable to disable the plugin in the test environment.

Edit: Documentation added on grails.org

Passing enums into a constructor

Take this error:

binding.GrailsDataBinder Unable to auto-create type, 'create' method not found.

I came across this yesterday, and although the cause was documented, two of us managed to miss the crucial sentence in those docs for about an hour, causing much scratching of heads:

"my problem is that when I pass a Status enum to a constructor of some other class"

So, setting enum properties on a domain object is fine:

Burger canHaz = new Burger();
canHaz.flavour = Flavours.CHEESE;

Just not doing so via a constructor call:

Burger willNeverHaz = new Burger(flavour:Flavours.VEGETARIAN);

The original post I found is here:

http://www.nabble.com/Using-enums-gives-%22Unable-to-auto-create-type,-'create'-method-not-found%22-error-td18890306.html

and the JIRA bug report here:

http://jira.codehaus.org/browse/GRAILS-3314

Hopefully this will save someone else lots of head scratching! :o)

Wednesday 10 December 2008

Nasty liquibase bug

We recently came across a nasty liquibase bug.

When you drop a not null constraint on a table and you do not add in the column data type liquibase creates the column with a null column type.

Eg:
dropnotnullconstraint tablename="pirates" columnname="crew" --- creates a null column type.

Easy fix:
dropnotnullconstraint tablename="pirates" columnname="crew" columndatatype="VARCHAR(255)"


Just wanted to give everyone a heads up on the problem so they don't burn half a day trying to figure out what is going on.

Thursday 27 November 2008

Counting using Grails/Hibernate criteria

Just a quick gotcha for grails criterias when you want to count the results from a query where there is a join involved. If you use

count = criteria.count {
projections {
rowCount()
}
query()
}

You'll get a count for each of the joins, you might think that you could use

count = criteria.count {
projections {
countDistinct('id')
}
query()
}

But that still doesn't work - you have to use

count = criteria.get {
projections {
countDistinct('id')
}
query()
}

Hmmm

Monday 24 November 2008

Autobase 0.5 plugin released

Robert Fischer just announced the latest version of the Autobase plugin which simplifies Liquibase usage by way of providing a Groovy DSL to use it by.

http://github.com/RobertFischer/autobase/wikis/example-usage

Much more compact and more Groovy! Looks like all ur Liquibase migrations r belong to Autobase nao!

Monday 17 November 2008

Groovy Awesomeness

Hey check this out:

Java:


import java.util.Calendar;
import java.util.Date;

public class PrintIndependenceDay {

public static void main(String[] args) {
Calendar calendar = Calendar.getInstance();
calendar.clear();
calendar.set(Calendar.MONTH, Calendar.JULY);
calendar.set(Calendar.DATE, 4);
calendar.set(Calendar.YEAR, 1776);
Date time = calendar.getTime();
System.out.println(time);
}
}


Now check the awesomeness groovy gives you:

def calendar = Calendar.instance
calendar.with {
clear()
set MONTH, JULY
set DATE, 4
set YEAR, 1776
println time
}


Here is the full article that explains everything: Getting Groovy with 'with'

rock on

Sunday 16 November 2008

Friday 14 November 2008

Fun With Groovy Categories

Just a quick stupid trick I discovered. Commons Lang's StringUtils class works quite nicely as a category for String, e.g.
    use(StringUtils) {
println 'i can has cheezburger'.substringAfter('i can has ') // prints 'cheezburger'
}
Not only that but the JetGroovy plugin for IntelliJ IDEA will autocomplete all the new String methods inside the use block.

Monday 10 November 2008

Checking HTTP Headers

Disclaimer: This post has nothing to do with Grails, other than the application we're testing is a grails app!

We're starting to "productize" our application. One of the stories is to ensure all javascript, CSS and image files are served with an appropriate set of HTTP headers so they can be cached effectively. For this we're using good old HTTP unit (I'll post about that more when we're finished), but in the meantime I've found the 'tcpdump' command very useful.
sudo tcpdump -A -ilo0 -t -s 1500 port 8080

-A prints ASCII
-i specifies the interface (I'm testing locally so lo0 is the loopback interface)
-t supresses timestamps
-s How many bytes to print (I wanted to see the full packet so set it to 1500)
port 8080 Tells tcpmon to only show me stuff going to or coming from port 8080

This results in output like

...GET /script/N259177905/bundles/stuff.js HTTP/1.1
User-Agent: httpunit/1.5
Cookie: splash=false
Cache-Control: no-cache
Pragma: no-cache
Host: localhost:8080
Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
Connection: keep-alive


IP localhost.http-alt > localhost.53121: . ack 1892 win 65535 >nop,nop,timestamp 219937537 219937537<
E..4.n@.@...............:.,A.........(.....
...
IP localhost.http-alt > localhost.53121: . 197931:214263(16332) ack 1892 win 65535 <nop,nop,timestamp 219937537 219937537>
E.@.q.@.@...............:.,A........=......
...HTTP/1.1 200 OK
Content-Type: text/javascript; charset=UTF-8
Expires: Sat, 10 Nov 2018 11:24:54 GMT
Cache-Control: public, max-age=315360000, post-check=315360000, pre-check=315360000
Last-Modified: Sun, 06 Nov 2005 12:00:00 GMT
ETag: 2740050219
Transfer-Encoding: chunked
Server: Jetty(6.1.4)

6000
;(function(){function r(val,args){for(var x=0;x<args.length;x++){val=val.replace('{'+x+'}',args[x]);}
return val;}
function p(){var val=arguments[0];var ret;if(val.indexOf('{0}')!=-1)
ret=function(){return r(val,arguments);}
else ret=function(){return val;}
...

I'm sure there's more I could do to fine tune the output, but it's a good starter for 10!

Wednesday 29 October 2008

GGUG meeting tomorrow

Just an FYI that this months Grails user group meeting is tomorrow night. Here is the link if you want to come. Grails meetup

Joda DateTime Ranges

Recently I was attempting to create a Range representing the difference between two DateTime values so I could assert in a test that a date time value fell into that range. I was using Joda Time's DateTime class. The java.util.Date class is compatible with Groovy's ObjectRange becuase it's decorated with next and previous methods by the GDK. Joda's DateTime isn't, so I ended up doing this:
    DateTime start = new DateTime()
DateTime end = start.plusYears(1)
Range range = start.millis..end.millis
I could have created a custom Range implementation but I'd have lost the option of using the x..y notation and I'm a sucker for brevity. I figured Groovy has an IntRange class so there's probably a LongRange as well. As it turns out there isn't and what I ended up with is an ObjectRange.

Fundamentally this works, although I did find something worth bearing in mind. When it came to my assertion I tried to do this:
    static void assertInRange(Range expected, actual) {
if (!expected.contains(actual) {
fail("Expected value in range:<${expected}> but was:<${actual}>")
}
}
Simple enough, right? Except running the code caused the test to hang. Looking into the implementation of ObjectRange it became clear why. Groovy's Range interface extends java.util.List and that's where the contains method is specified. For compatibility with the List interface ObjectRange iterates over the values between its from and to properties using the next method I mentioned above. When from and to are Long instances, it turns out this iteration takes a while - on my 2.16GHz iMac the contains method takes over 4½ minutes on a 1 year range!

The Range interface also specifies a containsWithinBounds method and ObjectRange implements this by simply checking that the argument is >= from and <=to. Changing my assertion to use containsWithinBounds got rid of the long wait.

What ObjectRange is doing does make perfect sense. Given two arbitrary objects that implement Comparable and the next and previous methods it needs to implement contains in a way that conforms to the List interface. When you think about how the GDK implements next and previous on java.util.Date for example you can see that there may be possible values of a given class that fall inside a range but would never be part of the List generated by repeatedly using next on the range's from property. This isn't true of Long, of course, but ObjectRange is a general purpose class. The IntRange class has a much optimised version of contains and so could a theoretical LongRange implementation.

Anyway, getting back to the example. After changing contains to containsWithinBounds the code ran much faster, unfortunately it also threw java.lang.OutOfMemoryError: Java heap space. It turns out this is down to how GString handles the ObjectRange value I inserted into it. It spots that the object it's got implements List and uses InvokerHelper.toListString on it. This will attempt to create a String in the form "[0, 1, 2, 3...]". As you can imagine, building such a String is going to take some doing with over 31½ billion elements in the List, hence the heap space ran out.

A final working implementation of my assertion method is:
    static void assertInRange(Range expected, actual) {
if (!expected.containsWithinBounds(actual)) {
fail "Expected:<${expected.toString()}> but was:<${actual}>"
}
}
I think going for the custom Range implementation in the first place might have actually been the right choice.

Saturday 18 October 2008

Gmock - Groovy Mock

Mock Object are key players in Unit Testing. Groovy support natively some mock object with MockFor and StubFor but their functionality are quite limited and the syntax heavy - you'll understand when you nest 7 'use' closures.

Gmock aims to simplified mocking in Groovy through a intuitive syntax and a great readability. In a nutshell a Gmock test look like:

void testTree(){
def mockTree = mock()
mockTree.load('fruit').returns('apple')
play {
assertEquals "apple", mockTree.load('fruit')
}
}
This is it!. Expectation are being setup by calling normal method on you mock object. The code under test is executed within the play closure and your mocks are automatically verified after it.

The current version gmock-0.2 support the most basic functionality you would expect from a mocking framework. Version 0.3 should see static method mocking and property mocking. Future development are described in the Roadmap.

Wednesday 15 October 2008

Selenium and Firefox 3

Those of you who've tried to run the selenium test suite using Firefox 3 as the browser have probably found that Firefox hangs trying to create the temporary profile. You could do some tedious mucking about with selenium.browser="*custom /usr/lib/firefox/firefox -no-remote -P selenium", manually configure the proxy, etc. However, it turns out there's a simpler solution.

Please note this is nothing to do with Selenium IDE, if you're having problems with that in conjunction with Firefox 3 this won't help.

Tuesday 14 October 2008

A “clearfix” CSS redux

We use a CSS technique called "clearfix" within several projects. This is implemented as a class (called "clearfix") that is applied to a parent element that contains floated child elements.

I wrote a post over at Code Couch that attempts to explain the reason for doing this in the first place, describes the CSS that we use and provides plenty of links for detailed reference.

Thursday 9 October 2008

XHTML — myths and reality

Tina Holmboe is a member of the XHTML Working Group and has an interesting write-up XHTML — myths and reality that discusses the current state of XHTML and dismisses some popular inaccuracies.

Having hyped up the post, it is rather old news - although it is discussed in simple (simplistic?) terms which I find is better than overloading with jargon.

Here are some hilights:

Lack of support for XHTML is a fact of life on the web in 2008. Prior to the 3.0 series of Firefox the XHTML processor in Gecko was so poor that Mozilla's own engineers recommended against it;
No version of Internet Explorer up to, and including, IE 8 support XHTML at all, and a number of other browsers such as Lynx were never written to handle XML in the first place.

I wrote a post for another blog a while back that looked into how you can identify the appropriate doctype for your web page - and stumbled across this "lack of support" for XHTML in modern browsers. You can read the original post How to choose a doctype if you are interested.

Whilst XHTML remains a much more structured (and cleaner) choice to deliver content to the browser, I have yet to work on a single project where it is actually required.

XHTML is the default for use within the Grails framework - even though the worlds most prolific browser doesn't support it. Hmmm.

Programmatic Transactions

By default grails starts a new transaction the first time it enters a service method and commits the transaction when it returns. You can disable this behaviour by declaring your service to be non transactional.

Sometimes you need more fine grained control. Here's a link to a great discussion which unfortunately doesn't seem to have made it into the Grails manual yet, however this is still talking about declarative transactions, rather than programmatic ones.

If you really do want programmatic transactions you can use
DomainObj.withTransactions { TransactionStatus ts ->
serviceA.doStuff()
serviceA.doMoreStuff()
serviceB.yawn()
}

But the caveat (and reason for this post) is that in the above scenario will only create a new transaction if one does not already exist. If the above is placed in a controller it will work fine. If it's placed in a service with
static boolean transactional = false

it will work fine, PROVIDING that the service is being called by a controller or other non-transactional service, however if your non-transactional service is called by a transactional one, the withTransactions block will have no effect.

Thursday 25 September 2008

Effective testing on Grails

Gus and I will be presenting on "Effective Testing on Grails" at the London Groovy and Grails User Group next week Wed 1st of October, @18.30. Please feel free to attend if this seems relevant to you. A synopsis of the talk and registration for the event are all available at the following link.

http://skillsmatter.com/event/java-jee/ggug

The UI decisions behind a typical web form

I stumbled across a very useful article recently and wanted to share it around. The article discusses the best placement of form submit/cancel buttons for a typical web form and backs this up with Heat Maps showing where the user spent time looking at elements on the page etc.

Check out the Primary & Secondary Actions in Web Forms blog post yourself.

There are a lot of very useful posts on that blog - all of them focused on useability and the "dark art" of UI decision making.

Inspecting Response Headers in the browser

I needed to inspect the headers being sent to various browsers today. Specifically we wanted to find out how Yahoo was able to deliver a "fresh" CAPTCHA image when you used the browser back button to return to the page with the CAPTCHA image on it.

We observed that this happened even with Javascript disabled, and so came to the conclusion that it must be something in the response headers.

In Firefox (v1.5+) there is a really useful extension that I have mentioned before (the Web Developer Toolbar extension) which can show the headers using Tools -> Web Developer -> Information -> View Response Headers.

When using Internet Explorer a solution is to use the IE HTTP Headers plugin (for IE v5.0+) which is available at http://www.blunck.se

For those interested, the solution to the problem was to add the following headers to the response:

static void doNotCacheResponse(HttpServletResponse response) {
response.setHeader('Pragma', 'no-cache')
response.setHeader('Cache-Control', 'no-cache,no-store')
response.setHeader('Expires', '-1')
}

We've tested that this works to force a re-request using Firefox 2, Firefox 3, IE6, IE7, Chrome, Safari 3 and Opera 9 - which is nice!

Tuesday 23 September 2008

Careful with what you Query.

The F5 BigIP load balancers have a feature called iRules. They allow you to manipulate and manage traffic in the Load Balancer.

They are really useful for checking or modifying cookies [HTTP::cookie], the host domain [HTTP::host], path [HTTP::path] and query parameters [HTTP::query] of URLs and requests.

A sligth word of warning though.... any checks made on the QUERY of a HTTP request actually search the URI of the referer as well. As an example, the following iRule would send a HTTP 200 response if you were either requesting a URL with a query parameter of "ThereIsAQuery" OR if you're refer page's URL included a query parameter of "ThereIsAQuery".

when HTTP_REQUEST {
if {
([HTTP::QUERY] eq "ThereIsAQuery")
}
{
HTTP::respond 200 content "<html><body>There is a Query</body></html>"
}
}


Other attributes such as [HTTP::host], [HTTP::path], and [HTTP::uri] are only valid for the requested URL. They are not checked in the referer header.

Monday 15 September 2008

Command Objects Revisited

I posted yesterday about command objects but a conversation I had this morning made me realise I'd neglected to mention one of the interesting and less obvious ways to use them.

Imagine a scenario where you have a form that is basically for saving domain object instances but has a few other fields, not directly properties of the domain object in question but things like choices of how to set property values (e.g. a radio button to choose whether or not to update the timestamp on the object), workflow type instructions to the controller (e.g. preview this object after saving it), etc. This is a very common scenario.

You can handle this neatly with command objects by nesting an instance of the domain class inside the command object class, e.g.
    class MyCommand {
boolean updateTimestamp
boolean previewAfterSave
MyDomain domainObj
}
Then name your form fields accordingly...
    <g:checkBox name="updateTimestamp" value="${command.updateTimestamp}" />
<g:checkbox name="previewAfterSave" value="${command.previewAfterSave}" />
<input type="hidden" name="domainObj.id" value="${command.domainObj.id}" />
<input type="text" name="domainObj.name" value="${command.domainObj.name}" />
<g:select name="domainObj.otherDomainObj" from="${OtherDomainClass.list()}" value="${command.domainObj.otherDomainObj.id}" optionKey="id" />
and so on.

Sunday 14 September 2008

Command Objects

We have a bunch of horrible code in our codebase to do with binding form fields to objects. To be fair it dates from a time when we were a lot newer to Grails and Grails and its documentation was a lot less mature. The other day I had to add a date picker to one of our forms and write the binding code for it. The g:datePicker tag creates some select boxes and a hidden field. Unlike other implementations I've used in the past it doesn't use Javascript to compose the select box values into the hidden field on form submit. Instead the hidden field has the value "struct" which I guess is a hint to GrailsDataBinder. To cut a long story short the code I ended up writing is pretty nasty. Especially considering that GrailsDataBinder is perfectly capable of handling this kind of binding without any code being written.

Instead of all this tedious messing about with params Grails controllers can, as most of you probably know, bind form submissions to domain objects or command objects. Binding to domain objects works by doing domainObj.properties = params or using the bindData method. So much most of us are used to (even if you wouldn't think so from looking at some of our code).

To have your controller action use a command object you simply specify it as an argument to the action closure. e.g.
    def save = { SaveCommand command ->
if (command.hasErrors()) {
render(view: 'index', model: [command: command])
}
// do some stuff
}
The binding is done for you. As you may guess from the command.hasErrors() above, command objects can have constraints exactly as domain objects do. They can even have services and other Spring application context artefacts wired in automatically just like controllers and services can.

A form that contained...
    <input type="text" name="name" value="${command?.name}" />
<g:datePicker name="birthday" value="${command?.birthday}" />
<g:select name="sandwich.id" from="${Sandwich.list()}" value="${command?.sandwich?.id" optionKey="id" />
Would bind to a command like this...
    class MyCommand {
String name // simple property
Date birthday // bound from multiple fields submitted by date picker
Sandwich sandwich // bound domain object
}


Unit testing is easy because you can simply construct the command object and pass it in to the controller action. You don't have to worry about setting a bunch of esoteric key/value pairs on params to get your controller's hairy binding logic to work. In an integration test, if you really want to, you can set params and test that the command binding works as you expect.

The only non trivial thing in unit tests (in integration tests this is not a problem) is testing command error handling. In unit tests commands won't have an errors property or hasErrors method like they do in a running app or integration test. I've come up with the following which I've raised as an enhancement request for the experimental testing plugin:
    def createCommand(Class clazz) {
def command = clazz.newInstance()
def commandErrors = new BeanPropertyBindingResult(command, GrailsClassUtils.getLogicalPropertyName(clazz.name, ''))
command.metaClass.getErrors = {-> commandErrors }
command.metaClass.hasErrors = {-> commandErrors.hasErrors() }
}

void testCommandWithBindingErrors() {
def command = createCommand(MyCommand)
command.errors.reject('birthday', 'nullable')
controller.action(command)
// assert the form is re-rendered with the command in the model, etc.
}


Command objects can also be used to handle multipart upload forms...
    <input type="file" name="avatar" />
Maps to...
    class MyCommand {
byte[] avatar
}
If you need access to the uploaded file's metadata (content type, original filename, etc.) instead of declaring the property as type byte[] use MultipartFile. Either way the binding works seamlessly and allows you to apply constraints to the uploaded file.

The weak area in binding at the moment seems to be one-to-many domain class relationships (which is why I didn't jump into refactoring the code I mentioned at the start of this post). That does require you to write hairy binding code and is probably worth a post in and of itself.

Friday 12 September 2008

None of the rest of us can fathom CSS either.

Dave Minter has written a blog entry Float like a wasp that goes on about the inadequacies and problems faced with implementing CSS and the frustrations of HTML. I think that he's got a point when he says:

You are not alone. None of the rest of us can fathom CSS either.

He has a hit-list of things he'd like to see made more simple... and they include the usual bug-bears that we have all experienced at some stage (some more than others, obviously).

There are solutions to all the things he would like to see... but the key thing is that these solutions are invariably not intuitive and require arcane knowledge (and the patience of a saint) to ensure conformity.

I particularly liked this quote:

One of my tips for fixing all that irks with CSS would be to buy two or three of those books describing machiavellian ways to conspire against the deficiencies of CSS (anything with "hacks" or "tips" in the title) and then beat up the standard until the easiest approach to implementing everything in the book could be covered in a pamphlet.

I, too, would like to see a more structure simplification (rather that the current direction of CSS which seems to be adding to the mess).

Sunday 7 September 2008

Pessimistic Locking With Grails

By default grails uses optimisic locking with versioning. In a nutshell this means that each of your domain objects is blessed with a version field and hibernate throws an exception if you ever try to update or lock an object with an old version number. For scenarios where you are happy to handle the exception, or just report the error to the user via a UI this is fine, however when this behaviour isn't ok you need to switch to pessimistic locking. This is where the fun begins.

Unfortunately the Grails pessimistic locking support is incomplete. There are scenarios when it will work, but scenarios when it wont. As of 1.03 the docs don't make this clear. For example, let's say you have a domain object called Account...
class Account {
String name
long balance

static mapping {
version false // Required to avoid stale object exceptions when hibernate attempts a lock
}
}

Obviously updates to the account balance need to be atomic. The Grails docs suggest the following...
Account account = Account.lock(id)
account.balance += amount
account.save()

The resulting SQL will look something like
select id, name, balance from account where id = 123 for update;
update account set balance = 101 where id = 123;

The above will work fine PROVIDING the account entity is not already in the hibernate session. If the account entity was previously retrieved the resulting SQL only re-selects the id.
select id from acount where id = 123 for update;
update account set balance = 101 where id = 123;

The db row is still locked, but because the account entity has not been refreshed your application could be working with stale data. The annoyance with the Account.lock(id) approach is that you always need to know the entity's id without obtaining the entity. So your code can never do
Account account = Account.findByName(name) // Maybe in a controller or other service
...
account = Account.lock(account.id)
account.balance += amount
account.save()

My initial workaround to the above is to duck into the hibernate API and do the following
sessionFactory.currentSession.refresh(account, LockMode.UPGRADE)
account.balance += amount
account.save()

Which appears ensures you have fresh data and a db lock. However this is ONLY true if your entity does not have any eager relationships (one-to-ones are eager by default). Incredibly hibernate retrieves the eager relationships before calling select for update on the main entity, meaning that the relationship data could be stale. I've tried fixing this with cascade: 'lock' but it did SFA. It now appears the best (and I use that term loosely) way to ensure fresh, locked data is to do the following
account.lock() // versioning must be disabled
account.refresh()
account.balance += amount

An alternative is to call "discard" on the object before you lock it
account.discard()
account = Account.lock(account.id)
account.balance += amount
account.save()

An interesting aside, which has nothing to do with grails but everything to do with concurrency is how you prepare to update multiple objects, e.g.
accountsToBeUpdated.each { Account account ->  
account.lock()
account.refresh()
account.balance += amount
account.save()
}

The above code might look OK at first glance, but what if two threads were to update an intersecting set of accounts?

Thread 1: accountA, accountB
Thread 2: accountB, accountA

If you were unlucky Thread 1 may obtain a lock on accountA at the same time that Thread 2 obtained a lock on accountB resulting in deadlock. The solution is to sort both sets of accounts using the same algorithm anywhere you attempt to locks multiple entites, e.g.
def sortedAccounts = accountsToBeUpdated.sort { a, b -> a.name <=> b.name }
sortedAccounts.each { Account account ->
account.lock()
account.refresh()
account.balance += amount
account.save()
}

Another concurrency scenario is when you want to modify the many side of a one-to-many relationship, e.g.
class Person {
Set subscriptions
}

In this case because a subscription is never shared between two people it may be ok to just lock the person object, e.g.
person.lock()
person.refresh()
person.addToSubscriptions(subscription)
person.save()

The reason I say 'may', is that this approach is dependent on all your other code taking the same approach when updating subscriptions and that it's not a performance bottleneck to lock the person. Choosing to lock the person is certainly an easy way to avoid creating duplicate subscriptions without the risk of a unique constraint violations.

Sunday 31 August 2008

Up The Limit

Recently while using the searchable plugin with Grails I've had exceptions citing too many open files. Linux systems typically limit resources on a per-user basis by various criteria such as number of processes and number of open files. Running ulimit -a I see the following:
core file size          (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 36864
max locked memory (kbytes, -l) 32
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 36864
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
On this system I can only open 1024 files at once - I can increase this limit in /etc/security/limits.conf by adding something similar to the following:
gus  soft nofile  16384
gus hard nofile 16384
To ensure that PAM actually takes notice of these limits on login check /etc/pam.d/login for an entry like:
session    required     pam_limits.so
Can haz filez!

Thursday 28 August 2008

Unit Testing Controllers With The Testing Plugin

Grails JUnit tests are, as you are no doubt aware, divided into unit tests and integration tests. Unit tests run fast, aren't dependent on the database or Spring context but mean you have to live without all the dynamic magic that Grails attaches to your artefacts (controllers, taglibs, domain classes, etc.) This can certainly lead to some sub-optimal test code.

Lets consider an example. I want to write some test coverage for a controller that looks up a domain object based on a parameter and places it in the model. Usual disclaimer: of course, in the real world I'd have written the test first, etc. Nothing much to the controller:
class LolrusController {
def show = {
def myLolrus = Lolrus.findByName(params.lolrusName)
render(view: 'lolrus', model: [lolrus: myLolrus])
}
}
and here's the domain class:
class Lolrus {
String name
String mood
static hasMany = [posessions: Item]
static constraints = {
name(unique: true)
posessions(validator: { posessions ->
return posessions.any { it.type == 'bukkit' }
})
}
String toString() { "Lolrus[$name]" }
boolean equals(Object o) { return o instanceof Lolrus && o.name == name }
int hashCode() { return 37 * name.hashCode() }
}
Okay, so a reasonable test (ignoring edge cases for now) could be to create a couple of domain objects, make a request to the controller and verify that the correct domain object is retrieved. Simple, right?:
class LolrusControllerTests extends GroovyTestCase {
def controller
def lolrus1, lolrus2

void setUp() {
controller = new LolrusController()

lolrus1 = new Lolrus(name: 'Hugh')
lolrus2 = new Lolrus(name: 'Alan')
[lolrus1, lolrus2]*.save(flush: true)
}

void testShowActionFindsCorrectLolrusAndSticksItInModel() {
controller.params.lolrusName = lolrus1.name
def model = controller.show()
assertEquals('/lolrus/show', controller.modelAndView.viewName)
assertEquals(lolrus1, controller.modelAndView.model.lolrus)
}
}
Unfortunately, not so simple. The test fails and reports
expected:<Lolrus[Hugh]> but was:<null>
What we've forgotten of course is that Lolrus has a mandatory field mood and a custom validation on its posessions association that requires it to have a bukkit. The two Lolrus instances we tried to create in setUp couldn't be saved. Great. Well, let's just add that to the test set up:
    void setUp() {
controller = new LolrusController()

lolrus1 = new Lolrus(name: 'Hugh', mood: 'worryingly cheerful')
lolrus1.addToPosessions new Item(type: 'bukkit')

lolrus2 = new Lolrus(name: 'Alan', mood: 'mildly distressed')
lolrus2.addToPosessions new Item(type: 'bukkit')

[lolrus1, lolrus2].each {
assert it.save(flush: true), it.errors
}
}
Okay, it works but look how much of what's going on in setUp is actually nothing to do with anything the test cares about. This is a very simple example with a single test case. Imagine how much worse that setUp method is going to get when we add more tests or the Lolrus domain object gets more complex... Imagine the refactoring required when the Lolrus domain object acquires other constraints... Imagine the query is more complex requiring more domain objects to be set up to really prove it works...

Okay, we could mock the Lolrus class, but the code for that isn't much cleaner and alarm bells start ringing when you start using mocks in an integration test. Shouldn't this be a unit test? Sure, that would be nice but we're using the render dynamic method in our controller, we'll have to mock that out on the metaClass, mock out the findByName method on Lolrus, etc. etc. Too much hassle right? Not any more.
grails install-plugin testing
Allows us to (among other things) write controller unit test cases like this:
class LolrusControllerUnitTests extends grails.test.ControllerUnitTestCase {
def controller
def lolrus1, lolrus2

void setUp() {
super.setUp()
controller = new LolrusController()

lolrus1 = new Lolrus(name: 'Hugh')
lolrus2 = new Lolrus(name: 'Alan')
mockDomain(Lolrus, [lolrus1, lolrus2])
}

void testShowActionFindsCorrectLolrusAndSticksItInModel() {
controller.params.lolrusName = lolrus1.name
controller.show()
assertEquals('show', renderArgs.view)
assertEquals(lolrus1, renderArgs.model.lolrus)
}
}
Much neater. We're not worrying about the constraints required to set up valid Lolrus instances, but neither are we having to mock out the findByName method.

The mockDomain method allows you to set up some domain objects for which all the usual Grails domain class dynamic methods will work - including findBy..., addTo..., etc. The collection passed as the second argument to mockDomain defines all the instances that exist and they can be retrieved just as a real domain object could. The mockDomain method co-exists happily with regular Groovy mocks and the testing plugin also provides some lighter mocking capabilities via its own mockFor method.

Additionally the controller's render method and params property work without any work on our part. This is true of all the dynamic properties and methods of the controller, request, response, params, controllerName, redirect, etc.

Not only that, but this test runs faster than the integration test as it's not configuring the Spring container or an in-memory database. That speed difference only grows as the project becomes more complex - it's a couple of seconds between the two tests here, but in a more complex project it's a lot more.

There are a couple of gotchas (the plugin isn't fully mature). First off you need to remember to call super.setUp() in order that all the Grails stuff happens. The ControllerUnitTestCase class will figure out based on the class naming convention which controller class it needs to mock up. Secondly, the domain class get method doesn't seem to work with non-standard identifier types at the moment (i.e. anything other than a Long). However, this unit testing support is intended to get rolled into Grails 1.1 so it should improve rapidly.

This is only part of what the testing plugin can do. Support for unit testing domain class constraints, etc. is also included. I've also been working on a TagLibUnitTestCase that works along the same lines as ControllerUnitTestCase.

Wednesday 27 August 2008

A Productivity Hint Explained

In an earlier post, I mentioned that writing code that looks like this:

myObject."$localVariable".attribute = 42

... is a bad idea for three reasons. 

Here's the three reasons I was looking for:

1 Violating encapsulation

You don't want to manipulate the guts of other objects directly. If we made a habit of just reaching into the objects near the current chunk of code and doing whatever we want, we'd soon have a codebase which is incredibly difficult to change because whatever piece of code you're looking at now could be potentially affected by another piece of code writing over local state. 

Unit tests work by exercising the public interface of the object i.e. those services that are made available to the community of other collaborating objects in the system. If any old thing can reach in and mess around with the internal state then there wouldn't be any point in having unit tests because you can't rely on the assumptions on which those tests are based.

Every programmer learning his or her first object language used to be taught encapsulation, but I'm still amazed how many really experienced people still don't seem to take this seriously. It's not an aesthetic thing. If you don't get this please give up.

Incidentally the  Sharble and Cohen study from Boeing provides some evidence that designs that use lots of getters and setters to centralise design in a few classes also makes the code slower because you need to execute more instructions just to (needlessly) push all the data around.

2 It makes the object graph very fragile

This is about the Law of Demeter. Basically when you write the expression myObject.x.y.z.doWhatever() you're assuming that there's a whole bunch of things stuck together, at the end of which is an object that can doWhatever(). 

The code "myObject.x.y.z.doWhatever()" makes a lot of needless assumptions:

  • myObject knows x
  • x knows y
  • y knows z

... and all it really needs is to be given z as a parameter.

If any of the relationships between anObject, x, y, and z changes  we get an entirely preventable  breakage.

Some people might find it difficult to restructure this code so that you get access to z without rummaging through an object graph (what some people still seem to call a "data structure".)  See what I've got to say  below about responsibility-driven design to help with this.

3 It's a gratuitous abuse of metaprogramming.
In other words, it's slower and far more complicated that it should be. Incidentally, the business with ."$whatever" is hiding $whatever from compiler  because it needs to be evaluated at runtime rather than compile time. It also makes things an order of magnitude more difficult for a refactoring IDE. I doubt that there's an easy way for the IDE to refactor things automatically when you rename an attribute or to even notice that something's wrong when you move the attribute to another class.

anObject."$whatever" = 42

The metaprogramming here with ."$whatever" only became necessary because we're trying to poke away at the guts of another object - something we shouldn't be attempting to do in the first place. We should get anObject to do something, rather than just treat it as a passive store of data (what some people used to more honestly describe as a "record".)



At least those are the three most pressing things that spring to mind.

A funnier treatment of points 1 and 2 can be found in The Tragic Tale of POTS and his friend SPOT. Grails developers who've read about POTS will appreciate the irony of having a DogWalkingService.


How to avoid writing this kind of thing / how to fix it

The best way to avoid writing code like this is to think in terms of "responsibility-driven design" - that's the fine art of putting the data right next to the code that needs it - in the same class. 

A long time ago, people doing extreme programming knew about this. In fact, RDD was invented by the guys who invented extreme programming. Shame it's not so widely used or publicised.

Doing CRC sessions is a very agile-friendly way to do RDD - and was done on the first ever XP project C3.


A lot of this stuff seems obvious (at least points 1 and 2). It's surprising how many people don't seem to get this. 

Extreme programming is founded on the concept of lowering the cost of change. The points I've made here aren't theoretical or style-related. This is vital. This is about the basic stuff that makes everything else we do possible.

Deploy extra WARs with your Grails App

Should you find yourself needing to deploy other web-app WARs with your Grails app, when you want the Grails app under development to run at the root context, this is for you.

This applies specifically to running the app on command line (grails run-app), as compared to packaged WARs; you would not need to do this in production for Jetty since Jetty applies the logic below already - we needed this to automate tests which spanned our app and its integration with another.

First off, if you haven't already deployed your Grails app to the root context, you should refer to this.

Now that you are running your grails app at the root context, you will need to find a suitable location (MY_OTHER_WARS below) to place any additional WARs so that the Grails startup can pick up on it.

This location is referenced in the $YOUR_PROJECT_HOME/scripts/Events.groovy where you will need to add the following

eventConfigureJetty = { Server server ->

   def grailsApp = server.handler
   server.removeHandler(grailsApp)

   def handlers = []
   handlers << grailsApp
   ContextHandlerCollection contextHandlerCollection = new ContextHandlerCollection()

   warProjects.each {
      WebAppContext otherWarWebContext = new WebAppContext("${basedir}/MY_OTHER_WARS/${it}.war", "/${it}")
      handlers << otherWarWebContext
   }

   handlers.each {
      contextHandlerCollection.addHandler(it)
   }

   server.setHandlers(contextHandlerCollection)
}

This ensures that when you start your application on the command line, the default mode of adding the grails app to the Jetty server is overridden with a Jetty ContextHandlerCollection.

The problem with not doing so and just adding more Handlers to the Jetty server means that Grails intercepts all contexts/URLs on the app, and will fail to pass the call for a context onto the appropriate other WAR.

Connection Pooling

I've noticed there isn't that much information about how to setup connection pooling with grails on the mailing lists so I figured I'd put up a quick post about it. In this example I want to create c3p0 connection pools for all profiles that use postgreSQL.In resources.groovy we add the following:
import com.mchange.v2.c3p0.ComboPooledDataSource
import org.codehaus.groovy.grails.commons.ConfigurationHolder

def config = ConfigurationHolder.config

if(config.dataSource.url =~ 'postgresql') {
dataSource(ComboPooledDataSource) {
driverClass = 'org.postgresql.Driver'
user = config.dataSource.username
password = config.dataSource.password
jdbcUrl = config.dataSource.url
maxStatements = 180
minPoolSize = 10
acquireIncrement = 5
maxPoolSize = 100
}
}
and something like this to your configuration (Config.groovy or wherever you keep your profile configurations):
dataSource {
username = 'postgres'
password = ''
url = 'jdbc:postgresql://127.0.0.1:5432/databasename'
}
Fire it up with the right profile and you should see the 10 initial connections to your postgres database by running
ps -U postgres u at the console.

Can haz cuneckshun puling!

Tuesday 26 August 2008

Using Liquibase DropAll Automagically

If you're using the grails liquibase plugin you might find it useful to have it run on application startup (e.g. when the war file is deployed) rather than using the grails command line target (i.e. grails migrate). One way to achieve this is to add an entry for the liquibase spring bean in your resources.groovy like so:

liquibase(SpringLiquibase) {
dataSource = dataSource
changeLog = "classpath:migrations/changelog.xml"
executeEnabled = true
}

For certain environments (continuous build etc.) it may also be useful to have liquibase clear out the database and build the schema from scratch. The main Liquibase class has a dropAll() method that looks like it'll do the trick (there's also a 'grails drop-all' command), so let's create a spring bean that'll run before the liquibase migration kicks in (it'd be a bit pointless to blow away the database after you've just created it). The spring
'depends-on' attribute should work nicely here but it would appear that grails' BeanBuilder does not support attributes (foiled again). We can, however, implement BeanFactoryPostProcessor to achieve the desired effect:

import java.sql.Connection
import liquibase.spring.SpringLiquibase
import org.springframework.beans.factory.config.BeanDefinition
import org.springframework.beans.factory.config.BeanFactoryPostProcessor
import org.springframework.beans.factory.config.ConfigurableListableBeanFactory

class LiquibaseDropAll extends SpringLiquibase implements BeanFactoryPostProcessor {

String runBefore = 'liquibase'

void afterPropertiesSet() {
Connection conn = null
conn = getDataSource().getConnection()
super.createLiquibase(conn).dropAll()
}

//make liquibase bean depend upon this bean so that dropAll runs before migration
void postProcessBeanFactory(ConfigurableListableBeanFactory beanFactory) {
BeanDefinition beanDefinition = beanFactory.getBeanDefinition(runBefore)
beanDefinition.setAttribute('depends-on', beanName)
}

}
The last thing to do is add this bean to your resources.groovy, most probably with a conditional based on environment:

def config = org.codehaus.groovy.grails.commons.ConfigurationHolder.config
if(config.dataSource.liquibase.dropAll) {
liquibaseDropAll(LiquibaseDropAll) {
dataSource = dataSource
}
}

Now just add something like:

dataSource {
...
liquibase.dropAll = true
}

to the environments you want to liquify and you're away.

Monday 25 August 2008

Lock, Stock and Two Fuming Developers

Oh Hai!

If you have the misfortune of having to deal with locks in your postgreSQL database, run the following to find out if there are any nasty ExclusiveLocks (whole table locks) on the loose:

select * from pg_locks where mode = 'ExclusiveLock';

There are 4 types of lock in postgreSQL, listed here in order of decreasing niceness:
  • AccessShareLock
  • RowShareLock
  • RowExclusiveLock
  • ExclusiveLock
If you're running some tests that create your lock condition it's sometimes useful to quickly look at what's happening on the console:

watch -n1 ps -U postgres u

Lastly, make sure your postgreSQL version doesn't have REPLICA printed on the side :-/

Tuesday 19 August 2008

When Groovy Property Access Attacks!

A quick best-practice hint...

Groovy's property access generally leads to terser code, however there's (at least) one time when using it is not a good idea; when you want to know what class an object is.

println x.class.name may work for most types of object, but what happens if x is, hmm let's say, a Map?

You probably won't be surprised to find the following code fails.

    def x = [:]
assert x.class == java.util.LinkedHashMap

Even more entertainingly, this code will throw NullPointerException

    def x = [:]
assert x.class.name == 'java.util.LinkedHashMap'

Tuesday 12 August 2008

Overriding constraints

I didn't know you could override constraints until the other day. Here is an example of overriding and then testing the constraints with the new testing plugin. Testing constraints with the new plugin is so easy I suggest every team uses it. We started using it for our team. Here's the code:

class Parent {
String name
String body

static constraints = {
name(blank: false, nullable:false)
body(blank: false, nullable:false)
}
}

class Child extends Parent{
static constraints = {
name(blank: true)
body(blank: true)
}
}


------------Tests using the testing plugin ---------------
import grails.test.GrailsUnitTestCase
import grails.test.MockUtils

class ChildTests extends GrailsUnitTestCase{

void testConstraints() {
// Mock the validate() method.
registerMetaClass(Child)
MockUtils.prepareForConstraintsTests(Child)

registerMetaClass(Parent)
MockUtils.prepareForConstraintsTests(Parent)

def testInstance = new Child()
def errors = testInstance.validate()
assertEquals 0, errors.size()

// Test the parent constraints
testInstance = new Parent(name:' ' )
errors = testInstance.validate()
assertEquals 2, errors.size()
assertEquals "blank", errors["name"]
assertEquals "nullable", errors["body"]
}
}

Happy testing :)

Monday 11 August 2008

Today's Object-Oriented Productivity Hint

Please try using encapsulation before attempting metaprogramming or functional programming.

Exercise:

Find three reasons not to write this kind of code:

myObject."$localVariable".attribute = 42

This kind of code is wrong on so many levels that if Ali G wrote web apps, he'd love it.


Last week's Retrospective

On Wednesday I ran my first retrospective. I decided to focus on some of the positive values that make up a good team: Simplicity, Feedback, Courage, Respect and Communication.

Here is a link to my write up of the retrospective.

Sending emails from Grails apps

We're trying this on for size...

import org.springframework.mail.MailSender
import org.springframework.mail.SimpleMailMessage

class EmailService {

boolean transactional = false
MailSender mailSender
SimpleMailMessage mailMessage // A template message auto-wired from resources.groovy

boolean sendMessage(String to, String subject, String body) {
boolean result = true
try {
SimpleMailMessage message = new SimpleMailMessage(mailMessage)
message.to = [to]
message.subject = subject
message.text = body
mailSender.send(message)
} catch(Exception e) {
log.error("Unable to send message to ${to} with subject ${subject}", e)
result = false
}
return result
}
}

import org.springframework.context.MessageSourceAware
import org.codehaus.groovy.grails.web.pages.GroovyPagesTemplateEngine
import org.springframework.context.MessageSource
import org.springframework.context.i18n.LocaleContextHolder
import org.codehaus.groovy.grails.web.pages.GroovyPageTemplate

class NotificationService implements MessageSourceAware {

EmailService emailService
GroovyPagesTemplateEngine groovyPagesTemplateEngine
MessageSource messageSource

boolean confirmRegistration(Person person) {
String subject = getMessage('email.registration.confirmation.subject', [])
return emailService.sendMessage(person.email, subject, getEmailBody('registrationConfirmation', [person: person]))
}

boolean registrationSuccess(Person person) {
String subject = getMessage('email.registration.successful.subject', [])
return emailService.sendMessage(person.email, subject, getEmailBody('registrationSuccessful', [person: person]))
}

private String getEmailBody(String templateName, Map model) {
return render("/modules/emails/_${templateName}.gsp", model)
}

private String render(String templateUri, Map model) {
// I wonder if templates can be cached
GroovyPageTemplate template = groovyPagesTemplateEngine.createTemplate(templateUri)
Writable w = template.make(model)
StringWriter sw = new StringWriter()
PrintWriter pw = new PrintWriter(sw)
w.writeTo(pw)
return sw.toString()
}

private String getMessage(String code, List args) {
return messageSource.getMessage(code, (Object[]) args, LocaleContextHolder.locale)
}

void setMessageSource(MessageSource messageSource) {
this.messageSource = messageSource
}
}


Few gotcha's though...

1. TagLibs don't work in the GSP templates. This causes a real headache if you need to use <g:message ... /> etc.

2. I'm not sure whether the templates can be cached. It's probably expensive to keep creatating them each time.

Testing

We're using wiser to stub the SMTP server, e.g.

import org.subethamail.wiser.Wiser
import org.subethamail.wiser.WiserMessage

class EmailServiceTests extends GroovyTestCase {

Wiser wiser
EmailService emailService

void setUp() {
wiser.messages.clear()
}

void testThatICanSendAnEmail() {
assertEquals(0, wiser.messages.size())
emailService.sendMessage('roundhouse@beta.xyz.com', 'Cool beans', 'Well done, you managed to register')
assertEquals(1, wiser.messages.size())

WiserMessage message = wiser.messages.iterator().next()
assertEquals('roundhouse@beta.xyz.com', message.envelopeReceiver)
assertEquals('Cool beans', message.mimeMessage.subject)
assertEquals('Well done, you managed to register', message.mimeMessage.content)
}

void testThatEmailServiceReturnsFalseOnFailure() {
emailService.log.logger.setLevel(org.apache.log4j.Level.OFF)
wiser.stop()
assertFalse('Unexpected successful email send', emailService.sendMessage('roundhouse@beta.xyz.com', 'subject', 'message'))
wiser.start()
emailService.log.logger.setLevel(org.apache.log4j.Level.ERROR)
}
}

We also use wiser in our selenium tests by adding the following action to our test fixture controller,


def smtp = {
List messages = wiser.messages.collect { message ->
String to = message.envelopeReceiver
String from = message.envelopeSender
String subject = message.mimeMessage.subject
String content = message.mimeMessage.content
return [to: to, from: from, subject: subject, content: content]
}
Map model = getJsonModel(messages)
render(view:'index', model: model)
}


Config

It's easy to pick whether to use the real smtp server or wiser in resources.groovy

switch(GrailsUtil.environment) {
case 'prod':
mailSender(org.springframework.mail.javamail.JavaMailSenderImpl) {
host = 'mail1.xyz.com'
}
mailMessage(org.springframework.mail.SimpleMailMessage) {
from = 'info@ xyz.com'
}
break
case ['qa', 'beta']:
mailSender(org.springframework.mail.javamail.JavaMailSenderImpl) {
host = 'mail1.xyz.com'
username = 'info@beta.xyz.com'
password = 'shh'
}
mailMessage(org.springframework.mail.SimpleMailMessage) {
from = 'info@beta.xyz.com'
}
break
default:
wiser(org.subethamail.wiser.Wiser) { bean ->
bean.initMethod = 'start'
bean.destroyMethod = 'stop'
bean.lazyInit = true
port = 2500
}
mailSender(org.springframework.mail.javamail.JavaMailSenderImpl) {
host = 'localhost'
port = 2500
}
mailMessage(org.springframework.mail.SimpleMailMessage) {
from = 'info@beta. xyz.com'
}
break
}

Tuesday 29 July 2008

Where did the Response go in 1.0.3?

We noticed that in the move from Grails 1.0.2 to 1.0.3, the response object inside Spring's RequestContextHolder has seemingly gone but it looks like it's always been somewhere else:

So:

def response = org.springframework.web.context.request.RequestContextHolder.requestAttributes.response

Becomes:

def response = org.springframework.web.context.request.RequestContextHolder.requestAttributes.currentResponse



Worth keeping an eye out for.

Thursday 24 July 2008

Experimental testing plugin

Saw this post on the grails mailing from a grails committer, Peter Ledbrook, might be worth checking out:

Hi everyone,

I have been working on some improvements to the Grails testing
framework recently which will go into Grails 1.1. Some of the support
classes though will work with Grails 1.0.x, so I have packaged them up
as a plugin. You can install it using:

grails install-plugin testing

Before you rush to download it, please be warned that it is nowhere
near a complete implementation yet, particularly
ControllerUnitTestCase. However, I would like to solicit feedback
early on so that we can really nail the problems that people are
having in testing. And if you can supply patches, great!

So, what lurks in the plugin? I'm planning to get some documentation
up soon, but for now I'll cover some common use cases. Before that,
the current set of classes are designed to be used in unit tests. As a
general trend, we want to encourage people to write unit tests in
preference to integration tests. We also want to make sure that unit
tests can be run from within an IDE, i.e. there should be no
requirement on a running Grails instance.

For the first example I'll show you how to test domain constraints.
Domain classes often lack logic, and so they don't get tested.
However, plenty of errors can creep in to the constraints so it's
worth validating them.

class MyDomain {
String name
Integer age

static constraints = {
name(nullable: false, blank: false)
age(nullable: false, min: 10, max: 100)
}
}

class MyDomainUnitTests extends GrailsUnitTestCase {
void testConstraints() {
// Mock the validate() method.
registerMetaClass(MyDomain)
MockUtils.
prepareForConstraintsTests(MyDomain)

// Test that a fresh new domain instance fails validation on
// the "nullable: false" constraints.
def testInstance = new MyDomain()
def errors = testInstance.validate()
assertEquals 2, errors.size()
assertEquals "nullable", errors["name"]
assertEquals "nullable", errors["age"]

// Test the other constraints
testInstance = new MyDomain(name: " ", age: 5)
errors = testInstance.validate()
assertEquals 2, errors.size()
assertEquals "blank", errors["name"]
assertEquals "min", errors["age"]
}
}

The main things to note here are:

1. We sub-class GrailsUnitTestCase
2. "registerMetaClass()" and "MockUtils.prepare...()" add the
validate() method to the domain class
3. We create instances of the domain class, call validate(), and check
whether any errors were found

On (2), the two lines will be replaced by a method on
GrailsUnitTestCase in the near future. On (3), the validate() method
returns a map of validation errors. Note that we check for the name of
the constraint, not the i18n error code associated with the
constraint.

That's it for domain constraints. For other unit tests, such as for
services and controllers, GrailsUnitTestCase provides the method
"mockDomain(Class, List)":

void testMethod() {
mockDomain(MyDomain, [
new MyDomain(name: "John Smith", age: 35),
new MyDomain(name: "Alice Smith", age: 64),
new MyDomain(name: "Irene Pane", age: 22),
new MyDomain(name: "Patrick Rose", age: 45) ])

def testService = new MyService()
testService.doSomething()
...
}

The method injects working versions of the dynamic methods and
properties that normally go with domain classes, in particular the
dynamic finders. Where appropriate, these injected methods/properties
use the given list of domain instances as a source of data. For
example, if MyService.doSomething called a dynamic finder like this:

MyDomain.findByNameLike("%Smith")

the mock property would return a list containing the "John Smith" and
"Alice Smith" MyDomain instances in that order. Another useful method
is "mockFor()" which returns an object that you can use pretty much
like the Groovy MockFor class:

def mockControl = mockFor(MyDomain)
mockControl.demand.save(1..1) {-> return true}
mockControl.demand.static.findByName(1..1) { name -> return [] }

The best thing about this method is that it works seemlessly with
"mockDomain()", i.e. you can readily override the methods provided by
"mockDomain()" via the mock object returned by "mockFor()".

Finally, there is a ControllerUnitTestCase, but it is in the very
early stages of development. I recommend you only use it if you're
willing to patch it up with the functionality you need. It will
automatically inject all the normal controller properties and methods,
but not much else. However, one nice feature I have implemented
already is the ability to set the body of the request to some XML
(either a string or builder markup), particularly useful for REST
controllers based on XML. I need to add support for JSON too.

That's it for now. I would certainly give GrailsUnitTestCase a go
because I have already found it much easier to write Grails unit tests
than I used to. If you want to raise issues or provide patches, please
add them to the main Grails JIRA, setting the fix version to 1.1 and
assigning them to me (username "pledbrook").

Cheers,
Glenn

Saturday 19 July 2008

Does the noise in my head bother you?

A quick post for those of you using the dynamic addTo* methods with manyToMany relationships under grails 1.0.3. You may notice alot of noise in your logs / test results - this is down to a stray println in DomainClassGrailsPlugin.groovy (line 173) which does the following:

println "$obj - ${obj.properties} - ${prop.otherSide.name}"

Options: patch grails, wait for 1.0.4, ignore it.

Tuesday 15 July 2008

Tricking Grails into persisting Enum properties

Although Grails 1.0.3 implements persistence of java.lang.Enum fields in GORM, unfortunately it sticks with the misguided Hibernate convention of persisting the Enum ordinal rather than the String value. In Hibernate at least you can override this default, in Grails currently you can't.

In the meantime, here's a workaround pattern that should be amenable to a low-impact refactor once Grails 1.0.4 drops.

    enum Flavr { /* some enum instances */ }

class Lolcat {
String flavrName

static transients = ['flavr']

static constraints = {
flavrName(nullable: true, inList: Flavr.values().collect{it.name()})
}

static mapping = {
columns {
flavrName(name: 'flavr')
}
}

Flavr getFlavr() {
return flavrName ? Flavr.valueOf(flavrName) : null
}

void setFlavr(Flavr flavr) {
flavrName = flavr?.name()
}
}


What we've done is create a transient Enum property backed by a String that GORM uses to persist the value. The constraint prevents any erroneous values being set directly to the String property. The column name mapping is an attempt at future proofing, so the column name won't need to be changed once 'flavr' becomes a full-blown persistent Enum.

As and when that's possible you simply replace 'String flavrName' with a 'Flavr flavr' and remove the transients, constraints and mapping blocks and the getter and setter.

Two downsides are 1) that dynamic finders and criteria queries must operate on real properties and not transient ones and 2) that you can pretty much guarantee some muppet will directly access 'flavrName' somewhere - something you can guard against much more effectively in Java than you can in Groovy.

Thursday 10 July 2008

Groovy operators

Max suggested I post something about this, so...

Groovy has some operators that vanilla Java doesn't. Here's a quick reference:

<=> (Spaceship)
Example:
a <=> b is equivalent to a.compareTo(b)

?: (Elvis)
Example:
x ?: y is equivalent to x != null ? x : y

=~ (Find)
Example:
Matcher m = "abc" =~ /a/

==~ (Match)
Example:
assert "abc" ==~ /\w+/ Note this is a full match not a partial.

~ (Create pattern)
Example:
~/abc/ is equivalent to new Pattern(/abc/)

*. (Spread)
Example:
a*.b is equivalent to a.collect { it.b }

.& (Method reference)
Example:
Closure c = a.&b gives you a reference to the method "b()" on object "a" that you can pass around as a Closure.

.@ (Property access)
Example:
def c = a.@b gives you the value of the property "b" on object "a" directly (i.e. without going through the getter). This can be useful on horrible classes like java.awt.Dimension where the type of a public property is different to the return type of its getter. Most other uses of it are probably best avoided.

?. (Null-safe dereference)
Example:
a?.b is equivalent to a == null ? null : a.b

as (Type coercion)
Example:
new Date() as String is equivalent to new Date().toString() which is a trivial example but you can do quite exciting type conversions by overriding asType.

is (Object identity)
Example:
a is b is equivalent to a == b in Java.

kthxbye.

Tuesday 8 July 2008

Ooops Mocks

Why do this ...
void testOne() {
def result
mockServiceA.use {
mockServiceB.use {
mockServiceC.use {
result = testMe.someMethod('a', 'b')
}
}
}
assertEquals(1, result)
}

void testTwo() {
def result
mockServiceA.use {
mockServiceB.use {
mockServiceC.use {
result = testMe.someOtherMethod('a', 'b')
}
}
}
assertEquals(1, result)
}


When you can do this...

void testOne() {
def result = executeWithMocks { testMe.someMethod('a', 'b') }
assertEquals(1, result)
}

void testTwo() {
def result = executeWithMocks { testMe.someOtherMethod('a', 'b') }
assertEquals(1, result)
}

private Object executeWithMocks(Closure closure) {
def result
mockServiceA.use {
mockServiceB.use {
mockServiceC.use {
result = closure()
}
}
}
return result
}

Friday 4 July 2008

Criteria queries + associations = confusion


Since we've been bitten by this problem twice in the same week now I thought it was worth a post.

Consider these classes:
    class Author {
String name
static hasMany = [books: Book]
}

class Book {
String name
static belongsTo = [author: Author]
}

Then let's attempt to find all authors who have written a book with a particular name:
    Author wg = new Author(name: 'William Gibson')
wg.addToBooks new Book(name: 'Pattern Recognition')
wg.addToBooks new Book(name: 'Spook Country')
assert wg.save(flush: true)

sessionFactory.currentSession.clear()

List authors = Author.withCriteria {
books {
eq('name', 'Spook Country')
}
}
assertEquals 1, authors.size()
assertEquals 2, authors[0].books.size() // FAILS result is 1

When you look at how Hibernate implements this query under the hood it's reasonably clear what's going on. It will be doing something roughly along the lines of:
    select * from author a, book b where a.id = b.author_id and b.name = ?

The problem is it then makes the mistake of thinking that the books collections of the author objects it has found can be initialised based on that result set.

Where this problem has bitten us is when objects fetched by such a query are also used by an unrelated bit of code that doesn't bother reading from the database as it quite reasonably thinks (some of) the objects is needs are in the Hibernate session already. Unfortunately, the view module related to this other bit of code is reliant on the elements of the collection that weren't loaded by our defective query. I'm sure you can imagine the fun to be had debugging that kind of problem.

I've been hacking around with the criteria attempting various permutations such as:
    // DOES NOT WORK
List authors = Author.withCriteria {
'in'('books.name', 'Spook Country')
}

and
    // ALSO DOES NOT WORK
List authors = Author.withCriteria {
'in'('books') {
eq('name', 'Spook Country')
}
}

But I've got nowhere yet. The best I've managed to do is:
    List authors = Author.withCriteria {
books {
eq('name', 'Spook Country')
}
}
// if you have a weak stomach look away now
authors*.refresh()

If you're wondering, yes that is going to execute n+1 selects.

Another option is to give up and use an HQL query.

This problem applies equally to one-to-many (and presumably many-to-many) associations using Set, List or Map.