STATE YOUR BIZNESS

Techie Books Suggestions

2011-04-07T22:47:00.001+01:00

(Or "How I Learned To Stop Worrying About Gramatically Correct Blog Titles")

I'm starting to add to the dev side of my techie library which, as you can appreciate, is pretty biased toward sysadmin stuff at the moment (DNS, firewall, exim, Jabber and so on).

So far I've added the two GinAs, and recently jQuery in Action, Javascript: The Good Bits, Javascript: The Definitive Guide and The Java Developer's Guide to Eclipse.

So what else should I have on my wish list? Freely downloadable is good too - it doesn't have to be expensive dead-tree format - especially if it can be Kindleised.

Would love to hear your thoughts and suggestions.

Remote Controlled Geb Functional Tests

2011-02-22T13:11:00.017Z

I've just started on a new project where Geb and Spock are our main functional testing tools. They're nice tools to work with but sometimes it's hard to write tests quickly when you can't see what's going on. I want to be able to step through tests like in Selenium IDE but without all the shortcomings of using fixture controllers.

One way to develop functional tests under Grails is to keep the app running while you launch tests from another jvm and use -baseUrl to target the remote jvm. You can do this from the command line or use the Functional Test Development plugin. Another way is to keep the app running and run the tests from within an IDE like IntelliJ; a bonus is that you can run individual tests/features which is great if you have a fairly large spec.

Whichever way you choose, you're going to need to setup data on the target jvm. On my last project we used fixture controllers that we would call from our functional tests but these grew into thousands of lines because re-use after a while became difficult. A better way to do it is to bundle the test data with the actual test which is what the Remote Control plugin allows us to do - the plugin documentation has a better explanation of what I mean.

With all that said, what I want to be able to do is:

Run my app up
Create my data in my setup block as though I was in an integration test.
Right click on the test from intellij and hit run (or use debug and step through line by line)

Take a look at the plugin docs, it's pretty straight forward but we had to override getBaseUrl and create our own BuildSettingsHolder because remote control doesn't like running outside of a grails command line. So our parent Spec looked like:

public ParentSpec extends GebSpec {
  String getBaseUrl() {
       "http://localhost:8080/monkeytails/"
  }

   def setupSpec() {

       if (BuildSettingsHolder.getSettings() == null) {
           BuildSettings bs = new BuildSettings()
           bs.metaClass.getFunctionalTestBaseUrl = { getBaseUrl() }

           BuildSettingsHolder.setSettings(bs)
       }
   }

   def cleanupSpec() {
       resetDatabase()
   }

   void resetDatabase() {
       def remote = new RemoteControl()
       remote {
           User.list().each{ it.delete(flush:true) }
           // Other teardown stuff goes here
           // ...
           return true
       }
  }
}

and then your tests:

    def "Test something"() {

      setup:
          def remote = new RemoteControl()
          remote {
              User user = new User()
              user.save(flush: true)
              return true //have to return something serializable
          }
      .
      .
  }

It's handy to have firebug in the firefox that webdriver opens so use the profile manager to create a profile called test and then change your IntelliJ JUnit Run configuration VM Parameters to include '-Dgeb.driver=firefox -Dwebdriver.firefox.profile=test'

Now you when you right click and run your test, IntelliJ will first compile everything and hopefully run your test. I say hopefully because I have a bunch of inline plugins so I have to exclude them from the compile path and hope for the best. Setting breakpoints and choosing debug lets you step through your test and let you can use firebug within the browser.

There's a few things to be aware of. Groovy's property setters don't seem to work so you can't do stuff like user.username, instead you have to use the setter or if there isn't one you can use setProperty(). Spread operators don't work too well either, so you can't do User.list()*.delete().

Update: forgot to mention that you should run the app in test mode or Remote Control will not set up its listener unlesss you set remoteControl.enabled to true in your app config

Testing cron expressions in Grails Jobs

2010-12-07T13:18:00.002Z

One big problem I have is I can never remember what the cron expression is supposed to be and once I've got it in the code if it is actually correct.

Very simple test below. I create a CronTrigger and pass in the expression from the job. I then check that it will fire by passing in a Calendar object. The only key here is you can't test something in the past hence why I am adding another day to the LocalDate object.

void testJobSchedule() {

def trigger = new CronTrigger("name", "group", JobName.cronExpression)

assertTrue "job should fire at 1:15", trigger.willFireOn(time(1, 15))

}

private Calendar time(int hour, int minute) {

new LocalDate().plusDays(1).toDateTime(new LocalTime(hour, minute, 0, 0)).toCalendar(Locale.UK)

}

Hopefully this helps someone out.

Unexpected search results

2010-11-19T09:58:00.001Z

Bit of friday fun - just tried searching for Node GString, and Google helpfully suggested...

Did you mean: nude g string

Using Liquibase DropAll Automagically 2

2010-11-19T00:44:00.003Z

A while back gus posted about how to get liquibase to drop all on application start up. He had to jump through some hoops because dependsOn wasn't working properly. This seems to be OK now so I think the process can be simplified a bit...

Assuming the changelog is specified in grails-app/conf/liquibase/master.xml

resources.groovy

if (config.liquibase.on) {
   liquibaseDropAll(LiquibaseDropAll) { bean ->
      dataSource = dataSource
      changeLog = "classpath:liquibase/master.xml"
      bean.initMethod = 'init'
   }

   liquibase(SpringLiquibase) { bean ->
      dataSource = dataSource
      changeLog = "classpath:liquibase/master.xml"
      bean.dependsOn = ['liquibaseDropAll']
   }
}

LiquibaseDropAll.groovy

import liquibase.spring.SpringLiquibase
import org.codehaus.groovy.grails.commons.ConfigurationHolder

class LiquibaseDropAll extends SpringLiquibase {

   void init() {
      if (ConfigurationHolder.config.liquibase.dropAll) {
         super.createLiquibase(dataSource.connection).dropAll()
      }
   }

}

Disclaimer: I've only just started using it so it may still thow up some gotchas.

Reducing backend load

2010-10-17T12:00:00.003+01:00

I've recently been looking at alternatives to F5 and I remember from this year's QCon a lot of people mentioning Zeus so I thought I'd give them a look.

So far they look promising; being able to run Zeus on your own generic hardware could be a cost saving in the long run. For example we had to buy some spare PSUs out of fear we wouldn't be able to buy them in the future.

One feature I really like the sound of is the "webcache!refresh_time". It smooths out the load to your backend by only sending one request to your appserver while serving the rest from cache. For sites that must have low cache times this makes a lot of sense. If you get 30 requests per second for an item, you only send one request to your backend as it expires from cache.

Turns out F5 also have a similar sounding feature called Fast Cache but for us it would be an additional module (read additional cost).

If anyone has experience with either I'd love to hear their thoughts.

Stubbing g.message

2010-09-23T08:11:00.002+01:00

It's easy to mock grails taglibs using gmock and hamcrest. You can do something like...


    ...
    def g

    void setup() {
        g = mock()        
        mock(tagLib).getG().returns(g).stub()
    }

    void someTest() {
        g.message(hasEntry('code', 'foo.bar')).returns('FOO BAR')
        String result
        play {
            result = tagLib.someMethod([:]).toString()
        }
        assert result.contains('FOO BAR')
    }
}

But when you've got a lot of calls to g.message this can become noisy and usually ends up being repeated in multiple tests. One option is to be relax the argument matching and switch to using a stub.


        g.message(instanceOf(Map)).returns('FOO BAR').stub()

This is fine for the tests where you don't really care about the message, but too loose for the ones you do. I'm playing with the an alternative which gives you both stubbing and the option of explicitly asserting g.message was called with the correct arguments. Be warned though it may raise a WTF exception on first glance.


    ...
    def g

    void setup() {
        g = mock()        
        mock(tagLib).getG().returns(g).stub()
        stubMessages(g)
    }

    void someTest() {
        String result
        play {
            result = tagLib.someMethod([:]).toString()
        }
        assert result.contains('code:foo.bar')
    }
}

class MessageMatcher extends BaseMatcher {

    String code = ''

    static void stubMessages(def g) {
        MessageMatcher matcher = new MessageMatcher()
        g.message(matcher).returns(matcher).stub()
    }

    boolean matches(Object o) {
        code = ((Map) o).containsKey('code') ? o.code : ''
        return true;
    }

    void describeTo(Description description) { }

    String toString() {
        return "code:${code}"
    }
}

The tricky bit was getting the g.message stub to return a value derived from the matcher arguments. I couldn't just do


g.message(matcher).returns(matcher.code).stub()

because matcher.code won't have a value at this point. The solution is to override the matcher's toString() method to return the code and rely on groovy / grails invoking toString() when adding the matcher to the output. This certainly violates the rule of least surprises, but I think I'm OK with that if it reduces duplication and the noise level of my tests - at least until I find a better way.

Clean TagLib Tests

2010-09-17T14:48:00.007+01:00

Despite my best efforts I've always found it hard to write clean taglib tests. Now thanks to Spock and GroovyShell things are getting easier...


    void setup() {
        mockDomain Invoice
    }

    def "Attachments icon has correct markup"() {
        given:
        Invoice invoice = new InvoiceBuilder().buildAndSave()

        when:
        renderAttachmentsIcon([target: invoice])

        then:
        valueOf('img.@id') == "toggle-attachments-${invoice.id}"
        valueOf('script') == "\$('#toggle-attachments-${invoice.id}').bind('click', Books.attachments.toggle);"
    }

Because I want to use GPath make assertions about the resulting HTML I've overriden TagLibSpec's methodMissing closure as follows...


    def methodMissing(String name, args) {
        String html = super.methodMissing(name, args)
        createDocument(html)
    }
    
    void createDocument(String html) {
        String xml = "<results>${html}</results>"
        document = new XmlSlurper().parseText(xml)
    }

And added helper methods for evaluating GPath expressions...


    String valueOf(String gPath) {
        evaluate(gPath).text()
    }

    GPathResult evaluate(String gPath) {
        new GroovyShell(new Binding(document: document)).evaluate("document.${gPath}")
    }

There's a little bit more to this story unfortunately...

Firstly my taglibs use the MarkupBuilder and when run from my Spock test this only outputs the opening tag of the first element I genererate. I haven't had a chance to look into this yet, but a workaround is to add "out << '' to the end of the taglib method

Secondly Spock interactions aren't yet as powerful as gmock, so I usually end up adding code to (g)mock grails taglibs.

Thirdly another one of my tests outputs   in the HTML which causes the XML parsing to barf. The solution is to map the &nbsp entity to a known character (in this case underscore).

Finally there was a bug in Grails 1.3.3 / Spock 0.5 which breaks mockDomain. This is reportedly fixed in 1.3.4 and the latest Spock code, but I haven't tried upgrading yet.

Here's how things really look...


@WithGMock
class MetaAttachmentsTagLibSpec extends TagLibSpec {

    def g
    def document

    void setup() {
        g = mock()        
        mock(tagLib).getG().returns(g).stub()
        mockDomain Invoice

        PluginManagerHolder.pluginManager = [hasGrailsPlugin: { String name -> true }] as GrailsPluginManager // Workaround for JIRA GRAILS-6482       
    }

    def cleanup() {
        PluginManagerHolder.pluginManager = null // Workaround for JIRA GRAILS-6482
    }

    def "Attachments icon has correct markup"() {
        given:
        Invoice invoice = new InvoiceBuilder().buildAndSave()
        g.resource(instanceOf(Map)).returns '/foo.jpg'

        when:
        renderAttachmentsIcon([target: invoice])

        then:
        valueOf('img.@id') == "toggle-attachments-${invoice.id}"
        valueOf('img.@src') == "/foo.jpg"
        valueOf('script') == "\$('#toggle-attachments-${invoice.id}').bind('click', Books.attachments.toggle);"
    }
    
    def methodMissing(String name, args) {
        String html
        play {
            html = super.methodMissing(name, args)
        }
        createDocument(html)
    }

    def createDocument(String html) {
        String xml = """<!DOCTYPE html [<!ENTITY nbsp "_">]>\n<results>${html}</results>"""
        document = new XmlSlurper().parseText(xml)
    }
}

And the method under test...


    def renderAttachmentsIcon = { Map attrs, def body ->
        String targetId = attrs.target.id
        String iconId = "toggle-attachments-${targetId}"
        String imgSrc = g.resource(dir:'/images/skin', file:'paperclip.png')

        MarkupBuilder builder = new MarkupBuilder(out)
        builder.img(id: iconId, src: imgSrc)
        builder.script(type:'text/javascript') {
            mkp.yield "\$('#${iconId}').bind('click', Books.attachments.toggle);"
        }
        out << '' // flush for unit tests        
    }

Internet Explorer and the 1 item remaining bug

2010-09-02T14:08:00.004+01:00

We've started using Selenium 2 / WebDriver to soak test our application, and were caught once again by a bug in ie7 and ie8 that randomly causes the browser to hang, waiting for some resource to finish loading, even though the page is visible and works fine.

I hit the problem a year or so ago and solved it by hacking something in prototype.js, but can't remember exactly what. Thankfully there's now a more recent version of prototype (we're stuck on grails 1.1.1 which is bundled with prototype 1.6.0), so I upgraded to prototype 1.6.1 and problem solved :)

Grails and unit#junit;4.8.1: configuration not found in junit#junit;4.8.1:

2010-08-07T09:33:00.004+01:00

After upgrading to grails 1.3.x I started getting the following error...


  ::::::::::::::::::::::::::::::::::::::::::::::

  ::          UNRESOLVED DEPENDENCIES         ::

  ::::::::::::::::::::::::::::::::::::::::::::::

  :: junit#junit;4.8.1: configuration not found in junit#junit;4.8.1: 'master'. It was required from org.grails.internal#Books;0.1 test

  ::::::::::::::::::::::::::::::::::::::::::::::

A found a few posts suggesting I needed to clear my ivy cache, but that did sfa. Turned out to be something in the jawr 3.3.2 plugin. It went away when I excluded junit from compile scope in the plugins dependencies.groovy


    dependencies {
        // specify dependencies here under either 'build', 'compile', 'runtime', 'test' or 'provided' scopes eg.
     compile ('net.jawr:jawr:3.3.2') {
      excludes "mail", "activation", "ejb","jms","jmxri","jmxremote", "junit"
 }

IntelliJ v9.0.3 adds support for Grails v1.3.x

2010-07-23T15:52:00.002+01:00

JetBrains released IntelliJ IDEA v9.0.3 two days ago, which includes support for Grails v1.3.x, along with nicer SVN merging.

See more details here:

http://www.jetbrains.com/idea/whatsnew/index.html

AppStatus Plugin 0.1 Released

2010-05-02T10:41:00.003+01:00

The AppStatus plugin enables you to easily display useful info about your application.

Install the plugin and navigate to http://yourapp/appStatus to see application.properties, server time and locale.

If you want to see more details you can specify the "providers" in resources.groovy

import uk.co.acuminous.app.status.provider.*
import uk.co.acuminous.app.status.AppStatusConfig

beans = {
    appStatusConfig(AppStatusConfig) {
        providers = [

            // !!!DANGER WILL ROBINSON!!!
            // The ApplicationConfigProvider will dump the entire
            // contents of Config.groovy including datasource.password
            // The SystemPropertiesProvider could also make available
            // potentially sensitive system variables

            properties: new ApplicationPropertiesProvider(),
            config: new ApplicationConfigProvider(),
            system: new SystemPropertiesProvider(),
            locale: new LocaleProvider()
        ]
    }
}

If you want to add your own providers just write something that implements the Provider interface and add it to the providers map.

Finally if you're already using jQuery and jQuery UI you pick an nicer view

beans = {
    appStatusConfig(AppStatusConfig) {
        providers = [
            // blah
        ]
        view = 'tabs'
    }
}

Pure Genius and mod_jk

2010-04-24T06:40:00.010+01:00

I'm currently developing an application for MegaBank which makes heavy use of AJAX. The latest cut was throwing javascript errors in UAT but not in development. A bit of investigation with firebug showed that when an AJAX request returned 404 (which it does if the user supplies a non-existent cost code for example), the next AJAX request would respond with the SSO login page instead of the expected json response. Further investigation showed that the second AJAX response had been served by a different node than the one it was sent to, resulting in the SSO challenge.

Request had a cookie:
jsessionid=ASAD123DFDFS83242SDFASD9234234.node1

Response has a set-cookie header:
jsessionid=SDFSDF234DFSLFSD324234880SDFSDL.node2

i.e. our sticky session was becoming unstuck.

We reported the incident to our support team, who's spanked it straight back over the net with the wonderfully helpful "It's an application config issue". Great. Thanks. Our app doesn't do anything clever with cookies. There is no logout button. We never invalidate the session. We'd already checked that inbound request had the right jsession id, and also that the associated response had a set-cookie header from the wrong node. Furthermore the app server logs show that the second request didn't even hit the right node. Back to web support with an offer to demonstrate the problem and walk through why we believe it's an issue with the load balancer. Even if it turned out to be an issue with the app we would need their help to diagnose it.

Offer accepted. And "Pure Genius" arrives (That's not his real name, but it is what was written on his cuff links). The first words out of his mouth were, "We support 100s of applications and the load balancer works fine. It must be your application". It doesn't mater that this is a bank, where 99% of the applications he supports are legacy and have probably never heard of AJAX. So I walk him through the process, show him the HTTP headers, show him the logs, explain that this only seems to happen after a 404. He goes away and helpfully reports...

"It's an application configuration issue."

Grrrrrrrr. Escalate. Now engineering areinvolved. Helpfully "Pure Genius" has already told them it's a problem with our app, so they've they've prioritised it to the bottom of their queue. Grrrrrrrr. Another round of the escalation game follows and we get someone from engineering who knows what they're doing and isn't wearing blinkers. Guess what? There's a bug in mod_jk which causes it to fail a session over if it gets a non 200 error code with no content, so we added the words "Pure Genius" to the response and we have sticky sessions again. I guess he helped after all.

Javassist Enhancement failed

2010-02-18T11:02:00.002Z

Just spent the last few hours debugging this.

def adoptChild = {
   Parent p = Parent.get(params.id)
   p.child = Child.findByName(params.name)
   assert p.save(flush:true)
   redirect(action: 'showParentWithChild', id: p.id)
}

def showParentWithChild() {
   Parent p = Parent.get(params.id) // Javassist Enhancement failed stack trace here
   render("OK")
}

Turned out that the child class had a private constructor (no idea why). I ditched the constructor and everything was back to normal.

Issues with Eclipse 3.5.1 and groovy plug-in

2010-01-26T15:56:00.003Z

For all you Linuxy peeps.

After installing Eclipse 3.5.1 (SDK version) I installed all the plugins I usually use but when I installed the groovy v2 plugin all my existing plugins disappeared along with the groovy one. Eclipse insisted that they were installed but they weren't there.

It seems that this combination doesn't like "shared installs". My eclipse installation was in /usr/local/eclipse and I was installing the groovy plugin as my regular user. Fail.

There are two solutions.

1 - install a local copy of eclipse.

Simple but it means an eclipse install per user on your machine.

2 - install the groovy plugin as root.

Means that you have to run it as root every time you want to upgrade the plugin.

Choose whichever evil is lesser for you. (I chose #2).

BTW you may also need this magic shell script line

export GDK_NATIVE_WINDOWS=true

if you find that some of the buttons in eclipse don't like being clicked on. Apparently this is due to be fixed in eclipse 3.5.2.

Hope this saves someone a few frustrated hours.

Ham and Chips

2010-01-08T17:12:00.006Z

Rob Fletcher's blog post on gMock inspired me to try out hamcrest, but I was surprised to find out there's no simple way to match collections in mocked method arguments - the best I could come up with was this...

mock(tagLib).fieldLabel(instanceOf(Map), instanceOf(Closure)).returns("<label></label>")

which doesn't tell me whether I'm calling fieldLabel with the expected attributes. After a bit of searching I found the MapContentMatcher in the Spring Integration library, but didn't want include the whole jar just for that. It also had dependencies other libraries and was heavily typed with generics. Here's the cut down groovy version...

import org.hamcrest.*
import static org.hamcrest.core.AllOf.*
import org.hamcrest.Factory as HamcrestFactory

class MapContentMatcher extends TypeSafeDiagnosingMatcher<Map> {

    Object key
    Object value
    
    MapContentMatcher(Object key, Object value) {
     this.key = key
        this.value = value
    }
    
    public boolean matchesSafely(Map actual, Description mismatchDescription) {
     return actual.containsKey(key) && value == actual[key]
    }
    
    public void describeTo(Description description) {
     description.appendText("$key:$value")
    }
    
    @HamcrestFactory
    public static Matcher hasAllEntries(Map expected) {
     return allOf(expected.collect { k, v ->
      new MapContentMatcher(k, v)
        })
    }    
}

Which can be used thus...

mock(tagLib).fieldLabel(hasAllEntries([id: expectedField, label:expectedHeading]), instanceOf(Closure)).returns("<label></label>")

Sitemesh layout applied to AJAX response (gotcha)

2010-01-07T07:28:00.006Z

Grails supports multiple ways to specify a sitemesh layout for your responses. One way I wasn't familiar with is by convention, i.e. if you have a controller 'BookController' and a layout in /views/layouts called book.gsp all the actions in BookController will be rendered with the book layout by default - even if you do render("some text") or render(template: "/sometemplate").

So if you're wondering why all your AJAX responses are getting wrapped up in a sitemesh template this could be the reason. You can stop this by changing the content type to text/plain or by renaming the layout and specifying it explicitly in your gsps using <meta name="layout" content="layoutname"/>.

Testing grails scripts

2009-10-07T08:41:00.002+01:00

Not sure if everyone caught the new functionality that Peter has put into 1.2M3. It's the ability to test grails scripts. Heres the link to his blog post:
http://www.cacoethes.co.uk/blog/groovyandgrails/testing-your-grails-scripts

Build Server Woes (mptscsi task abort)

2009-09-17T08:37:00.004+01:00

I'm posting this here for two reasons:

in the hope that someone out there will find this useful

personal therapy

Scenario:

We've got a couple of build servers (x86_64 linux, openVZ) that have been having some disk I/O problems. These boxes (boxen) run various virtual machines related to our product builds - hudson masters & slaves, distribution servers, puppet master, test mail server, munin server... yada yada. You get the idea. They're kinda important.

The problem manifests itself by first reporting errors like the following:


Sep 16 14:28:51 hn3 mptscsih: ioc0: attempting task abort! (sc=ffff880422a348c0)
Sep 16 14:28:51 hn3 sd 0:1:2:0: [sda] CDB: cdb[0]=0x2a: 2a 00 12 b2 fc 9f 00 00 08 00
Sep 16 14:28:51 hn3 mptscsih: ioc0: Issue of TaskMgmt failed!

followed shortly by the volume in question getting offlined into readonly mode:


Sep 16 14:29:41 hn3 mptscsih: ioc0: host reset: SUCCESS (sc=ffff880422a348c0)
Sep 16 14:29:41 hn3 sd 0:1:2:0: Device offlined - not ready after error recovery

This ain't that helpful when you've got a whole load of hungry VMs wanting to write stuff to disk. A closer look at the disk controller yields the following:


>lspci
...
0b:00.0 SCSI storage controller: LSI Logic / Symbios Logic SAS1068E PCI-Express Fusion-MPT SAS (rev 08)
...

A quick search through our messages shows the following related information:


>dmesg | grep -i mpt
Fusion MPT base driver 3.04.07
Fusion MPT SPI Host driver 3.04.07
Fusion MPT FC Host driver 3.04.07
Fusion MPT SAS Host driver 3.04.07
mptsas 0000:0b:00.0: PCI INT A -> GSI 35 (level, low) -> IRQ 35
mptbase: ioc0: Initiating bringup
mptbase: ioc0: PCI-MSI enabled
mptsas 0000:0b:00.0: setting latency timer to 64
Fusion MPT misc device (ioctl) driver 3.04.07
mptctl: Registered with Fusion MPT base driver
mptctl: /dev/mptctl @ (major,minor=10,220)

A quick google turns up quite a few different issues with these controllers but no clear resolution (no surprise).

To cut a (very) long story short we appear to have a solution by using the drivers supplied by LSI rather than those shipped with the latest linux kernel. Patching the kernel (by replacing the drivers/message/fusion folder with the equivalent found in LSI's MPTLINUX_RHEL5_SLES10_PH16-4.18.00.00-1.zip distribution) with version 4.18 of the MPT drivers has yielded an (apparently) stable system, tested under reasonably high load (load average ~20).

Incidentally, for those of you who like acronyms, MPT stands for 'Message passing technology'.

My work here is done.

Dynamic UrlMapping using request parameters

2009-08-17T16:06:00.002+01:00

Might be old news to some, but I just found you can assign closures to the action and controller values in URL mappings, e.g.

"/update/$controller/$target" {
   action = {
      "update" + params.target[0].toUpperCase() + params.target.substring(1);
   }
}

Great for mapping restful style urls to meaningful action names, e.g.

"/update/patient/speciality" => PatientController.updateSpeciality

Selenium IDE 1.0.1

2009-07-05T15:23:00.007+01:00

I recently upgraded to Selenium 1.0.1 and found that everything worked fine except that some asserts were failing around stored variables. e.g.

storeText //h2 varname

It turned out that if the text is hidden on the page then all that is stored is ''. In our case it was fixture pages that was returning some data as hidden page elements for reference.

Selenium IDE

2009-06-24T16:35:00.003+01:00

Just a quick post to say that if any of you out there spend your day resizing and moving Selenium IDE after reopening it (it doesn't remember the previous window state), you can write a little script to do it for you using the handy 'wmctrl' :)

(on linux)
wmctrl -r 'Mozilla Firefox' -e 0,0,25,1330,1125
wmctrl -r "Mozilla Firefox" -b remove,maximized_vert,maximized_horz
wmctrl -a 'Mozilla Firefox'
wmctrl -r 'Selenium IDE' -e 0,1350,25,575,1125
wmctrl -a 'Selenium IDE'

The snippet above moves, unmaximises and raises firefox before moving and activiating the Selenium IDE.

Custom constraint

2009-06-23T20:11:00.002+01:00

The other day we were doing some refactoring and we started to look into how all our validation is done. We noticed that we have a lot of duplication in our validators and wanted to do something about it. We came across the following blog post that describes how to create a custom validator. This is a great little blog post and allowed us to kill a lot code.

One thing that we did notice was that they suggest that you put the registering of the custom validator in the Config.groovy file. We found this problematic and found that the better solution was to put it in the resources.groovy file. Another problem that we came across was that you needed to register the constraint in the unit test if you wanted to be able to test constraints. This makes sense because resources.groovy isn't called in unit tests.

To register your constraint use the following line of groovy code:

ConstrainedProperty.registerNewConstraint(PhoneNumberConstraint.NAME, PhoneNumberConstraint)

Hopefully this helps with some refactoring.
Glenn

Selenium CSS locators

2009-04-29T09:43:00.002+01:00

We've got a project with quite a large set of tests, and we have both Firefox and IE7 continuous builds using selenium. The annoying thing is that something that takes 5 minutes in Firefox can take 30+ minutes with IE (I kid you not!).

It turns out this is a common problem and is due to the lack for native xpath support in IE. So for all the selenium tests that use xpath=//..... in IE the xpath is actually being evaluated using javascript ... ouch. An alternative is to use css locators, you can't do this everywhere but as well as improving the IE performance it can also provide some rather tidy rules. For example


xpath=//div[contains(@class,'balance')]
or
css=.balance

xpath=//div[@id,'topLeft')//span[contains(@class,'name')]  
or 
css=#topLeft .name

So even if you dont' care about IE using css selectors might be a lot nicer. (It might also improve your css skills!)

GMock Talk

2009-04-23T09:08:00.001+01:00

I'll be giving a talk about GMock next wednesday the 29th.

Please register if you are interested:

http://skillsmatter.com/event/java-jee/gmock-be-groovy-when-writing-tests

instanceof vs. HibernateProxy and inheritance

2009-04-15T09:21:00.001+01:00

I posted something over on my blog about the exciting things that can happen when Hibernate proxies your base class to wrap an instance of your sub-class.

IE hangs with "Waiting for 1 resource to load"

2009-04-12T13:06:00.002+01:00

Turns out this is a common problem with libraries like prototype and jquery. There's a good discussion of it here...

http://groups.google.com/group/prototype-scriptaculous/browse_thread/thread/a1c745463251a95d?pli=1

The suggested solution of replacing src=:// with src=blank.js worked for us.

Timing Selenium

2009-04-03T15:33:00.013+01:00

We finally figured a pain free way of profiling our selenium test suite. It could use some refinement, but we were only looking for a quick win.

Add the following test to the start of your suite


open|/|
storeEval|{lastTime = new Date().getTime(); 'x'}|x
storeEval|{LOG.info = function(message) { var now = new Date().getTime(); var duration = now - lastTime; this.warn(message + duration); lastTime = now}; 'x';}|x

When your run your suite the log file file contain lines like...


warn: Starting test /selenium-server/tests/suite-tests/conversation/SearchConversationTest.html119
warn: Executing: |open | /fixture/tearDownTestData |  |47
warn: Executing: |assertTitle | Fixture |  |239
warn: Executing: |invokeUrl | /fixture/reindexSearch |  |28
warn: Executing: |invokeUrl | /fixture/setUpPersons |  |121
warn: Executing: |invokeUrl | /fixture/setUpConversationsForSearch |  |212
warn: Executing: |createCookie | splash=false | path=/ |16638

The duration for each step has been appended to the NEXT log line, i.e. setUpConversationsForSearch took 16638ms
The next step is to parse this file into something readable. We wrote a groovy script (see below), which generates output like:


===SELENIUM TEST TIMINGS===
69763=HomepageTest
54106=SubscribeToTribeConversationTest
50902=SubscribeToTagTest
48734=TagDedupingTest
44614=SubscribeToPrivateConversationTest

===INDIVIDUAL COMMAND TIMINGS===
153127=loginAndWait   hits=291 avg=526.2096219931
103428=open  /person/ hits=241 avg=429.1618257261
90431=open   /logout  hits=224 avg=403.7098214286
70059=invokeUrl   /fixture/setUpConversationsForSearch  hits=4 avg=17514.75
61926=open   /login  hits=246 avg=251.7317073171
61851=open   /  hits=133 avg=465.045112782
61034=createMessageAndWait   hits=114 avg=535.3859649123

So now it's trivial to identify slow running tests / test steps when optimising our build. It's also useful to spot slow loading pages.

Here's the groovy script, it's tailored to our environment but a good starting point.


import org.codehaus.groovy.tools.LoaderConfiguration
import org.codehaus.groovy.tools.RootLoader
import org.codehaus.groovy.grails.commons.cfg.ConfigurationHelper
import org.codehaus.groovy.grails.commons.ConfigurationHolder
import groovy.text.GStringTemplateEngine
import org.apache.commons.lang.StringUtils

Ant.property(environment:'env')
grailsHome = Ant.antProject.properties.'env.GRAILS_HOME'
grailsEnv = System.properties.'grails.env';
pluginDir = this.binding['seleniumPluginDir']

includeTargets <<> timings = []

int nextId = 0
String currentTest = 'anonymous-test'
SeleniumTiming previousTiming
SeleniumTiming currentTiming

class SeleniumTiming {
  String id
  String test
  String command
  String target
  String value
  int duration = 0

  String getTargetKey() {

      def trimmedTarget = target?.trim()
      def trimmedCommand = command?.trim()

      if (trimmedCommand == 'open' && trimmedTarget ==~ '/person/\\w+') {
          return '/person/'
      } else if (trimmedCommand == 'open' && trimmedTarget ==~ '/tag/[\\w-]+') {
          return '/tag/'
      } else if (trimmedCommand == 'open' && trimmedTarget ==~ '/station/[\\w-]+') {
          return '/station/'
      } else if (trimmedCommand == 'open' && trimmedTarget ==~ '/line/[\\w-]+') {
          return '/line/'
      } else if (trimmedCommand == 'open' && trimmedTarget ==~ '/conversation/subscribe?.*') {
          return '/conversation/subscribe'
      } else if (trimmedCommand == 'open' && trimmedTarget ==~ '/conversation/unsubscribe?.*') {
          return '/conversation/unsubscribe'
      } else if (trimmedCommand == 'loginAndWait') {
          return ''
      } else if (trimmedCommand == 'sendMessageAndWait') {
          return ''
      } else if (trimmedCommand == 'createMessageAndWait') {
          return ''
      } else if (trimmedCommand == 'reply') {
          return ''
      } else if (trimmedCommand == 'replyAndWait') {
          return ''
      } else if (trimmedCommand == 'registerAndWait') {
          return ''
      } else if (trimmedCommand == 'reportThisAndWait') {
          return ''
      } else {
          return target
      }
  }

  String toString() {
      "[$test] $command($target,$value):$duration"
  }
}

def parseTestLine = { String input ->
  currentTest = input.substring(input.lastIndexOf(File.separator)+1, input.lastIndexOf('.html'))
}

def parseCommandLine = { String input ->
  List data = input.tokenize('|')
  if(data.size() == 5) {
      previousTiming = currentTiming
      currentTiming = new SeleniumTiming(id:nextId++, test: currentTest, command:data[1],
          target:data[2], value:data[3])
      timings << duration =" Integer.parseInt(data[4])" parsefile =" {">
  input.eachLine { String line ->
      if(line.startsWith('warn: Starting test')) {
          parseTestLine line          
      } else if(line.startsWith('warn: Executing: |')) {
          parseCommandLine line
      }
  }
}

target ('default': 'Generate Selenium Timings Report') {
loadSeleniumConfig seleniumConfig
new File(seleniumConfig.reportdir).eachFileMatch(~/.*\.html/, parseFile)

  def longestFirst = [ compare: { a, b -> a.equals(b) ? 0: Math.abs(a) < overalltimings =" new">
  overallTimings[(values*.duration).sum()] = key
}

  Map commandTimings = new TreeMap(longestFirst)
timings.groupBy({ "${it.command} ${it.targetKey}" }).each { key, values ->
   int totalDuration = (values*.duration).sum()
      int noOfValues = values.size()
      commandTimings[totalDuration] = "$key hits=${noOfValues} avg=${totalDuration / noOfValues}"

}

  new File(seleniumConfig.reportdir, 'timings.txt').withPrintWriter { writer ->
      writer.println '===SELENIUM TEST TIMINGS==='
      overallTimings.each { writer.println it }
      writer.println '===INDIVIDUAL COMMAND TIMINGS==='
      commandTimings.each { writer.println it }
  }
    
  commandTimings.each { println it }

}

Release of gmock 0.7.0

2009-03-21T10:51:00.002Z

GMock 0.7.0 has just been release. This brings two long awaited features: strict ordering and partial mocking.

Strict ordering is accomplished through the ordered closure. Here is an example with an hypothetic cached cat database:
def database = mock()
def cache = mock()
ordered {
  database.open()
  cache.get("select * from cat").returns(null)
  database.query("select * from cat").returns(["cat1", "cat2"])
  cache.put("select * from cat", ["cat1", "cat2"])
  database.close()
}

The partial mocking is performed simply by using the mock method on your concrete object. Here is how it works with a grails controller:
def controller = new SomeController()
mock(controller).params.returns = [id: 3]

GMock 0.7.0 is the last release compatible with Groovy 1.5.x. Support for Groovy 1.6.0 is coming soon.

Grails 1.1 released

2009-03-10T18:41:00.003Z

Just in case anyone missed it, the latest version of Grails was officially released today.

Gsp reloading for other environments

2009-03-03T09:32:00.002Z

Gsp reloading does not happen on environments other than development. To get around this you can either pass in a command line argument of -Dgrails.gsp.enable.reload=true or put grails.gsp.enable.reload=true into the environment config for the environment that you want gsp reloading.

This came to light because we do not use production data in our development environment but sometimes it's useful to have production data to test with or to work out a bug. We have a separate environment config for the times we want to run with prod data so this little flag has become a life-saver.

New in Groovy 1.6

2009-03-02T09:35:00.002Z

Since grails is built on groovy it's important to keep up to date with what's new in the next release of groovy. Here's a link to the new features:

whats new in groovy 1.6

Groovy 1.6 will be released with grails 1.1

Running Selenium Tests In Grails 1.1

2009-03-01T04:26:00.004Z

Anyone experimenting with Grails 1.1 might be interested to know that I've got the Selenium plugin working. If you put the following entries in your project's grails-app/conf/BuildConfig.groovy:

grails.plugin.repos.discovery.energizedwork="https://svn.energizedwork.com/skunkworks/grails/plugins"
grails.plugin.repos.distribution.energizedwork="https://svn.energizedwork.com/skunkworks/grails/plugins"

Then type grails install-plugin ew-selenium you should be good to go.

The command to run tests is grails run-selenium. The script now runs in the test environment by default so you no longer need to use grails test run-selenium.

You may need to configure the browser Selenium uses. To do so edit test/selenium/selenese/conf/SeleniumConfig.groovy and set the selenium.browser property. For some reason Firefox 3 needs to be specified as *chrome. For example on a Mac I have to use the setting

selenium.browser = "*chrome /Applications/Firefox.app/Contents/MacOS/firefox-bin"

I have no idea why this is necessary. I have upgraded the Selenium Server version included in the plugin but I found this was also necessary with the old 0.9.2 version.

I changed the plugin name to ew-selenium as there is a plugin called selenium on the main Grails plugin repository and it looks like Grails will ignore conflicting names on other repositories.

Hitlers nightly build fails

2009-02-19T23:03:00.004Z

With a whole bunch of Energizers en route to Canadia post-haste I'm feeling distinctly in de-mob happy mode.

Thought I would post a funny, yet development related, video which @unclebobmartin just shared

Hitler's Nightly Build Fails

http://www.youtube.com/watch?v=Azl4nqLn4-Y

Its worth watching the the end.. Its get better n better :) Cheers, @franklywatson

Case-insensitive ordering using HSQLDB

2009-02-11T17:09:00.004Z

We've recently begun the process of testing our app on Postgres, and came across a problem where our 'order' statements were returning results in a different order to HSQLDB.

Postgres was ignoring the case of items, whereas HSQLDB was not - so if I were to order the strings "Al" and "AM", HSQLDB would show "AM" before "Al", but Postgres would show "Al" before "AM".

Given that the behaviour of Postgres (case-insensitive ordering) was the desired behaviour, we wanted to get HSQLDB to do the same. After a lot of searching any many dead ends, we found that you simply use ".ignoreCase()" after an order criteria:

List books = Book.withCriteria() {
        ilike('authorName', "%${searchKey}%")
        order('authorName', 'asc').ignoreCase()
        maxResults(max)
}

Immutable Objects in Groovy

2009-02-11T13:07:00.003Z

Anyone familiar with Joshua Bloch's book Effective Java will know about the importance of immutable objects. However, writing classes like that in Groovy always seemed to me to be problematic. You can declare the properties final but if the property type is itself mutable encapsulation is broken. Defining properties as private wasn't enough as Groovy lets you access them from anywhere regardless (I'm still not entirely clear why the language even supports it). A combination of final properties with overridden get methods where property types are mutable works but then you're writing almost as much code as you'd need to in Java and who knows what someone can get away with by using the .@ operator!

Google provided the solution this morning. It turns out there is an @Immutable annotation in Groovy that solves the problem in a very neat way even giving you an implementation of equals, hashCode and toString for free. Those methods tend to be pretty much templated in immutable classes anyway so getting effective implementations without writing any code makes a lot of sense and feels very 'Groovy'.

Binding to collection fields on command objects

2009-02-10T17:45:00.006Z

One of the cool new features of Grails 1.1 is the ability to bind form elements to collection properties of domain objects. Unfortunately it doesn't quite work out of the box for command objects.

Suppose I have a command class like this:

    class MyCommand {
        Map things = [:]
    }

and my form posts params like this:

    things[key1] = value1
    things[key2] = value2
    things[key3] = value3

I'd hope that the binder could cope with this and populate my command object's things property with the appropriate keys and values. It doesn't work as it seems that Grails attempts to figure out what the type of the object contained in the collection is. For domain objects this is possible since the hasMany closure can be used. Command objects don't have this available and so you end up with a nasty stack trace boiling down to a NullPointerException.

However there is a way to 'fool' the data binder. All it's trying to do is figure out the value type for the Map so it can use an appropriate PropertyEditor to decode the HTTP request parameter value. In this example the value is just a String. Initializing the Map like this provides a workaround:

    import org.apache.commons.collections.MapUtils
    import org.apache.commons.collections.FactoryUtils

    class MyCommand {
        Map things = MapUtils.lazyMap([:], FactoryUtils.constantFactory(''))
    }

That's using a LazyMap from Commons Collections which (in case you can't guess) is a Map implementation that populates a default value when get is called for an unmapped key. The Map in the command object will use the empty string as its default value which is enough for the Grails data binder to figure out the value type and bind our parameters.

I haven't tried the same trick with richer data types yet (e.g. a collection of domain objects in a command) but I don't think there's any reason why a similar workaround returning a different value from the Factory shouldn't work.

Testing Url Mappings

2009-02-01T14:59:00.007Z

If you have to support legacy urls, want some quick feedback when putting your url mappings together or want to verify the parameters are being parsed correctly, here's a little grails integration-test snippet that you may find useful:


def grailsUrlMappingsHolderBean

void testMappingForPersonController() {
def mappingInfo = grailsUrlMappingsHolderBean.match('/person/pain')
assertEquals 'person', mappingInfo.controllerName
assertEquals 'show', mappingInfo.actionName
assertEquals 'pain', mappingInfo.parameters.id
}

The grails testing plugin provides custom assertions for doing this and more (assertUrlMapping etc.).

kthxbye

On selenium testing...

2009-01-22T21:48:00.003Z

I updated my blog to list some useful tips for writing more resilient Selenium XPath tests.

Please feel free to add more suggestions or otherwise comment.

Groovy Truth and the instanceof Operator

2009-01-06T12:00:00.002Z

I just stumbled across a little gotcha which would have failed compilation in Java, but in Groovy compiles but doesn't do what the developer intended.

    if (!obj instanceof SomeEnum) throw new IllegalArgumentException()
    // do something with obj.name()

Clearly, the intention was to throw IllegalArgumentException if obj is not the correct type, but instead the code was throwing MissingMethodException from the access to obj.name().

The problem is missing braces. What Groovy is actually evaluating is

    if (false instanceof SomeEnum)

!obj evaluates to false as obj is not null. The correct code would be:

    if (!(obj instanceof SomeEnum)) throw new IllegalArgumentException()

End of year burp..

2008-12-30T16:58:00.003Z

I've been getting plenty end of year roundups in my reader and thought I might add something forward looking to the blog to encourage the team to use 2009 to investigate some interesting new things which we could all benefit from.

For Grails I would suggest that anyone working on new project (ahem, you know who you are) using Liquibase (or planning to), should seriously consider using Autobase. Its DSL for Liquibase, so keeps things more Groovy and less XML (we loved stepping off the spring.xml train, so now we're in Liquibase XML fun... why?).

The HttpBuilder DSL was announced without too much fanfare, which I know we use a lot. Again, another handy tool to jump on.

If you haven't heard of Robert Fischer by now then you've been sleeping beside your pair, but his new book is sure to be a boon to us all: Grails Persistence with GORM and GSQL. Get your pre-orders in, its out on the 26th of Jan.

Another interesting looking book is ScrumBan by Corey Ladas. We're always talking about improving how we work, and the Lean folks are challenging Scrum'mers to think a little more like them.

Which brings me to my last and more unusual mention which is that David Anderson managed to convince the people at Agile 2009 to setup the Agile Frontier Stage. David Anderson gave the singularly most captivating presentation at Agile Vancouver 2008 and stands out to me to challenge (in a very healthy and practical way) a lot of the orthodoxy which Scrum has become entrenched with. He's good people, see him present!

Thats its from me for this year

Happy New Years all :)

Invalidating Single Cached Items

2008-12-30T16:05:00.003Z

Should have RTFM on this one but it was stuck on the board for so long and even F5 support couldn't figure out so I thought I'd post it here.

Sometimes we need to remove content from our CMS which means our app servers will serve up 404s for those requests. But we found that our F5 kept serving content from its stand-in cache for another 24 hours after the max-age had expired. We thought the stand-in cache was only there for when all the nodes in your pool had died so it looks like it's doing a little more than that.

Anyway, sometimes when we remove stuff from 'live' we really want it gone so we searched around and found that you can remove a single item from the F5 cache using a SOAP request to the management interface. The bad news is that you have to provide usernames and passwords in the clear but there's a nice way to get the same effect using invalidation triggers. At the back of Chapter 11 in the Policy Management Guide there's a straightforward example of what to do.

Handy Hints in BASH

2008-12-11T15:50:00.003Z

Ever wanted to diff the output from 2 commands but didn't want to have to write the output to temporary files first? Well bash subshells come to the rescue.

Let's say you want to compare the list of files in 2 different directories. Try this:

diff <(ls /path/to/dir1) <(ls /path/to/dir2)

The bits in brackets are run in a subshell and then we redirect the output of that subshell to the stdin of diff. (On a personal note I prefer to run diff with the -y switch to get a side-by-side diff, but that's me).

Of course you can get really tricky with these things. How about making sure the file permissions are right as well?

diff -y <(ls -l /path/to/dir1 | awk '{print $1, $NF}') <(ls -l /path/to/dir2 | awk '{print $1, $NF}')

(All that should be on one line. This is my first post so be gentle on me if it comes out looking like an uncombed monkey.)

So, have fun with subshells. And be careful out there.

Caching Grails Services

2008-12-11T14:23:00.009Z

I've been plugging away for a few days on adding caching to Grails services. At first I tried to go down the road of decorating the services' metaClasses but since Grails uses an AOP proxy to add transactional behaviour to services this wouldn't work - the metaClass I got from the injected service object was not the same one my plugin had added stuff to. Also annotations on the service class were unavailable from the proxy.

The Spring Modules project has a nice caching implementation that uses annotations to decorate methods with caching and flushing behaviour. Getting this to work with Grails, was reasonably straightforward. As it turns out one additional config line made the difference between the spring-modules example config and something that worked in Grails.

I based the config on the annotation example from the Spring Modules documentation, converting the Spring XML to Grails BeanBuilder format:

    import org.springframework.aop.framework.autoproxy.*
    import org.springmodules.cache.annotations.*
    import org.springmodules.cache.interceptor.caching.*
    import org.springmodules.cache.interceptor.flush.*

    def doWithSpring = {

        // declaration of cacheManager and cacheProviderFacade omitted - implementation specific

        autoproxy(DefaultAdvisorAutoProxyCreator) {
            proxyTargetClass = true
        }

        cachingAttributeSource(AnnotationCachingAttributeSource)

        cachingInterceptor(MetadataCachingInterceptor) {
            cacheProviderFacade = ref("cacheProviderFacade")
            cachingAttributeSource = ref("cachingAttributeSource")
            def props = new Properties()
            props.myCachingModel = 'cacheName=MY_CACHE_NAME'
            cachingModels = props
        }

        cachingAttributeSourceAdvisor(CachingAttributeSourceAdvisor, ref("cachingInterceptor"))

        flushingAttributeSource(AnnotationFlushingAttributeSource)

        flushingInterceptor(MetadataFlushingInterceptor) {
            cacheProviderFacade = ref("cacheProviderFacade")
            flushingAttributeSource = ref("flushingAttributeSource")
            def props = new Properties()
            props.myFlushingModel = 'cacheNames=MY_CACHE_NAME'
            flushingModels = props
        }

        flushingAttributeSourceAdvisor(FlushingAttributeSourceAdvisor, ref("flushingInterceptor"))
 }

The vital bit is the proxyTargetClass = true on the autoproxy bean. I won't pretend to understand what that's doing - I just noticed that's how ServicesGrailsPlugin was getting the transactional proxies to work.

After that it's a simple case of annotating service methods (in fact methods on any Spring-managed bean but services are the obvious use case).

    import org.springmodules.cache.annotations.*

    @Cacheable(modelId = "myCachingModel")
    def getSomethingInAnExpensiveWay(param1, param2) {
        // ...
    }

    @CacheFlush(modelId = "myFlushingModel")
    def updateSomething(param1, param2) {
        // ...
    }

This works fine on transactional and non-transactional Grails services. You can see the cache operations by setting log4j.logger.org.springmodules.cache = "trace".

The actual caching implementation used can be anything - spring-modules supports ehcache, oscache, JBoss cache, JCS, etc. All that's required is to wire in the cacheManager and cacheProviderFacade beans as described in the Spring Modules documentation.

I've bundled the results up as a Grails plugin (grails install-plugin springcache). It uses a ConcurrentHashMap backed simple cache implementation by default but wiring in ehcache or oscache is easy.

Caching and flushing models are configured in Config.groovy, e.g.:

    springcache {
        cachingModels {
            cachingModel1 = 'cacheName=CACHE_1'
            cachingModel2 = 'cacheName=CACHE_2'
        }
        flushingModels {
            flushingModel1 = 'cacheNames=CACHE_1,CACHE_2'
        }
    }

Particular caching implementations may have further options or require additional external config (such as ehcache's ehcache.xml file).

To disable the plugin you can set springcache.disabled=true. For example, it may be desirable to disable the plugin in the test environment.

Edit: Documentation added on grails.org

Passing enums into a constructor

2008-12-11T08:07:00.006Z

Take this error:

binding.GrailsDataBinder Unable to auto-create type, 'create' method not found.

I came across this yesterday, and although the cause was documented, two of us managed to miss the crucial sentence in those docs for about an hour, causing much scratching of heads:

"my problem is that when I pass a Status enum to a constructor of some other class"

So, setting enum properties on a domain object is fine:

Burger canHaz = new Burger();
canHaz.flavour = Flavours.CHEESE;

Just not doing so via a constructor call:

Burger willNeverHaz = new Burger(flavour:Flavours.VEGETARIAN);

The original post I found is here:

http://www.nabble.com/Using-enums-gives-%22Unable-to-auto-create-type,-'create'-method-not-found%22-error-td18890306.html

and the JIRA bug report here:

http://jira.codehaus.org/browse/GRAILS-3314

Hopefully this will save someone else lots of head scratching! :o)

Nasty liquibase bug

2008-12-10T09:03:00.003Z

We recently came across a nasty liquibase bug.

When you drop a not null constraint on a table and you do not add in the column data type liquibase creates the column with a null column type.

Eg:
dropnotnullconstraint tablename="pirates" columnname="crew" --- creates a null column type.

Easy fix:
dropnotnullconstraint tablename="pirates" columnname="crew" columndatatype="VARCHAR(255)"

Just wanted to give everyone a heads up on the problem so they don't burn half a day trying to figure out what is going on.

Counting using Grails/Hibernate criteria

2008-11-27T12:52:00.002Z

Just a quick gotcha for grails criterias when you want to count the results from a query where there is a join involved. If you use

count = criteria.count {
projections {
rowCount()
}
query()
}

You'll get a count for each of the joins, you might think that you could use

count = criteria.count {
projections {
countDistinct('id')
}
query()
}

But that still doesn't work - you have to use

count = criteria.get {
projections {
countDistinct('id')
}
query()
}

Hmmm

Autobase 0.5 plugin released

2008-11-24T03:15:00.004Z

Robert Fischer just announced the latest version of the Autobase plugin which simplifies Liquibase usage by way of providing a Groovy DSL to use it by.

http://github.com/RobertFischer/autobase/wikis/example-usage

Much more compact and more Groovy! Looks like all ur Liquibase migrations r belong to Autobase nao!

Groovy Awesomeness

2008-11-17T13:41:00.004Z

Hey check this out:

Java:


import java.util.Calendar;
import java.util.Date;

public class PrintIndependenceDay {

  public static void main(String[] args) {
    Calendar calendar = Calendar.getInstance();
    calendar.clear();
    calendar.set(Calendar.MONTH, Calendar.JULY);
    calendar.set(Calendar.DATE, 4);
    calendar.set(Calendar.YEAR, 1776);
    Date time = calendar.getTime();
    System.out.println(time);
  }
}

Now check the awesomeness groovy gives you:

def calendar = Calendar.instance
calendar.with {
clear()
set MONTH, JULY
set DATE, 4
set YEAR, 1776
println time
}

Here is the full article that explains everything: Getting Groovy with 'with'

rock on

Grails 1.0.4 Released

2008-11-16T14:39:00.003Z

Grails 1.0.4 has just been released

Hand me a blue card please...

Fun With Groovy Categories

2008-11-14T11:15:00.005Z

Just a quick stupid trick I discovered. Commons Lang's StringUtils class works quite nicely as a category for String, e.g.

    use(StringUtils) {
        println 'i can has cheezburger'.substringAfter('i can has ') // prints 'cheezburger'
    }

Not only that but the JetGroovy plugin for IntelliJ IDEA will autocomplete all the new String methods inside the use block.

Checking HTTP Headers

2008-11-10T11:17:00.003Z

Disclaimer: This post has nothing to do with Grails, other than the application we're testing is a grails app!

We're starting to "productize" our application. One of the stories is to ensure all javascript, CSS and image files are served with an appropriate set of HTTP headers so they can be cached effectively. For this we're using good old HTTP unit (I'll post about that more when we're finished), but in the meantime I've found the 'tcpdump' command very useful.

sudo tcpdump -A -ilo0 -t -s 1500 port 8080

-A prints ASCII
-i specifies the interface (I'm testing locally so lo0 is the loopback interface)
-t supresses timestamps
-s How many bytes to print (I wanted to see the full packet so set it to 1500)
port 8080 Tells tcpmon to only show me stuff going to or coming from port 8080

This results in output like


...GET /script/N259177905/bundles/stuff.js HTTP/1.1
User-Agent: httpunit/1.5
Cookie: splash=false
Cache-Control: no-cache
Pragma: no-cache
Host: localhost:8080
Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
Connection: keep-alive


IP localhost.http-alt > localhost.53121: . ack 1892 win 65535 >nop,nop,timestamp 219937537 219937537<
E..4.n@.@...............:.,A.........(.....
...
IP localhost.http-alt > localhost.53121: . 197931:214263(16332) ack 1892 win 65535 <nop,nop,timestamp 219937537 219937537>
E.@.q.@.@...............:.,A........=......
...HTTP/1.1 200 OK
Content-Type: text/javascript; charset=UTF-8
Expires: Sat, 10 Nov 2018 11:24:54 GMT
Cache-Control: public, max-age=315360000, post-check=315360000, pre-check=315360000
Last-Modified: Sun, 06 Nov 2005 12:00:00 GMT
ETag: 2740050219
Transfer-Encoding: chunked
Server: Jetty(6.1.4)

6000
;(function(){function r(val,args){for(var x=0;x<args.length;x++){val=val.replace('{'+x+'}',args[x]);}
return val;}
function p(){var val=arguments[0];var ret;if(val.indexOf('{0}')!=-1)
ret=function(){return r(val,arguments);}
else ret=function(){return val;}
...

I'm sure there's more I could do to fine tune the output, but it's a good starter for 10!

GGUG meeting tomorrow

2008-10-29T09:33:00.001Z

Just an FYI that this months Grails user group meeting is tomorrow night. Here is the link if you want to come. Grails meetup

Joda DateTime Ranges

2008-10-29T08:36:00.006Z

Recently I was attempting to create a Range representing the difference between two DateTime values so I could assert in a test that a date time value fell into that range. I was using Joda Time's DateTime class. The java.util.Date class is compatible with Groovy's ObjectRange becuase it's decorated with next and previous methods by the GDK. Joda's DateTime isn't, so I ended up doing this:

    DateTime start = new DateTime()
    DateTime end = start.plusYears(1)
    Range range = start.millis..end.millis

I could have created a custom Range implementation but I'd have lost the option of using the x..y notation and I'm a sucker for brevity. I figured Groovy has an IntRange class so there's probably a LongRange as well. As it turns out there isn't and what I ended up with is an ObjectRange.

Fundamentally this works, although I did find something worth bearing in mind. When it came to my assertion I tried to do this:

    static void assertInRange(Range expected, actual) {
        if (!expected.contains(actual) {
            fail("Expected value in range:<${expected}> but was:<${actual}>")
        }
    }

Simple enough, right? Except running the code caused the test to hang. Looking into the implementation of ObjectRange it became clear why. Groovy's Range interface extends java.util.List and that's where the contains method is specified. For compatibility with the List interface ObjectRange iterates over the values between its from and to properties using the next method I mentioned above. When from and to are Long instances, it turns out this iteration takes a while - on my 2.16GHz iMac the contains method takes over 4½ minutes on a 1 year range!

The Range interface also specifies a containsWithinBounds method and ObjectRange implements this by simply checking that the argument is >= from and <=to. Changing my assertion to use containsWithinBounds got rid of the long wait.

What ObjectRange is doing does make perfect sense. Given two arbitrary objects that implement Comparable and the next and previous methods it needs to implement contains in a way that conforms to the List interface. When you think about how the GDK implements next and previous on java.util.Date for example you can see that there may be possible values of a given class that fall inside a range but would never be part of the List generated by repeatedly using next on the range's from property. This isn't true of Long, of course, but ObjectRange is a general purpose class. The IntRange class has a much optimised version of contains and so could a theoretical LongRange implementation.

Anyway, getting back to the example. After changing contains to containsWithinBounds the code ran much faster, unfortunately it also threw java.lang.OutOfMemoryError: Java heap space. It turns out this is down to how GString handles the ObjectRange value I inserted into it. It spots that the object it's got implements List and uses InvokerHelper.toListString on it. This will attempt to create a String in the form "[0, 1, 2, 3...]". As you can imagine, building such a String is going to take some doing with over 31½ billion elements in the List, hence the heap space ran out.

A final working implementation of my assertion method is:

    static void assertInRange(Range expected, actual) {
        if (!expected.containsWithinBounds(actual)) {
            fail "Expected:<${expected.toString()}> but was:<${actual}>"
        }
    }

I think going for the custom Range implementation in the first place might have actually been the right choice.

Gmock - Groovy Mock

2008-10-18T09:54:00.003+01:00

Mock Object are key players in Unit Testing. Groovy support natively some mock object with MockFor and StubFor but their functionality are quite limited and the syntax heavy - you'll understand when you nest 7 'use' closures.

Gmock aims to simplified mocking in Groovy through a intuitive syntax and a great readability. In a nutshell a Gmock test look like:


  void testTree(){
    def mockTree = mock()
    mockTree.load('fruit').returns('apple')
    play {
      assertEquals "apple", mockTree.load('fruit')
    }
  }

This is it!. Expectation are being setup by calling normal method on you mock object. The code under test is executed within the play closure and your mocks are automatically verified after it.

The current version gmock-0.2 support the most basic functionality you would expect from a mocking framework. Version 0.3 should see static method mocking and property mocking. Future development are described in the Roadmap.

Selenium and Firefox 3

2008-10-15T16:09:00.003+01:00

Those of you who've tried to run the selenium test suite using Firefox 3 as the browser have probably found that Firefox hangs trying to create the temporary profile. You could do some tedious mucking about with selenium.browser="*custom /usr/lib/firefox/firefox -no-remote -P selenium", manually configure the proxy, etc. However, it turns out there's a simpler solution.

Please note this is nothing to do with Selenium IDE, if you're having problems with that in conjunction with Firefox 3 this won't help.

A “clearfix” CSS redux

2008-10-14T22:53:00.002+01:00

We use a CSS technique called "clearfix" within several projects. This is implemented as a class (called "clearfix") that is applied to a parent element that contains floated child elements.

I wrote a post over at Code Couch that attempts to explain the reason for doing this in the first place, describes the CSS that we use and provides plenty of links for detailed reference.

XHTML — myths and reality

2008-10-09T09:56:00.004+01:00

Tina Holmboe is a member of the XHTML Working Group and has an interesting write-up XHTML — myths and reality that discusses the current state of XHTML and dismisses some popular inaccuracies.

Having hyped up the post, it is rather old news - although it is discussed in simple (simplistic?) terms which I find is better than overloading with jargon.

Here are some hilights:

Lack of support for XHTML is a fact of life on the web in 2008. Prior to the 3.0 series of Firefox the XHTML processor in Gecko was so poor that Mozilla's own engineers recommended against it;

No version of Internet Explorer up to, and including, IE 8 support XHTML at all, and a number of other browsers such as Lynx were never written to handle XML in the first place.

I wrote a post for another blog a while back that looked into how you can identify the appropriate doctype for your web page - and stumbled across this "lack of support" for XHTML in modern browsers. You can read the original post How to choose a doctype if you are interested.

Whilst XHTML remains a much more structured (and cleaner) choice to deliver content to the browser, I have yet to work on a single project where it is actually required.

XHTML is the default for use within the Grails framework - even though the worlds most prolific browser doesn't support it. Hmmm.

Programmatic Transactions

2008-10-09T09:18:00.002+01:00

By default grails starts a new transaction the first time it enters a service method and commits the transaction when it returns. You can disable this behaviour by declaring your service to be non transactional.

Sometimes you need more fine grained control. Here's a link to a great discussion which unfortunately doesn't seem to have made it into the Grails manual yet, however this is still talking about declarative transactions, rather than programmatic ones.

If you really do want programmatic transactions you can use

DomainObj.withTransactions { TransactionStatus ts ->
   serviceA.doStuff()
   serviceA.doMoreStuff()
   serviceB.yawn()
}

But the caveat (and reason for this post) is that in the above scenario will only create a new transaction if one does not already exist. If the above is placed in a controller it will work fine. If it's placed in a service with

static boolean transactional = false

it will work fine, PROVIDING that the service is being called by a controller or other non-transactional service, however if your non-transactional service is called by a transactional one, the withTransactions block will have no effect.

Effective testing on Grails

2008-09-25T17:26:00.003+01:00

Gus and I will be presenting on "Effective Testing on Grails" at the London Groovy and Grails User Group next week Wed 1st of October, @18.30. Please feel free to attend if this seems relevant to you. A synopsis of the talk and registration for the event are all available at the following link.

http://skillsmatter.com/event/java-jee/ggug

The UI decisions behind a typical web form

2008-09-25T15:51:00.004+01:00

I stumbled across a very useful article recently and wanted to share it around. The article discusses the best placement of form submit/cancel buttons for a typical web form and backs this up with Heat Maps showing where the user spent time looking at elements on the page etc.

Check out the Primary & Secondary Actions in Web Forms blog post yourself.

There are a lot of very useful posts on that blog - all of them focused on useability and the "dark art" of UI decision making.

Inspecting Response Headers in the browser

2008-09-25T14:57:00.005+01:00

I needed to inspect the headers being sent to various browsers today. Specifically we wanted to find out how Yahoo was able to deliver a "fresh" CAPTCHA image when you used the browser back button to return to the page with the CAPTCHA image on it.

We observed that this happened even with Javascript disabled, and so came to the conclusion that it must be something in the response headers.

In Firefox (v1.5+) there is a really useful extension that I have mentioned before (the Web Developer Toolbar extension) which can show the headers using Tools -> Web Developer -> Information -> View Response Headers.

When using Internet Explorer a solution is to use the IE HTTP Headers plugin (for IE v5.0+) which is available at http://www.blunck.se

For those interested, the solution to the problem was to add the following headers to the response:

static void doNotCacheResponse(HttpServletResponse response) {
 response.setHeader('Pragma', 'no-cache')
 response.setHeader('Cache-Control', 'no-cache,no-store')
 response.setHeader('Expires', '-1')
}

We've tested that this works to force a re-request using Firefox 2, Firefox 3, IE6, IE7, Chrome, Safari 3 and Opera 9 - which is nice!

Careful with what you Query.

2008-09-23T09:04:00.002+01:00

The F5 BigIP load balancers have a feature called iRules. They allow you to manipulate and manage traffic in the Load Balancer.

They are really useful for checking or modifying cookies [HTTP::cookie], the host domain [HTTP::host], path [HTTP::path] and query parameters [HTTP::query] of URLs and requests.

A sligth word of warning though.... any checks made on the QUERY of a HTTP request actually search the URI of the referer as well. As an example, the following iRule would send a HTTP 200 response if you were either requesting a URL with a query parameter of "ThereIsAQuery" OR if you're refer page's URL included a query parameter of "ThereIsAQuery".

when HTTP_REQUEST {
  if {
      ([HTTP::QUERY] eq "ThereIsAQuery")
     }
  {
    HTTP::respond 200 content "<html><body>There is a Query</body></html>"
  }
}

Other attributes such as [HTTP::host], [HTTP::path], and [HTTP::uri] are only valid for the requested URL. They are not checked in the referer header.

Command Objects Revisited

2008-09-15T17:26:00.007+01:00

I posted yesterday about command objects but a conversation I had this morning made me realise I'd neglected to mention one of the interesting and less obvious ways to use them.

Imagine a scenario where you have a form that is basically for saving domain object instances but has a few other fields, not directly properties of the domain object in question but things like choices of how to set property values (e.g. a radio button to choose whether or not to update the timestamp on the object), workflow type instructions to the controller (e.g. preview this object after saving it), etc. This is a very common scenario.

You can handle this neatly with command objects by nesting an instance of the domain class inside the command object class, e.g.

    class MyCommand {
        boolean updateTimestamp
        boolean previewAfterSave
        MyDomain domainObj
    }

Then name your form fields accordingly...

    <g:checkBox name="updateTimestamp" value="${command.updateTimestamp}" />
    <g:checkbox name="previewAfterSave" value="${command.previewAfterSave}" />
    <input type="hidden" name="domainObj.id" value="${command.domainObj.id}" />
    <input type="text" name="domainObj.name" value="${command.domainObj.name}" />
    <g:select name="domainObj.otherDomainObj" from="${OtherDomainClass.list()}" value="${command.domainObj.otherDomainObj.id}" optionKey="id" />

and so on.

Command Objects

2008-09-14T07:28:00.009+01:00

We have a bunch of horrible code in our codebase to do with binding form fields to objects. To be fair it dates from a time when we were a lot newer to Grails and Grails and its documentation was a lot less mature. The other day I had to add a date picker to one of our forms and write the binding code for it. The g:datePicker tag creates some select boxes and a hidden field. Unlike other implementations I've used in the past it doesn't use Javascript to compose the select box values into the hidden field on form submit. Instead the hidden field has the value "struct" which I guess is a hint to GrailsDataBinder. To cut a long story short the code I ended up writing is pretty nasty. Especially considering that GrailsDataBinder is perfectly capable of handling this kind of binding without any code being written.

Instead of all this tedious messing about with params Grails controllers can, as most of you probably know, bind form submissions to domain objects or command objects. Binding to domain objects works by doing domainObj.properties = params or using the bindData method. So much most of us are used to (even if you wouldn't think so from looking at some of our code).

To have your controller action use a command object you simply specify it as an argument to the action closure. e.g.

    def save = { SaveCommand command ->
        if (command.hasErrors()) {
            render(view: 'index', model: [command: command])
        }
        // do some stuff
    }

The binding is done for you. As you may guess from the command.hasErrors() above, command objects can have constraints exactly as domain objects do. They can even have services and other Spring application context artefacts wired in automatically just like controllers and services can.

A form that contained...

    <input type="text" name="name" value="${command?.name}" />
    <g:datePicker name="birthday" value="${command?.birthday}" />
    <g:select name="sandwich.id" from="${Sandwich.list()}" value="${command?.sandwich?.id" optionKey="id" />

Would bind to a command like this...

    class MyCommand {
        String name // simple property
        Date birthday // bound from multiple fields submitted by date picker
        Sandwich sandwich // bound domain object
    }

Unit testing is easy because you can simply construct the command object and pass it in to the controller action. You don't have to worry about setting a bunch of esoteric key/value pairs on params to get your controller's hairy binding logic to work. In an integration test, if you really want to, you can set params and test that the command binding works as you expect.

The only non trivial thing in unit tests (in integration tests this is not a problem) is testing command error handling. In unit tests commands won't have an errors property or hasErrors method like they do in a running app or integration test. I've come up with the following which I've raised as an enhancement request for the experimental testing plugin:

    def createCommand(Class clazz) {
        def command = clazz.newInstance()
        def commandErrors = new BeanPropertyBindingResult(command, GrailsClassUtils.getLogicalPropertyName(clazz.name, ''))
        command.metaClass.getErrors = {-> commandErrors }
        command.metaClass.hasErrors = {-> commandErrors.hasErrors() }
    }

    void testCommandWithBindingErrors() {
        def command = createCommand(MyCommand)
        command.errors.reject('birthday', 'nullable')
        controller.action(command)
        // assert the form is re-rendered with the command in the model, etc.
    }

Command objects can also be used to handle multipart upload forms...

    <input type="file" name="avatar" />

Maps to...

    class MyCommand {
        byte[] avatar
    }

If you need access to the uploaded file's metadata (content type, original filename, etc.) instead of declaring the property as type byte[] use MultipartFile. Either way the binding works seamlessly and allows you to apply constraints to the uploaded file.

The weak area in binding at the moment seems to be one-to-many domain class relationships (which is why I didn't jump into refactoring the code I mentioned at the start of this post). That does require you to write hairy binding code and is probably worth a post in and of itself.

None of the rest of us can fathom CSS either.

2008-09-12T10:38:00.003+01:00

Dave Minter has written a blog entry Float like a wasp that goes on about the inadequacies and problems faced with implementing CSS and the frustrations of HTML. I think that he's got a point when he says:

You are not alone. None of the rest of us can fathom CSS either.

He has a hit-list of things he'd like to see made more simple... and they include the usual bug-bears that we have all experienced at some stage (some more than others, obviously).

There are solutions to all the things he would like to see... but the key thing is that these solutions are invariably not intuitive and require arcane knowledge (and the patience of a saint) to ensure conformity.

I particularly liked this quote:

One of my tips for fixing all that irks with CSS would be to buy two or three of those books describing machiavellian ways to conspire against the deficiencies of CSS (anything with "hacks" or "tips" in the title) and then beat up the standard until the easiest approach to implementing everything in the book could be covered in a pamphlet.

I, too, would like to see a more structure simplification (rather that the current direction of CSS which seems to be adding to the mess).

Pessimistic Locking With Grails

2008-09-07T12:26:00.020+01:00

By default grails uses optimisic locking with versioning. In a nutshell this means that each of your domain objects is blessed with a version field and hibernate throws an exception if you ever try to update or lock an object with an old version number. For scenarios where you are happy to handle the exception, or just report the error to the user via a UI this is fine, however when this behaviour isn't ok you need to switch to pessimistic locking. This is where the fun begins.

Unfortunately the Grails pessimistic locking support is incomplete. There are scenarios when it will work, but scenarios when it wont. As of 1.03 the docs don't make this clear. For example, let's say you have a domain object called Account...

class Account {
  String name
  long balance

  static mapping {
     version false // Required to avoid stale object exceptions when hibernate attempts a lock
  }
}

Obviously updates to the account balance need to be atomic. The Grails docs suggest the following...

Account account = Account.lock(id)
account.balance += amount
account.save()

The resulting SQL will look something like

select id, name, balance from account where id = 123 for update;
update account set balance = 101 where id = 123;

The above will work fine PROVIDING the account entity is not already in the hibernate session. If the account entity was previously retrieved the resulting SQL only re-selects the id.

select id from acount where id = 123 for update;
update account set balance = 101 where id = 123;

The db row is still locked, but because the account entity has not been refreshed your application could be working with stale data. The annoyance with the Account.lock(id) approach is that you always need to know the entity's id without obtaining the entity. So your code can never do

Account account = Account.findByName(name) // Maybe in a controller or other service
...
account = Account.lock(account.id)
account.balance += amount
account.save()

My initial workaround to the above is to duck into the hibernate API and do the following

sessionFactory.currentSession.refresh(account, LockMode.UPGRADE)
account.balance += amount
account.save()

Which appears ensures you have fresh data and a db lock. However this is ONLY true if your entity does not have any eager relationships (one-to-ones are eager by default). Incredibly hibernate retrieves the eager relationships before calling select for update on the main entity, meaning that the relationship data could be stale. I've tried fixing this with cascade: 'lock' but it did SFA. It now appears the best (and I use that term loosely) way to ensure fresh, locked data is to do the following

account.lock() // versioning must be disabled
account.refresh()
account.balance += amount

An alternative is to call "discard" on the object before you lock it

account.discard()
account = Account.lock(account.id)
account.balance += amount
account.save()

An interesting aside, which has nothing to do with grails but everything to do with concurrency is how you prepare to update multiple objects, e.g.

accountsToBeUpdated.each { Account account ->  
  account.lock()
  account.refresh()
  account.balance += amount
  account.save() 
}

The above code might look OK at first glance, but what if two threads were to update an intersecting set of accounts?

Thread 1: accountA, accountB
Thread 2: accountB, accountA

If you were unlucky Thread 1 may obtain a lock on accountA at the same time that Thread 2 obtained a lock on accountB resulting in deadlock. The solution is to sort both sets of accounts using the same algorithm anywhere you attempt to locks multiple entites, e.g.

def sortedAccounts = accountsToBeUpdated.sort { a, b -> a.name <=> b.name }
sortedAccounts.each { Account account ->
  account.lock()
  account.refresh()
  account.balance += amount
  account.save() 
}

Another concurrency scenario is when you want to modify the many side of a one-to-many relationship, e.g.

class Person {
  Set subscriptions
}

In this case because a subscription is never shared between two people it may be ok to just lock the person object, e.g.

person.lock()
person.refresh()
person.addToSubscriptions(subscription)
person.save()

The reason I say 'may', is that this approach is dependent on all your other code taking the same approach when updating subscriptions and that it's not a performance bottleneck to lock the person. Choosing to lock the person is certainly an easy way to avoid creating duplicate subscriptions without the risk of a unique constraint violations.

Up The Limit

2008-08-31T16:01:00.005+01:00

Recently while using the searchable plugin with Grails I've had exceptions citing too many open files. Linux systems typically limit resources on a per-user basis by various criteria such as number of processes and number of open files. Running ulimit -a I see the following:

core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 36864
max locked memory       (kbytes, -l) 32
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 36864
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

On this system I can only open 1024 files at once - I can increase this limit in /etc/security/limits.conf by adding something similar to the following:

gus  soft nofile  16384
gus  hard nofile  16384

To ensure that PAM actually takes notice of these limits on login check /etc/pam.d/login for an entry like:

session    required     pam_limits.so

Can haz filez!

Unit Testing Controllers With The Testing Plugin

2008-08-28T17:24:00.005+01:00

Grails JUnit tests are, as you are no doubt aware, divided into unit tests and integration tests. Unit tests run fast, aren't dependent on the database or Spring context but mean you have to live without all the dynamic magic that Grails attaches to your artefacts (controllers, taglibs, domain classes, etc.) This can certainly lead to some sub-optimal test code.

Lets consider an example. I want to write some test coverage for a controller that looks up a domain object based on a parameter and places it in the model. Usual disclaimer: of course, in the real world I'd have written the test first, etc. Nothing much to the controller:

class LolrusController {
   def show = {
       def myLolrus = Lolrus.findByName(params.lolrusName)
       render(view: 'lolrus', model: [lolrus: myLolrus])
   }
}

and here's the domain class:

class Lolrus {
   String name
   String mood
   static hasMany = [posessions: Item]
   static constraints = {
       name(unique: true)
       posessions(validator: { posessions ->
           return posessions.any { it.type == 'bukkit' }
       })
   }
   String toString() { "Lolrus[$name]" }
   boolean equals(Object o) { return o instanceof Lolrus && o.name == name }
   int hashCode() { return 37 * name.hashCode() }
}

Okay, so a reasonable test (ignoring edge cases for now) could be to create a couple of domain objects, make a request to the controller and verify that the correct domain object is retrieved. Simple, right?:

class LolrusControllerTests extends GroovyTestCase {
   def controller
   def lolrus1, lolrus2

   void setUp() {
       controller = new LolrusController()

       lolrus1 = new Lolrus(name: 'Hugh')
       lolrus2 = new Lolrus(name: 'Alan')
       [lolrus1, lolrus2]*.save(flush: true)
   }

   void testShowActionFindsCorrectLolrusAndSticksItInModel() {
       controller.params.lolrusName = lolrus1.name
       def model = controller.show()
       assertEquals('/lolrus/show', controller.modelAndView.viewName)
       assertEquals(lolrus1, controller.modelAndView.model.lolrus)
   }
}

Unfortunately, not so simple. The test fails and reports

expected:<Lolrus[Hugh]> but was:<null>

What we've forgotten of course is that Lolrus has a mandatory field mood and a custom validation on its posessions association that requires it to have a bukkit. The two Lolrus instances we tried to create in setUp couldn't be saved. Great. Well, let's just add that to the test set up:

    void setUp() {
       controller = new LolrusController()

       lolrus1 = new Lolrus(name: 'Hugh', mood: 'worryingly cheerful')
       lolrus1.addToPosessions new Item(type: 'bukkit')

       lolrus2 = new Lolrus(name: 'Alan', mood: 'mildly distressed')
       lolrus2.addToPosessions new Item(type: 'bukkit')

       [lolrus1, lolrus2].each {
           assert it.save(flush: true), it.errors
       }
   }

Okay, it works but look how much of what's going on in setUp is actually nothing to do with anything the test cares about. This is a very simple example with a single test case. Imagine how much worse that setUp method is going to get when we add more tests or the Lolrus domain object gets more complex... Imagine the refactoring required when the Lolrus domain object acquires other constraints... Imagine the query is more complex requiring more domain objects to be set up to really prove it works...

Okay, we could mock the Lolrus class, but the code for that isn't much cleaner and alarm bells start ringing when you start using mocks in an integration test. Shouldn't this be a unit test? Sure, that would be nice but we're using the render dynamic method in our controller, we'll have to mock that out on the metaClass, mock out the findByName method on Lolrus, etc. etc. Too much hassle right? Not any more.

grails install-plugin testing

Allows us to (among other things) write controller unit test cases like this:

class LolrusControllerUnitTests extends grails.test.ControllerUnitTestCase {
   def controller
   def lolrus1, lolrus2

   void setUp() {
       super.setUp()
       controller = new LolrusController()

       lolrus1 = new Lolrus(name: 'Hugh')
       lolrus2 = new Lolrus(name: 'Alan')
       mockDomain(Lolrus, [lolrus1, lolrus2])
   }

   void testShowActionFindsCorrectLolrusAndSticksItInModel() {
       controller.params.lolrusName = lolrus1.name
       controller.show()
       assertEquals('show', renderArgs.view)
       assertEquals(lolrus1, renderArgs.model.lolrus)
   }
}

Much neater. We're not worrying about the constraints required to set up valid Lolrus instances, but neither are we having to mock out the findByName method.

The mockDomain method allows you to set up some domain objects for which all the usual Grails domain class dynamic methods will work - including findBy..., addTo..., etc. The collection passed as the second argument to mockDomain defines all the instances that exist and they can be retrieved just as a real domain object could. The mockDomain method co-exists happily with regular Groovy mocks and the testing plugin also provides some lighter mocking capabilities via its own mockFor method.

Additionally the controller's render method and params property work without any work on our part. This is true of all the dynamic properties and methods of the controller, request, response, params, controllerName, redirect, etc.

Not only that, but this test runs faster than the integration test as it's not configuring the Spring container or an in-memory database. That speed difference only grows as the project becomes more complex - it's a couple of seconds between the two tests here, but in a more complex project it's a lot more.

There are a couple of gotchas (the plugin isn't fully mature). First off you need to remember to call super.setUp() in order that all the Grails stuff happens. The ControllerUnitTestCase class will figure out based on the class naming convention which controller class it needs to mock up. Secondly, the domain class get method doesn't seem to work with non-standard identifier types at the moment (i.e. anything other than a Long). However, this unit testing support is intended to get rolled into Grails 1.1 so it should improve rapidly.

This is only part of what the testing plugin can do. Support for unit testing domain class constraints, etc. is also included. I've also been working on a TagLibUnitTestCase that works along the same lines as ControllerUnitTestCase.

A Productivity Hint Explained

2008-08-27T22:30:00.014+01:00

In an earlier post, I mentioned that writing code that looks like this:

myObject."$localVariable".attribute = 42

... is a bad idea for three reasons.

Here's the three reasons I was looking for:

1 Violating encapsulation

You don't want to manipulate the guts of other objects directly. If we made a habit of just reaching into the objects near the current chunk of code and doing whatever we want, we'd soon have a codebase which is incredibly difficult to change because whatever piece of code you're looking at now could be potentially affected by another piece of code writing over local state.

Unit tests work by exercising the public interface of the object i.e. those services that are made available to the community of other collaborating objects in the system. If any old thing can reach in and mess around with the internal state then there wouldn't be any point in having unit tests because you can't rely on the assumptions on which those tests are based.

Every programmer learning his or her first object language used to be taught encapsulation, but I'm still amazed how many really experienced people still don't seem to take this seriously. It's not an aesthetic thing. If you don't get this please give up.

Incidentally the Sharble and Cohen study from Boeing provides some evidence that designs that use lots of getters and setters to centralise design in a few classes also makes the code slower because you need to execute more instructions just to (needlessly) push all the data around.

2 It makes the object graph very fragile

This is about the Law of Demeter. Basically when you write the expression myObject.x.y.z.doWhatever() you're assuming that there's a whole bunch of things stuck together, at the end of which is an object that can doWhatever().

The code "myObject.x.y.z.doWhatever()" makes a lot of needless assumptions:

myObject knows x
x knows y
y knows z

... and all it really needs is to be given z as a parameter.

If any of the relationships between anObject, x, y, and z changes we get an entirely preventable breakage.

Some people might find it difficult to restructure this code so that you get access to z without rummaging through an object graph (what some people still seem to call a "data structure".) See what I've got to say below about responsibility-driven design to help with this.

3 It's a gratuitous abuse of metaprogramming.

In other words, it's slower and far more complicated that it should be. Incidentally, the business with ."$whatever" is hiding $whatever from compiler because it needs to be evaluated at runtime rather than compile time. It also makes things an order of magnitude more difficult for a refactoring IDE. I doubt that there's an easy way for the IDE to refactor things automatically when you rename an attribute or to even notice that something's wrong when you move the attribute to another class.

anObject."$whatever" = 42

The metaprogramming here with ."$whatever" only became necessary because we're trying to poke away at the guts of another object - something we shouldn't be attempting to do in the first place. We should get anObject to do something, rather than just treat it as a passive store of data (what some people used to more honestly describe as a "record".)

At least those are the three most pressing things that spring to mind.

A funnier treatment of points 1 and 2 can be found in The Tragic Tale of POTS and his friend SPOT. Grails developers who've read about POTS will appreciate the irony of having a DogWalkingService.

How to avoid writing this kind of thing / how to fix it

The best way to avoid writing code like this is to think in terms of "responsibility-driven design" - that's the fine art of putting the data right next to the code that needs it - in the same class.

A long time ago, people doing extreme programming knew about this. In fact, RDD was invented by the guys who invented extreme programming. Shame it's not so widely used or publicised.

Doing CRC sessions is a very agile-friendly way to do RDD - and was done on the first ever XP project C3.

A lot of this stuff seems obvious (at least points 1 and 2). It's surprising how many people don't seem to get this.

Extreme programming is founded on the concept of lowering the cost of change. The points I've made here aren't theoretical or style-related. This is vital. This is about the basic stuff that makes everything else we do possible.

Deploy extra WARs with your Grails App

2008-08-27T17:18:00.009+01:00

Should you find yourself needing to deploy other web-app WARs with your Grails app, when you want the Grails app under development to run at the root context, this is for you.

This applies specifically to running the app on command line (grails run-app), as compared to packaged WARs; you would not need to do this in production for Jetty since Jetty applies the logic below already - we needed this to automate tests which spanned our app and its integration with another.

First off, if you haven't already deployed your Grails app to the root context, you should refer to this.

Now that you are running your grails app at the root context, you will need to find a suitable location (MY_OTHER_WARS below) to place any additional WARs so that the Grails startup can pick up on it.

This location is referenced in the $YOUR_PROJECT_HOME/scripts/Events.groovy where you will need to add the following

eventConfigureJetty = { Server server ->

def grailsApp = server.handler
server.removeHandler(grailsApp)

def handlers = []
handlers << grailsApp
ContextHandlerCollection contextHandlerCollection = new ContextHandlerCollection()

warProjects.each {
   WebAppContext otherWarWebContext = new WebAppContext("${basedir}/MY_OTHER_WARS/${it}.war", "/${it}")
   handlers << otherWarWebContext
}

handlers.each {
   contextHandlerCollection.addHandler(it)
}

server.setHandlers(contextHandlerCollection)
}

This ensures that when you start your application on the command line, the default mode of adding the grails app to the Jetty server is overridden with a Jetty ContextHandlerCollection.

The problem with not doing so and just adding more Handlers to the Jetty server means that Grails intercepts all contexts/URLs on the app, and will fail to pass the call for a context onto the appropriate other WAR.

Connection Pooling

2008-08-27T11:18:00.004+01:00

I've noticed there isn't that much information about how to setup connection pooling with grails on the mailing lists so I figured I'd put up a quick post about it. In this example I want to create c3p0 connection pools for all profiles that use postgreSQL.In resources.groovy we add the following:

import com.mchange.v2.c3p0.ComboPooledDataSource
import org.codehaus.groovy.grails.commons.ConfigurationHolder

def config = ConfigurationHolder.config

if(config.dataSource.url =~ 'postgresql') {
     dataSource(ComboPooledDataSource) {
      driverClass = 'org.postgresql.Driver'
      user = config.dataSource.username
      password = config.dataSource.password
      jdbcUrl = config.dataSource.url
         maxStatements = 180
         minPoolSize = 10
         acquireIncrement = 5
         maxPoolSize = 100
     }
}

and something like this to your configuration (Config.groovy or wherever you keep your profile configurations):

dataSource {
 username = 'postgres'
 password = ''
 url = 'jdbc:postgresql://127.0.0.1:5432/databasename'
}

Fire it up with the right profile and you should see the 10 initial connections to your postgres database by running ps -U postgres u at the console.

Can haz cuneckshun puling!

Using Liquibase DropAll Automagically

2008-08-26T09:41:00.005+01:00

If you're using the grails liquibase plugin you might find it useful to have it run on application startup (e.g. when the war file is deployed) rather than using the grails command line target (i.e. grails migrate). One way to achieve this is to add an entry for the liquibase spring bean in your resources.groovy like so:


liquibase(SpringLiquibase) {
 dataSource = dataSource
 changeLog = "classpath:migrations/changelog.xml"
 executeEnabled = true
}

For certain environments (continuous build etc.) it may also be useful to have liquibase clear out the database and build the schema from scratch. The main Liquibase class has a dropAll() method that looks like it'll do the trick (there's also a 'grails drop-all' command), so let's create a spring bean that'll run before the liquibase migration kicks in (it'd be a bit pointless to blow away the database after you've just created it). The spring 'depends-on' attribute should work nicely here but it would appear that grails' BeanBuilder does not support attributes (foiled again). We can, however, implement BeanFactoryPostProcessor to achieve the desired effect:


import java.sql.Connection
import liquibase.spring.SpringLiquibase
import org.springframework.beans.factory.config.BeanDefinition
import org.springframework.beans.factory.config.BeanFactoryPostProcessor
import org.springframework.beans.factory.config.ConfigurableListableBeanFactory

class LiquibaseDropAll extends SpringLiquibase implements BeanFactoryPostProcessor {
 
 String runBefore = 'liquibase'
 
 void afterPropertiesSet() {
  Connection conn = null
  conn = getDataSource().getConnection()
  super.createLiquibase(conn).dropAll()
 }
 
 //make liquibase bean depend upon this bean so that dropAll runs before migration
 void postProcessBeanFactory(ConfigurableListableBeanFactory beanFactory) {
  BeanDefinition beanDefinition = beanFactory.getBeanDefinition(runBefore)
  beanDefinition.setAttribute('depends-on', beanName)
 }
 
}

The last thing to do is add this bean to your resources.groovy, most probably with a conditional based on environment:


     def config = org.codehaus.groovy.grails.commons.ConfigurationHolder.config
  if(config.dataSource.liquibase.dropAll) {
   liquibaseDropAll(LiquibaseDropAll) {
    dataSource = dataSource
   }
  }

Now just add something like:


dataSource {
      ...
 liquibase.dropAll = true
}

to the environments you want to liquify and you're away.

Lock, Stock and Two Fuming Developers

2008-08-25T22:05:00.002+01:00

Oh Hai!

If you have the misfortune of having to deal with locks in your postgreSQL database, run the following to find out if there are any nasty ExclusiveLocks (whole table locks) on the loose:

select * from pg_locks where mode = 'ExclusiveLock';

There are 4 types of lock in postgreSQL, listed here in order of decreasing niceness:

AccessShareLock
RowShareLock
RowExclusiveLock
ExclusiveLock

If you're running some tests that create your lock condition it's sometimes useful to quickly look at what's happening on the console:

watch -n1 ps -U postgres u

Lastly, make sure your postgreSQL version doesn't have REPLICA printed on the side :-/

When Groovy Property Access Attacks!

2008-08-19T13:17:00.005+01:00

A quick best-practice hint...

Groovy's property access generally leads to terser code, however there's (at least) one time when using it is not a good idea; when you want to know what class an object is.

println x.class.name may work for most types of object, but what happens if x is, hmm let's say, a Map?

You probably won't be surprised to find the following code fails.

    def x = [:]
    assert x.class == java.util.LinkedHashMap

Even more entertainingly, this code will throw NullPointerException

    def x = [:]
    assert x.class.name == 'java.util.LinkedHashMap'

Overriding constraints

2008-08-12T14:08:00.006+01:00

I didn't know you could override constraints until the other day. Here is an example of overriding and then testing the constraints with the new testing plugin. Testing constraints with the new plugin is so easy I suggest every team uses it. We started using it for our team. Here's the code:


class Parent {
  String name
  String body

     static constraints = {
         name(blank: false, nullable:false)
         body(blank: false, nullable:false)
      }
}

class Child extends Parent{
      static constraints = {
        name(blank: true)
        body(blank: true)
      }
}


------------Tests using the testing plugin ---------------
import grails.test.GrailsUnitTestCase
import grails.test.MockUtils

class ChildTests extends GrailsUnitTestCase{

 void testConstraints() {
      // Mock the validate() method.
      registerMetaClass(Child)
      MockUtils.prepareForConstraintsTests(Child)

      registerMetaClass(Parent)
      MockUtils.prepareForConstraintsTests(Parent)

      def testInstance = new Child()
      def errors = testInstance.validate()
      assertEquals 0, errors.size()

      // Test the parent constraints
      testInstance = new Parent(name:' ' )
      errors = testInstance.validate()
      assertEquals 2, errors.size()
      assertEquals "blank", errors["name"]
      assertEquals "nullable", errors["body"]
   }
}

Happy testing :)

Today's Object-Oriented Productivity Hint

2008-08-11T23:45:00.008+01:00

Please try using encapsulation before attempting metaprogramming or functional programming.

Exercise:

Find three reasons not to write this kind of code:

myObject."$localVariable".attribute = 42

This kind of code is wrong on so many levels that if Ali G wrote web apps, he'd love it.

Last week's Retrospective

2008-08-11T13:50:00.003+01:00

On Wednesday I ran my first retrospective. I decided to focus on some of the positive values that make up a good team: Simplicity, Feedback, Courage, Respect and Communication.

Here is a link to my write up of the retrospective.

Sending emails from Grails apps

2008-08-11T08:48:00.004+01:00

We're trying this on for size...

import org.springframework.mail.MailSender
import org.springframework.mail.SimpleMailMessage

class EmailService {

    boolean transactional = false
    MailSender mailSender
    SimpleMailMessage mailMessage // A template message auto-wired from resources.groovy

    boolean sendMessage(String to, String subject, String body) {
        boolean result = true
        try {
            SimpleMailMessage message = new SimpleMailMessage(mailMessage)
            message.to = [to]
            message.subject = subject
            message.text = body
            mailSender.send(message)
        } catch(Exception e) {
            log.error("Unable to send message to ${to} with subject ${subject}", e)
            result = false
        }
        return result
    }
}

import org.springframework.context.MessageSourceAware
import org.codehaus.groovy.grails.web.pages.GroovyPagesTemplateEngine
import org.springframework.context.MessageSource
import org.springframework.context.i18n.LocaleContextHolder
import org.codehaus.groovy.grails.web.pages.GroovyPageTemplate

class NotificationService implements MessageSourceAware {

    EmailService emailService
    GroovyPagesTemplateEngine groovyPagesTemplateEngine
    MessageSource messageSource

    boolean confirmRegistration(Person person) {
        String subject = getMessage('email.registration.confirmation.subject', [])
        return emailService.sendMessage(person.email, subject, getEmailBody('registrationConfirmation', [person: person]))
    }

    boolean registrationSuccess(Person person) {
        String subject = getMessage('email.registration.successful.subject', [])
        return emailService.sendMessage(person.email, subject, getEmailBody('registrationSuccessful', [person: person]))
    }

    private String getEmailBody(String templateName, Map model) {
        return render("/modules/emails/_${templateName}.gsp", model)
    }

    private String render(String templateUri, Map model) {
        // I wonder if templates can be cached        
        GroovyPageTemplate template = groovyPagesTemplateEngine.createTemplate(templateUri)
        Writable w = template.make(model)
        StringWriter sw = new StringWriter()
        PrintWriter pw = new PrintWriter(sw)
        w.writeTo(pw)
        return sw.toString()
    }

    private String getMessage(String code, List args) {
        return messageSource.getMessage(code, (Object[]) args, LocaleContextHolder.locale)        
    }

    void setMessageSource(MessageSource messageSource) {
        this.messageSource = messageSource
    }
}

Few gotcha's though...

1. TagLibs don't work in the GSP templates. This causes a real headache if you need to use <g:message ... /> etc.

2. I'm not sure whether the templates can be cached. It's probably expensive to keep creatating them each time.

Testing

We're using wiser to stub the SMTP server, e.g.

import org.subethamail.wiser.Wiser
import org.subethamail.wiser.WiserMessage

class EmailServiceTests extends GroovyTestCase {

    Wiser wiser
    EmailService emailService

    void setUp() {
        wiser.messages.clear()
    }

    void testThatICanSendAnEmail() {
        assertEquals(0, wiser.messages.size())
        emailService.sendMessage('roundhouse@beta.xyz.com', 'Cool beans', 'Well done, you managed to register')
        assertEquals(1, wiser.messages.size())

        WiserMessage message = wiser.messages.iterator().next()
        assertEquals('roundhouse@beta.xyz.com', message.envelopeReceiver)
        assertEquals('Cool beans', message.mimeMessage.subject)
        assertEquals('Well done, you managed to register', message.mimeMessage.content)
    }
    
    void testThatEmailServiceReturnsFalseOnFailure() {
        emailService.log.logger.setLevel(org.apache.log4j.Level.OFF)
        wiser.stop()
        assertFalse('Unexpected successful email send', emailService.sendMessage('roundhouse@beta.xyz.com', 'subject', 'message'))
        wiser.start()
        emailService.log.logger.setLevel(org.apache.log4j.Level.ERROR)        
    }    
} 

We also use wiser in our selenium tests by adding the following action to our test fixture controller,


def smtp = {
    List messages = wiser.messages.collect { message ->
        String to = message.envelopeReceiver
        String from = message.envelopeSender
        String subject = message.mimeMessage.subject
        String content = message.mimeMessage.content
        return [to: to, from: from, subject: subject, content: content]
    }
    Map model = getJsonModel(messages)
    render(view:'index', model: model)
}



Config

It's easy to pick whether to use the real smtp server or wiser in resources.groovy

switch(GrailsUtil.environment) {
  case 'prod':
            mailSender(org.springframework.mail.javamail.JavaMailSenderImpl) {
                host = 'mail1.xyz.com'
            }
            mailMessage(org.springframework.mail.SimpleMailMessage) {
                from = 'info@ xyz.com'
            }
        break
        case ['qa', 'beta']:
            mailSender(org.springframework.mail.javamail.JavaMailSenderImpl) {
                host = 'mail1.xyz.com'
                username = 'info@beta.xyz.com'
                password = 'shh'
            }
            mailMessage(org.springframework.mail.SimpleMailMessage) {
                from = 'info@beta.xyz.com'
            }
        break
        default:
            wiser(org.subethamail.wiser.Wiser) { bean ->
                bean.initMethod = 'start'
                bean.destroyMethod = 'stop'
                bean.lazyInit = true
                port = 2500
            }
            mailSender(org.springframework.mail.javamail.JavaMailSenderImpl) {
                host = 'localhost'
                port = 2500
            }
            mailMessage(org.springframework.mail.SimpleMailMessage) {
                from = 'info@beta. xyz.com'
            }
        break
    }



Ruby on Rails vs Groovy on Grails vs Cobol on Cogs
2008-08-06T10:51:00.001+01:00
Everyone has heard of rails and grails but check out cobol on cogs:

http://www.coboloncogs.org


Where did the Response go in 1.0.3?
2008-07-29T09:40:00.002+01:00
We noticed that in the move from Grails 1.0.2 to 1.0.3, the response object inside Spring's RequestContextHolder has seemingly gone but it looks like it's always been somewhere else:

So:

def response = org.springframework.web.context.request.RequestContextHolder.requestAttributes.response

Becomes:

def response = org.springframework.web.context.request.RequestContextHolder.requestAttributes.currentResponse



Worth keeping an eye out for.




Experimental testing plugin
2008-07-24T09:24:00.002+01:00
Saw this post on the grails mailing from a grails committer, Peter Ledbrook,  might be worth checking out:

Hi everyone,
 
 I have been working on some improvements to the Grails testing
 framework recently which will go into Grails 1.1. Some of the support
 classes though will work with Grails 1.0.x, so I have packaged them up
 as a plugin. You can install it using:
 
   grails install-plugin testing
 
 Before you rush to download it, please be warned that it is nowhere
 near a complete implementation yet, particularly
 ControllerUnitTestCase. However, I would like to solicit feedback
 early on so that we can really nail the problems that people are
 having in testing. And if you can supply patches, great!
 
 So, what lurks in the plugin? I'm planning to get some documentation
 up soon, but for now I'll cover some common use cases. Before that,
 the current set of classes are designed to be used in unit tests. As a
 general trend, we want to encourage people to write unit tests in
 preference to integration tests. We also want to make sure that unit
 tests can be run from within an IDE, i.e. there should be no
 requirement on a running Grails instance.
 
 For the first example I'll show you how to test domain constraints.
 Domain classes often lack logic, and so they don't get tested.
 However, plenty of errors can creep in to the constraints so it's
 worth validating them.
 
   class MyDomain {
       String name
       Integer age
 
       static constraints = {
           name(nullable: false, blank: false)
           age(nullable: false, min: 10, max: 100)
       }
   }
 
   class MyDomainUnitTests extends GrailsUnitTestCase {
       void testConstraints() {
           // Mock the validate() method.
           registerMetaClass(MyDomain)
           MockUtils.prepareForConstraintsTests(MyDomain)
 
           // Test that a fresh new domain instance fails validation on
           // the "nullable: false" constraints.
           def testInstance = new MyDomain()
           def errors = testInstance.validate()
           assertEquals 2, errors.size()
           assertEquals "nullable", errors["name"]
           assertEquals "nullable", errors["age"]
 
           // Test the other constraints
           testInstance = new MyDomain(name: "  ", age: 5)
           errors = testInstance.validate()
           assertEquals 2, errors.size()
           assertEquals "blank", errors["name"]
           assertEquals "min", errors["age"]
       }
 }
 
 The main things to note here are:
 
 1. We sub-class GrailsUnitTestCase
 2. "registerMetaClass()" and "MockUtils.prepare...()" add the
 validate() method to the domain class
 3. We create instances of the domain class, call validate(), and check
 whether any errors were found
 
 On (2), the two lines will be replaced by a method on
 GrailsUnitTestCase in the near future. On (3), the validate() method
 returns a map of validation errors. Note that we check for the name of
 the constraint, not the i18n error code associated with the
 constraint.
 
 That's it for domain constraints. For other unit tests, such as for
 services and controllers, GrailsUnitTestCase provides the method
 "mockDomain(Class, List)":
 
   void testMethod() {
       mockDomain(MyDomain, [
               new MyDomain(name: "John Smith", age: 35),
               new MyDomain(name: "Alice Smith", age: 64),
               new MyDomain(name: "Irene Pane", age: 22),
               new MyDomain(name: "Patrick Rose", age: 45) ])
 
       def testService = new MyService()
       testService.doSomething()
       ...
   }
 
 The method injects working versions of the dynamic methods and
 properties that normally go with domain classes, in particular the
 dynamic finders. Where appropriate, these injected methods/properties
 use the given list of domain instances as a source of data. For
 example, if MyService.doSomething called a dynamic finder like this:
 
   MyDomain.findByNameLike("%Smith")
 
 the mock property would return a list containing the "John Smith" and
 "Alice Smith" MyDomain instances in that order. Another useful method
 is "mockFor()" which returns an object that you can use pretty much
 like the Groovy MockFor class:
 
   def mockControl = mockFor(MyDomain)
   mockControl.demand.save(1..1) {-> return true}
   mockControl.demand.static.findByName(1..1) { name -> return [] }
 
 The best thing about this method is that it works seemlessly with
 "mockDomain()", i.e. you can readily override the methods provided by
 "mockDomain()" via the mock object returned by "mockFor()".
 
 Finally, there is a ControllerUnitTestCase, but it is in the very
 early stages of development. I recommend you only use it if you're
 willing to patch it up with the functionality you need. It will
 automatically inject all the normal controller properties and methods,
 but not much else. However, one nice feature I have implemented
 already is the ability to set the body of the request to some XML
 (either a string or builder markup), particularly useful for REST
 controllers based on XML. I need to add support for JSON too.
 
 That's it for now. I would certainly give GrailsUnitTestCase a go
 because I have already found it much easier to write Grails unit tests
 than I used to. If you want to raise issues or provide patches, please
 add them to the main Grails JIRA, setting the fix version to 1.1 and
 assigning them to me (username "pledbrook").

Cheers,
Glenn




Does the noise in my head bother you?
2008-07-19T15:04:00.003+01:00
A quick post for those of you using the dynamic addTo* methods with manyToMany relationships under grails 1.0.3. You may notice alot of noise in your logs / test results - this is down to a stray println in DomainClassGrailsPlugin.groovy (line 173) which does the following:

println "$obj - ${obj.properties} - ${prop.otherSide.name}" 

Options: patch grails, wait for 1.0.4, ignore it. 


Tricking Grails into persisting Enum properties
2008-07-15T13:34:00.002+01:00
Although Grails 1.0.3 implements persistence of java.lang.Enum fields in GORM, unfortunately it sticks with the misguided Hibernate convention of persisting the Enum ordinal rather than the String value. In Hibernate at least you can override this default, in Grails currently you can't.

In the meantime, here's a workaround pattern that should be amenable to a low-impact refactor once Grails 1.0.4 drops.

    enum Flavr { /* some enum instances */ }

    class Lolcat {
        String flavrName
    
        static transients = ['flavr']

        static constraints = {
            flavrName(nullable: true, inList: Flavr.values().collect{it.name()})
        }
        
        static mapping = {
            columns {
                flavrName(name: 'flavr')
            }
        }
    
        Flavr getFlavr() {
            return flavrName ? Flavr.valueOf(flavrName) : null
        }
    
        void setFlavr(Flavr flavr) {
            flavrName = flavr?.name()
        }
    }


What we've done is create a transient Enum property backed by a String that GORM uses to persist the value. The constraint prevents any erroneous values being set directly to the String property. The column name mapping is an attempt at future proofing, so the column name won't need to be changed once 'flavr' becomes a full-blown persistent Enum.

As and when that's possible you simply replace 'String flavrName' with a 'Flavr flavr' and remove the transients, constraints and mapping blocks and the getter and setter.

Two downsides are 1) that dynamic finders and criteria queries must operate on real properties and not transient ones and 2) that you can pretty much guarantee some muppet will directly access 'flavrName' somewhere - something you can guard against much more effectively in Java than you can in Groovy.


Groovy operators
2008-07-10T12:06:00.010+01:00
Max suggested I post something about this, so...

Groovy has some operators that vanilla Java doesn't. Here's a quick reference:

<=> (Spaceship)
Example:
a <=> b is equivalent to a.compareTo(b)

?: (Elvis)
Example:
x ?: y is equivalent to x != null ? x : y

=~ (Find)
Example:
Matcher m = "abc" =~ /a/

==~ (Match)
Example:
assert "abc" ==~ /\w+/ Note this is a full match not a partial.

~ (Create pattern)
Example:
~/abc/ is equivalent to new Pattern(/abc/)

*. (Spread)
Example:
a*.b is equivalent to a.collect { it.b }

.& (Method reference)
Example:
Closure c = a.&b gives you a reference to the method "b()" on object "a" that you can pass around as a Closure.

.@ (Property access)
Example:
def c = a.@b gives you the value of the property "b" on object "a" directly (i.e. without going through the getter). This can be useful on horrible classes like java.awt.Dimension where the type of a public property is different to the return type of its getter. Most other uses of it are probably best avoided.

?. (Null-safe dereference)
Example:
a?.b is equivalent to a == null ? null : a.b

as (Type coercion)
Example:
new Date() as String is equivalent to new Date().toString() which is a trivial example but you can do quite exciting type conversions by overriding asType.

is (Object identity)
Example:
a is b is equivalent to a == b in Java.

kthxbye.


Ooops Mocks
2008-07-08T23:00:00.002+01:00
Why do this ...
void testOne() {
   def result
   mockServiceA.use {
      mockServiceB.use {
         mockServiceC.use {
            result = testMe.someMethod('a', 'b')
         }
      }
   }
   assertEquals(1, result)
}

void testTwo() {
   def result
   mockServiceA.use {
      mockServiceB.use {
         mockServiceC.use {
            result = testMe.someOtherMethod('a', 'b')
         }
      }
   }
   assertEquals(1, result)
}


When you can do this...

void testOne() {
   def result = executeWithMocks { testMe.someMethod('a', 'b') }
   assertEquals(1, result)
}

void testTwo() {
   def result = executeWithMocks { testMe.someOtherMethod('a', 'b') }
   assertEquals(1, result)
}

private Object executeWithMocks(Closure closure) {
   def result
   mockServiceA.use {
      mockServiceB.use {
         mockServiceC.use {
            result = closure()
         }
      }
   }
   return result
}



Criteria queries + associations = confusion
2008-07-04T16:57:00.008+01:00

Since we've been bitten by this problem twice in the same week now I thought it was worth a post.

Consider these classes:
    class Author {
        String name
        static hasMany = [books: Book]
    }

    class Book {
        String name
        static belongsTo = [author: Author]
    }

Then let's attempt to find all authors who have written a book with a particular name:
    Author wg = new Author(name: 'William Gibson')
    wg.addToBooks new Book(name: 'Pattern Recognition')
    wg.addToBooks new Book(name: 'Spook Country')
    assert wg.save(flush: true)

    sessionFactory.currentSession.clear()

    List authors = Author.withCriteria {
        books {
            eq('name', 'Spook Country')
        }
    }
    assertEquals 1, authors.size()
    assertEquals 2, authors[0].books.size() // FAILS result is 1

When you look at how Hibernate implements this query under the hood it's reasonably clear what's going on. It will be doing something roughly along the lines of:
    select * from author a, book b where a.id = b.author_id and b.name = ?

The problem is it then makes the mistake of thinking that the books collections of the author objects it has found can be initialised based on that result set.

Where this problem has bitten us is when objects fetched by such a query are also used by an unrelated bit of code that doesn't bother reading from the database as it quite reasonably thinks (some of) the objects is needs are in the Hibernate session already. Unfortunately, the view module related to this other bit of code is reliant on the elements of the collection that weren't loaded by our defective query. I'm sure you can imagine the fun to be had debugging that kind of problem.

I've been hacking around with the criteria attempting various permutations such as:
    // DOES NOT WORK
    List authors = Author.withCriteria {
        'in'('books.name', 'Spook Country')
    }

and
    // ALSO DOES NOT WORK
    List authors = Author.withCriteria {
        'in'('books') {
            eq('name', 'Spook Country')
        }
    }

But I've got nowhere yet. The best I've managed to do is:
    List authors = Author.withCriteria {
        books {
            eq('name', 'Spook Country')
        }
    }
    // if you have a weak stomach look away now
    authors*.refresh()

If you're wondering, yes that is going to execute n+1 selects.

Another option is to give up and use an HQL query.

This problem applies equally to one-to-many (and presumably many-to-many) associations using Set, List or Map.


Good Unit Testing Practice
2008-07-02T14:48:00.008+01:00
A common but misguided practice that is often seen in unit test designs, is to test some kind of a false moniker on an object, rather than the actual properties of the object that should be tested.
For example. We have a simple "Dog" object and we want to get a list of all Dogs of a certain breed,ordered by the date of birth of the dog.



class Dog{
Breed breed;
Date dob;
String name;

}


When testing using this moniker style we might test like this (notice that i have introduced errors but the tests still pass since they are testing the moniker and not the properties that should be tested)...


void testGetAllAlsationsOrderedByAgeUsingMonikers(){

Dog fido = new Dog(name:'alsation born last year', dob: '01/01/2006', breed:Breed.ALSATION)
Dog woofer = new Dog(name:'alsation born 2 years ago', dob:'01/01/2008',breed:Breed.LABRADOR)
Dog mutley = new Dog(name:'alsation born this year', dob: '01/01/2008', breed:Breed.ALSATION)

List dogs = DogService.getDogByBreed(Breed.ALSATION)
assertTrue 'alsation born 2 years ago', dogs[0].name
assertTrue 'alsation born born last year', dogs[1].name
assertTrue 'alsation born this year', dogs[2].name

}


the problem with this is that we are actually not testing for the real properties that we should be testing for, and we are relying on the fact that we didn't introduce any errors in our test set up. which, incidentally,we did, just to prove the point!


A much better way to test this would be the following...



void testGetAllAlsationsOrderedByAgeCorrectly(){

Dog fido = new Dog(name:'not relevant', dob: '01/01/2006', breed:Breed.ALSATION)
Dog woofer = new Dog(name:'not relevant', dob: '01/01/2008', breed:Breed.LABRADOR)
Dog mutley = new Dog(name:'not relevant', dob: '01/01/2008', breed:Breed.ALSATION)

List dogs = DogService.getDogByBreed(Breed.ALSATION)
assertEquals(2, dogs.size())
assertTrue( isOrderedByDateOfBirth( dogs) )
assertTrue ( dogs.every{ Dog dog -> dog.breed=Breed.ALSATION} )

}

private boolean isOrderedByDateOfBirth( List dogs) {

def dobComparator = [compare:{Dog dog1, Dog dog2 ->
 dog1.dob == dob2.dob ? 0 :dog1.dob > dog2.dob? 1 : -1 }] as Comparator
return dogs.sort(dobComparator) == dogs


}



Must-have Firefox Plugins for Web Development
2008-06-27T10:37:00.002+01:00
Firefox Plugins for Web Development
When developing new code (or updating existing code), I use a very small number of extensions for Firefox.

Joe Hewitt's Firebug plugin
 - You can debug and inline Javascript, inspect and edit the DOM elements on the page and update CSS rules on-the-fly. This is probably the number one tool for me.

Visit his site and from there install Firebug

Chris Pederick's Web Developer toolbar
 - Adds the ability to disable Javascript, CSS and view all kinds of DOM structure information. Built-in validation tools allow for markup validation against the W3C validators.

Download the plugin for Firefox

Alex Sirota's ColorZilla plugin (http://www.iosart.com/firefox/colorzilla/)
 - Allows you to inspect the colour of elements on the page. This is useful for choosing colours for CSS rules (and when an image editor is not readily available to do the job).

Download the ColorZilla 1.9 for Firefox
Of course there are others that I use as well (Selenium IDE, YSlow and TamperData are definitely worth a mention), but these represent the core.
Whilst this addresses Firefox, there are plenty of tools available for other browsers - look for a post covering some of those for Internet Explorer and Safari.



Practical Front End Coding Resources
2008-06-25T13:16:00.004+01:00
Here are some useful blogs, articles, books and forums that I think are representative of the sites that I use for front-end problems, solutions and inspiration. I hope you get some mileage from them!

CSS/HTML
Troubleshooting - Position is Everything - Excellent resource site for IE CSS related bugs
http://www.positioniseverything.net/explorer.html

Troubleshooting - TekTips forum site for HTML/CSS Forum:
http://www.tek-tips.com/threadminder.cfm?pid=215

Book - Bulletproof Web Design by Dan Cederholm
http://www.amazon.co.uk/Bulletproof-Web-Design-Flexibility-Protecting/dp/0321509021

Book - Web Standards related Book - Web Standards Solutions: The Markup and Style Handbook by Dan Cederholm
http://www.amazon.co.uk/Web-Standards-Solutions-Markup-Handbook/dp/1590593812

Inspiration - CSS Zen Garden - Showcase site that shows what CSS can achieve
http://www.csszengarden.com/

Resource - A List Apart - Articles on HTML, CSS and Web Development
http://www.alistapart.com/

Reference - W3Schools CSS resource (tutorial and reference material)
http://www.w3schools.com/CSS/

Reference - W3Schools HTML resource
http://www.w3schools.com/HTML/

Javascript/AJAX

Ajaxian - Blog about all things AJAX related
http://www.ajaxian.com/

Troubleshooting - TekTips forum site for Javascript Forum:
http://www.tek-tips.com/threadminder.cfm?pid=216

Reference - W3Schools Javascript resource
http://www.w3schools.com/JS/

SEO

Article - USA Today - SEO Article that offers good introduction to SEO
http://www.usatoday.com/tech/products/services/2008-06-22-google-search-engine-optimization_N.htm




The Quartz Plugin
2008-06-23T23:10:00.006+01:00
The docs for the quartz plugin are more than adequate for installation and usage, but they don't give you any help how to test. The first step is to make sure quartz is stopped on your test machines, otherwise jobs will fire at random during a test run and breaking things.

You do this by editing QuartzConfig.groovy and setting autoStartup=false for your test / dev environments.

Next I added a quartz action to our FixtureController. The job name is simply the class name.

class FixtureController {
   def quartzScheduler

   def quartz = {
        if (params.triggerJob) {
            quartzScheduler.start()
            quartzScheduler.triggerJob(params.triggerJob, 'GRAILS_JOBS')
        } else if (params.standby) {
            quartzScheduler.standby()            
        }
        render("<pre>${quartzScheduler.metaData}</pre>")
   }
}


You can now trigger your job by hitting http://server:port/fixture/quartz?triggerJob=MyJob

The quartz jobs are (obviously) asynchronous, so your test harness will need some way of checking when they've started, and when they are finished. In the past we used a status page with a meta-refresh, that rendered the word 'continue' when some condition had been met. Once you know the job has started, set quartz back into standby in order to minimise the risk of other jobs kicking off half way through your tests.


Stemming and the Grails Searchable Plugin
2008-06-23T22:31:00.015+01:00
The standard analyizer that compass defaults to doesn't support stemming. The closest you can get is to use wild card searches. If you want to go the whole hog, here's what you have to do...

1. Open /grails-app/conf/SearchableConfiguration.groovy and replace

Map compassSettings = [:]

with

Map compassSettings = ['compass.engine.analyzer.default.type': 'snowball',
                             'compass.engine.analyzer.default.name': 'English',
                             'compass.engine.analyzer.search.type': 'snowball',
                             'compass.engine.analyzer.search.name': 'English']

2. Use queryString(<luceneQuery>, <params>) in your search service rather than the 'term' or 'wildcard' commands. e.g.

queryString("name:${match}", [boost: 10])

You're supposed to be able to specify the analyzer to use as a parameter to queryString, but I haven't tried this yet, as for my app I only need to support English.

3. Don't even bother trying to get stemming working if you use term, the plugin simply ignores any analyzer settings and constructs the StandardAnalyser from a hard coded class name. I raised a feature request for this, but it was bounced because Maurice thinks we should be forced to write lucene queries by hand. Never mind that the point of whole point of compass was to isolate your from the underlying search engine technology, or that DSLs are a much nicer mechanism for query creation than string manipulation. Ho Hum.


Pessimistic Locking in Grails
2008-06-23T22:01:00.006+01:00
I blogged about this a few weeks ago, then decided I really ought to create a few test cases to back up what I was spouting and raise a JIRA. Finally done. Here it is.

GRAILS-3156

Good news is the grails guys have scheduled a fix for v1.0.4

For anyone with an aversion to clicking links the long and short of it is that you can't pessimistically lock anything with the grails API. Instead you have to fall back to using hibernate and even then the only reliable way seems to be to clear the session and perform either a session.get(DomainObj.class, id, LockMode.UPGRADE) or session.load(DomainObj.class, id, LockMode.UPGRADE).

If you're not keen on clearing the entire session, you could try evicting the object you're about to lock instead, but this can cause other problems. Failing that you could write some db dependent SQL like 'LOCK <table> IN ACCESS EXCLUSIVE MODE'. Just make sure you check the db docs and pick the least restrictive lock that will do the job.


Refactoring GANT
2008-06-20T17:51:00.002+01:00
Its quite easy to build complicated GANT scripts - and not much has been written about how to refactor them.

Its not so obvious that you can call other targets directly, and with a bit of care you can re-use some of the built in grails targets like testApp.

The first step is to include anohter script like this:

includeTargets << new File("${basedir}/scripts/OtherScript.groovy")

Note: this does have implications in that you can overwrite variables and targets by including (and it does this silently without warning).

You can then reference another target from your target like this:

target(runMigration: "Run the Migration Tests") {
      testMigration()
      testApp()
}

There are some caveats however - one being that the useful "testApp" target (provided by grails) has a built in System.exit which causes your entire script to fail. You can get around this by hooking the exit event and storing the error code instead of exiting like this:

target('exit':"override exit") { code ->
    // Save the exit code.
    testAppExitCode = code
}


While all this does help - there is still a slight "whiff" to gant programming,  however some of these tricks can help keep your gant scripts tidier.

Tim


String, GString type and equality
2008-06-18T13:07:00.004+01:00
This might be very schoolboy blog for some of you - I still hope that it might be useful to avoid some bugs.

Some weeks back my pair and I were fixing a bug that finally boiled down to the fact that the contains() method on a List returned false if the list contained the GString counterpart of the same literal.

Having played a bit I learned a couple of things about GStrings. Please find some bullet points and the code snippet bellow.

Type:
1. def var1 = "this is a String instead of a GString even if it is surrounded by double quotes - can has no dollar sign"
2. def var2 = "this is a GString because it contains a dollar sign - $var1"
3. String var3 = "this is a string because it was defined as a String"
4. String var4 = "$var3  - this is still a string because it was defined as a String"

Equality
1. a String and a GString can be ==
2. a String and a GString are NEVER equals()
3. as a consequence a list containing a literal will not return true if the type (GString/String) does not match


class WhereIsYourDummyTests extends GroovyTestCase {
def e = "y"
def dummy = "dumm$e"

void assertTypeIsStringForAFixGString() {
   assert e instanceof java.lang.String
   assert dummy instanceof groovy.lang.GString
}

void testEqualsNotEquals() {
    assert "dummy" == dummy
    assert dummy == "dummy"

    // WATCH OUT HERE
    assertFalse("dummy".equals(dummy))
    assertFalse(dummy.equals("dummy"))
}

void testContainsDoesntContain() {
   assert ['dummy'].contains("dummy")
   assert ["dummy"].contains('dummy')
   assert [dummy].contains(dummy)

   // THIS IS TRICKY:
   assert ![dummy].contains("dummy")
   assert ![dummy].contains('dummy')
   assert !["dummy"].contains(dummy)
   assert !['dummy'].contains(dummy)

   assert [dummy.toString()].contains("dummy")
   assert [dummy.toString()].contains('dummy')
}

void testContainsIfVariablesAreStringInsteadOfDef() {
   String e = "y"                  // String instead of def
   String dummy = "dumm$e"         // String instead of def

   assert ['dummy'].contains("dummy")
   assert ["dummy"].contains('dummy')
   assert [dummy].contains(dummy)

   assert [dummy].contains("dummy")
   assert [dummy].contains('dummy')
   assert ["dummy"].contains(dummy)
   assert ['dummy'].contains(dummy)

   assert [dummy.toString()].contains("dummy")
   assert [dummy.toString()].contains('dummy')
}
}




Common exception types and when to use them
2008-06-17T11:05:00.003+01:00
Maybe this is obvious, but it is worth having in the back of your mind when writing code: Common Java unchecked exception types.


Further usefulness with showsource.. for rendered templates
2008-06-13T12:26:00.002+01:00
If you wanted to get the source view on rendered templates


Groovy & Grails Tutorials
2008-06-10T13:10:00.001+01:00
Found this link on groovy and grails tutorials.  Might be useful to have a look every once in a while:
http://www.grailstutorials.com


High Maturity Agile Team
2008-06-01T20:11:00.003+01:00
Just if you might have missed it, click here



Debugging with Internet Explorer
2008-05-23T10:08:00.004+01:00
Debugging with IE isn't something that you can really do without installing some sort of 3rd-party software.

To debug JavaScript, I've always used Visual Interdev, and for CSS, the IE Developer Toolbar. The problem with Visual Interdev is that its cost puts people off.

There are some good free tools available, however. I haven't used these before, but after re-installing windows, I thought I'd give them a go:

DebugBar is a plugin for IE that gives you a DOM inspector, an HTTP inspector, a JavaScript inspector and console, validation tools, and a whole load more. It's pretty much a "Firebug for IE" (although sadly, without a JavaScript debugger) :

http://www.debugbar.com/

Web Development Helper is a plugin for IE that gives you some useful tools (DOM inspector, logging HTTP requests, script error call stacks) :

http://projects.nikhilk.net/WebDevHelper/Default.aspx

Visual Web Developer 2008 is an application that lets you design and build web pages. It also comes with a JavaScript debugger :

http://www.microsoft.com/express/vwd/

There's a good article here about how to debug JavaScript using it :

http://www.berniecode.com/blog/2007/03/08/how-to-debug-javascript-with-visual-web-developer-express/


I'll update this post after I've used them for a while, although my first impressions are that DebugBar is probably the way to go.