Viewing Liquibase's Generated Changelog At Runtime

I know grails 2.x has a migrations plugin, but if you still using / prefer the liquibase one then read on...

I use liquibase in all environments "north" of development to manage the database schema and reference data, but still use create-drop while developing locally. It can be a pain to construct changesets by hand, but it's also awkward to run grails generate-changelog (especially if development normally uses an in memory database).  Much better if you could see the generated changelog while the application is running. Here's how...

import liquibase.database.*
import liquibase.diff.*
import java.sql.Connection

class LiquibaseController {

    def dataSource
   
    def index() {
        Connection connection = dataSource.connection
       
        DatabaseFactory factory = DatabaseFactory.getInstance()
        Database database = factory.findCorrectDatabaseImplementation(connection)
        database.setDefaultSchemaName('schemaName')
       
        Diff diff = new Diff(database, 'schemaName')
        DiffResult diffResult = diff.compare()
        diffResult.printChangeLog(new PrintStream(response.outputStream), database)
               
        response.contentType = 'text/xml'
        response.outputStream.flush()
    } 
}

S.

Techie Books Suggestions

(Or "How I Learned To Stop Worrying About Gramatically Correct Blog Titles")

I'm starting to add to the dev side of my techie library which, as you can appreciate, is pretty biased toward sysadmin stuff at the moment (DNS, firewall, exim, Jabber and so on).

So far I've added the two GinAs, and recently jQuery in Action, Javascript: The Good Bits, Javascript: The Definitive Guide and The Java Developer's Guide to Eclipse.

So what else should I have on my wish list? Freely downloadable is good too - it doesn't have to be expensive dead-tree format - especially if it can be Kindleised.

Would love to hear your thoughts and suggestions.

Remote Controlled Geb Functional Tests

I've just started on a new project where Geb and Spock are our main functional testing tools. They're nice tools to work with but sometimes it's hard to write tests quickly when you can't see what's going on. I want to be able to step through tests like in Selenium IDE but without all the shortcomings of using fixture controllers.

One way to develop functional tests under Grails is to keep the app running while you launch tests from another jvm and use -baseUrl to target the remote jvm. You can do this from the command line or use the Functional Test Development plugin. Another way is to keep the app running and run the tests from within an IDE like IntelliJ; a bonus is that you can run individual tests/features which is great if you have a fairly large spec.

Whichever way you choose, you're going to need to setup data on the target jvm. On my last project we used fixture controllers that we would call from our functional tests but these grew into thousands of lines because re-use after a while became difficult. A better way to do it is to bundle the test data with the actual test which is what the Remote Control plugin allows us to do - the plugin documentation has a better explanation of what I mean.

With all that said, what I want to be able to do is:

• Run my app up
  
• Create my data in my setup block as though I was in an integration test.
  
• Right click on the test from intellij and hit run (or use debug and step through line by line)

Take a look at the plugin docs, it's pretty straight forward but we had to override getBaseUrl and create our own BuildSettingsHolder because remote control doesn't like running outside of a grails command line. So our parent Spec looked like:

public ParentSpec extends GebSpec {
  String getBaseUrl() {
       "http://localhost:8080/monkeytails/"
  }

   def setupSpec() {

       if (BuildSettingsHolder.getSettings() == null) {
           BuildSettings bs = new BuildSettings()
           bs.metaClass.getFunctionalTestBaseUrl = { getBaseUrl() }

           BuildSettingsHolder.setSettings(bs)
       }
   }

   def cleanupSpec() {
       resetDatabase()
   }

   void resetDatabase() {
       def remote = new RemoteControl()
       remote {
           User.list().each{ it.delete(flush:true) }
           // Other teardown stuff goes here
           // ...
           return true
       }
  }
}

and then your tests:

    def "Test something"() {

      setup:
          def remote = new RemoteControl()
          remote {
              User user = new User()
              user.save(flush: true)
              return true //have to return something serializable
          }
      .
      .
  }

It's handy to have firebug in the firefox that webdriver opens so use the profile manager to create a profile called test and then change your IntelliJ JUnit Run configuration VM Parameters to include '-Dgeb.driver=firefox -Dwebdriver.firefox.profile=test'

Now you when you right click and run your test, IntelliJ will first compile everything and hopefully run your test. I say hopefully because I have a bunch of inline plugins so I have to exclude them from the compile path and hope for the best. Setting breakpoints and choosing debug lets you step through your test and let you can use firebug within the browser.

There's a few things to be aware of. Groovy's property setters don't seem to work so you can't do stuff like user.username, instead you have to use the setter or if there isn't one you can use setProperty(). Spread operators don't work too well either, so you can't do User.list()*.delete().

Update: forgot to mention that you should run the app in test mode or Remote Control will not set up its listener unlesss you set remoteControl.enabled to true in your app config

Testing cron expressions in Grails Jobs

One big problem I have is I can never remember what the cron expression is supposed to be and once I've got it in the code if it is actually correct.

Very simple test below. I create a CronTrigger and pass in the expression from the job. I then check that it will fire by passing in a Calendar object. The only key here is you can't test something in the past hence why I am adding another day to the LocalDate object.

void testJobSchedule() {

def trigger = new CronTrigger("name", "group", JobName.cronExpression)

assertTrue "job should fire at 1:15", trigger.willFireOn(time(1, 15))

}

private Calendar time(int hour, int minute) {

new LocalDate().plusDays(1).toDateTime(new LocalTime(hour, minute, 0, 0)).toCalendar(Locale.UK)

}

Hopefully this helps someone out.

Unexpected search results

Bit of friday fun - just tried searching for Node GString, and Google helpfully suggested...

Did you mean: nude g string

Using Liquibase DropAll Automagically 2

A while back gus posted about how to get liquibase to drop all on application start up. He had to jump through some hoops because dependsOn wasn't working properly. This seems to be OK now so I think the process can be simplified a bit...

Assuming the changelog is specified in grails-app/conf/liquibase/master.xml

resources.groovy

if (config.liquibase.on) {
   liquibaseDropAll(LiquibaseDropAll) { bean ->
      dataSource = dataSource
      changeLog = "classpath:liquibase/master.xml"
      bean.initMethod = 'init'
   }

   liquibase(SpringLiquibase) { bean ->
      dataSource = dataSource
      changeLog = "classpath:liquibase/master.xml"
      bean.dependsOn = ['liquibaseDropAll']
   }
}

LiquibaseDropAll.groovy

import liquibase.spring.SpringLiquibase
import org.codehaus.groovy.grails.commons.ConfigurationHolder

class LiquibaseDropAll extends SpringLiquibase {

   void init() {
      if (ConfigurationHolder.config.liquibase.dropAll) {
         super.createLiquibase(dataSource.connection).dropAll()
      }
   }

}




Disclaimer: I've only just started using it so it may still thow up some gotchas.

Reducing backend load

I've recently been looking at alternatives to F5 and I remember from this year's QCon a lot of people mentioning Zeus so I thought I'd give them a look.

So far they look promising; being able to run Zeus on your own generic hardware could be a cost saving in the long run. For example we had to buy some spare PSUs out of fear we wouldn't be able to buy them in the future.

One feature I really like the sound of is the "webcache!refresh_time". It smooths out the load to your backend by only sending one request to your appserver while serving the rest from cache. For sites that must have low cache times this makes a lot of sense. If you get 30 requests per second for an item, you only send one request to your backend as it expires from cache.

Turns out F5 also have a similar sounding feature called Fast Cache but for us it would be an additional module (read additional cost).

If anyone has experience with either I'd love to hear their thoughts.

Stubbing g.message

It's easy to mock grails taglibs using gmock and hamcrest. You can do something like...

    ...
    def g

    void setup() {
        g = mock()        
        mock(tagLib).getG().returns(g).stub()
    }

    void someTest() {
        g.message(hasEntry('code', 'foo.bar')).returns('FOO BAR')
        String result
        play {
            result = tagLib.someMethod([:]).toString()
        }
        assert result.contains('FOO BAR')
    }
}

But when you've got a lot of calls to g.message this can become noisy and usually ends up being repeated in multiple tests. One option is to be relax the argument matching and switch to using a stub.

        g.message(instanceOf(Map)).returns('FOO BAR').stub()

This is fine for the tests where you don't really care about the message, but too loose for the ones you do. I'm playing with the an alternative which gives you both stubbing and the option of explicitly asserting g.message was called with the correct arguments. Be warned though it may raise a WTF exception on first glance.

    ...
    def g

    void setup() {
        g = mock()        
        mock(tagLib).getG().returns(g).stub()
        stubMessages(g)
    }

    void someTest() {
        String result
        play {
            result = tagLib.someMethod([:]).toString()
        }
        assert result.contains('code:foo.bar')
    }
}

class MessageMatcher extends BaseMatcher {

    String code = ''

    static void stubMessages(def g) {
        MessageMatcher matcher = new MessageMatcher()
        g.message(matcher).returns(matcher).stub()
    }

    boolean matches(Object o) {
        code = ((Map) o).containsKey('code') ? o.code : ''
        return true;
    }

    void describeTo(Description description) { }

    String toString() {
        return "code:${code}"
    }
}

The tricky bit was getting the g.message stub to return a value derived from the matcher arguments. I couldn't just do

g.message(matcher).returns(matcher.code).stub()

because matcher.code won't have a value at this point. The solution is to override the matcher's toString() method to return the code and rely on groovy / grails invoking toString() when adding the matcher to the output. This certainly violates the rule of least surprises, but I think I'm OK with that if it reduces duplication and the noise level of my tests - at least until I find a better way.

Clean TagLib Tests

Despite my best efforts I've always found it hard to write clean taglib tests. Now thanks to Spock and GroovyShell things are getting easier...

    void setup() {
        mockDomain Invoice
    }

    def "Attachments icon has correct markup"() {
        given:
        Invoice invoice = new InvoiceBuilder().buildAndSave()

        when:
        renderAttachmentsIcon([target: invoice])

        then:
        valueOf('img.@id') == "toggle-attachments-${invoice.id}"
        valueOf('script') == "\$('#toggle-attachments-${invoice.id}').bind('click', Books.attachments.toggle);"
    }

Because I want to use GPath make assertions about the resulting HTML I've overriden TagLibSpec's methodMissing closure as follows...

    def methodMissing(String name, args) {
        String html = super.methodMissing(name, args)
        createDocument(html)
    }
    
    void createDocument(String html) {
        String xml = "<results>${html}</results>"
        document = new XmlSlurper().parseText(xml)
    }    

And added helper methods for evaluating GPath expressions...

    String valueOf(String gPath) {
        evaluate(gPath).text()
    }

    GPathResult evaluate(String gPath) {
        new GroovyShell(new Binding(document: document)).evaluate("document.${gPath}")
    }

There's a little bit more to this story unfortunately...

Firstly my taglibs use the MarkupBuilder and when run from my Spock test this only outputs the opening tag of the first element I genererate. I haven't had a chance to look into this yet, but a workaround is to add "out << '' to the end of the taglib method

Secondly Spock interactions aren't yet as powerful as gmock, so I usually end up adding code to (g)mock grails taglibs.

Thirdly another one of my tests outputs &nbsp; in the HTML which causes the XML parsing to barf. The solution is to map the &nbsp entity to a known character (in this case underscore).

Finally there was a bug in Grails 1.3.3 / Spock 0.5 which breaks mockDomain. This is reportedly fixed in 1.3.4 and the latest Spock code, but I haven't tried upgrading yet.

Here's how things really look...

@WithGMock
class MetaAttachmentsTagLibSpec extends TagLibSpec {

    def g
    def document

    void setup() {
        g = mock()        
        mock(tagLib).getG().returns(g).stub()
        mockDomain Invoice

        PluginManagerHolder.pluginManager = [hasGrailsPlugin: { String name -> true }] as GrailsPluginManager // Workaround for JIRA GRAILS-6482       
    }

    def cleanup() {
        PluginManagerHolder.pluginManager = null // Workaround for JIRA GRAILS-6482
    }

    def "Attachments icon has correct markup"() {
        given:
        Invoice invoice = new InvoiceBuilder().buildAndSave()
        g.resource(instanceOf(Map)).returns '/foo.jpg'

        when:
        renderAttachmentsIcon([target: invoice])

        then:
        valueOf('img.@id') == "toggle-attachments-${invoice.id}"
        valueOf('img.@src') == "/foo.jpg"
        valueOf('script') == "\$('#toggle-attachments-${invoice.id}').bind('click', Books.attachments.toggle);"
    }
    
    def methodMissing(String name, args) {
        String html
        play {
            html = super.methodMissing(name, args)
        }
        createDocument(html)
    }

    def createDocument(String html) {
        String xml = """<!DOCTYPE html [<!ENTITY nbsp "_">]>\n<results>${html}</results>"""
        document = new XmlSlurper().parseText(xml)
    }
}

And the method under test...

    def renderAttachmentsIcon = { Map attrs, def body ->
        String targetId = attrs.target.id
        String iconId = "toggle-attachments-${targetId}"
        String imgSrc = g.resource(dir:'/images/skin', file:'paperclip.png')

        MarkupBuilder builder = new MarkupBuilder(out)
        builder.img(id: iconId, src: imgSrc)
        builder.script(type:'text/javascript') {
            mkp.yield "\$('#${iconId}').bind('click', Books.attachments.toggle);"
        }
        out << '' // flush for unit tests        
    }    

Internet Explorer and the 1 item remaining bug

We've started using Selenium 2 / WebDriver to soak test our application, and were caught once again by a bug in ie7 and ie8 that randomly causes the browser to hang, waiting for some resource to finish loading, even though the page is visible and works fine.

I hit the problem a year or so ago and solved it by hacking something in prototype.js, but can't remember exactly what. Thankfully there's now a more recent version of prototype (we're stuck on grails 1.1.1 which is bundled with prototype 1.6.0), so I upgraded to prototype 1.6.1 and problem solved :)

Grails and unit#junit;4.8.1: configuration not found in junit#junit;4.8.1:

After upgrading to grails 1.3.x I started getting the following error...

  ::::::::::::::::::::::::::::::::::::::::::::::

  ::          UNRESOLVED DEPENDENCIES         ::

  ::::::::::::::::::::::::::::::::::::::::::::::

  :: junit#junit;4.8.1: configuration not found in junit#junit;4.8.1: 'master'. It was required from org.grails.internal#Books;0.1 test

  ::::::::::::::::::::::::::::::::::::::::::::::

A found a few posts suggesting I needed to clear my ivy cache, but that did sfa. Turned out to be something in the jawr 3.3.2 plugin. It went away when I excluded junit from compile scope in the plugins dependencies.groovy

    dependencies {
        // specify dependencies here under either 'build', 'compile', 'runtime', 'test' or 'provided' scopes eg.
     compile ('net.jawr:jawr:3.3.2') {
      excludes "mail", "activation", "ejb","jms","jmxri","jmxremote", "junit"
 }

IntelliJ v9.0.3 adds support for Grails v1.3.x

JetBrains released IntelliJ IDEA v9.0.3 two days ago, which includes support for Grails v1.3.x, along with nicer SVN merging.

See more details here:

http://www.jetbrains.com/idea/whatsnew/index.html

AppStatus Plugin 0.1 Released

The AppStatus plugin enables you to easily display useful info about your application.

Install the plugin and navigate to http://yourapp/appStatus to see application.properties, server time and locale.

If you want to see more details you can specify the "providers" in resources.groovy

import uk.co.acuminous.app.status.provider.*
import uk.co.acuminous.app.status.AppStatusConfig

beans = {
    appStatusConfig(AppStatusConfig) {
        providers = [

            // !!!DANGER WILL ROBINSON!!!
            // The ApplicationConfigProvider will dump the entire
            // contents of Config.groovy including datasource.password
            // The SystemPropertiesProvider could also make available
            // potentially sensitive system variables

            properties: new ApplicationPropertiesProvider(),
            config: new ApplicationConfigProvider(),
            system: new SystemPropertiesProvider(),
            locale: new LocaleProvider()
        ]
    }
}

If you want to add your own providers just write something that implements the Provider interface and add it to the providers map.

Finally if you're already using jQuery and jQuery UI you pick an nicer view

beans = {
    appStatusConfig(AppStatusConfig) {
        providers = [
            // blah
        ]
        view = 'tabs'
    }
}

Pure Genius and mod_jk

I'm currently developing an application for MegaBank which makes heavy use of AJAX. The latest cut was throwing javascript errors in UAT but not in development. A bit of investigation with firebug showed that when an AJAX request returned 404 (which it does if the user supplies a non-existent cost code for example), the next AJAX request would respond with the SSO login page instead of the expected json response. Further investigation showed that the second AJAX response had been served by a different node than the one it was sent to, resulting in the SSO challenge.

Request had a cookie:
jsessionid=ASAD123DFDFS83242SDFASD9234234.node1

Response has a set-cookie header:
jsessionid=SDFSDF234DFSLFSD324234880SDFSDL.node2

i.e. our sticky session was becoming unstuck.

We reported the incident to our support team, who's spanked it straight back over the net with the wonderfully helpful "It's an application config issue". Great. Thanks. Our app doesn't do anything clever with cookies. There is no logout button. We never invalidate the session. We'd already checked that inbound request had the right jsession id, and also that the associated response had a set-cookie header from the wrong node. Furthermore the app server logs show that the second request didn't even hit the right node. Back to web support with an offer to demonstrate the problem and walk through why we believe it's an issue with the load balancer. Even if it turned out to be an issue with the app we would need their help to diagnose it.

Offer accepted. And "Pure Genius" arrives (That's not his real name, but it is what was written on his cuff links). The first words out of his mouth were, "We support 100s of applications and the load balancer works fine. It must be your application". It doesn't mater that this is a bank, where 99% of the applications he supports are legacy and have probably never heard of AJAX. So I walk him through the process, show him the HTTP headers, show him the logs, explain that this only seems to happen after a 404. He goes away and helpfully reports...

"It's an application configuration issue."

Grrrrrrrr. Escalate. Now engineering areinvolved. Helpfully "Pure Genius" has already told them it's a problem with our app, so they've they've prioritised it to the bottom of their queue. Grrrrrrrr. Another round of the escalation game follows and we get someone from engineering who knows what they're doing and isn't wearing blinkers. Guess what? There's a bug in mod_jk which causes it to fail a session over if it gets a non 200 error code with no content, so we added the words "Pure Genius" to the response and we have sticky sessions again. I guess he helped after all.

Javassist Enhancement failed

Just spent the last few hours debugging this.

def adoptChild = {
   Parent p = Parent.get(params.id)
   p.child = Child.findByName(params.name)
   assert p.save(flush:true)
   redirect(action: 'showParentWithChild', id: p.id)
}

def showParentWithChild() {
   Parent p = Parent.get(params.id) // Javassist Enhancement failed stack trace here
   render("OK")
}

Turned out that the child class had a private constructor (no idea why). I ditched the constructor and everything was back to normal.

Issues with Eclipse 3.5.1 and groovy plug-in

For all you Linuxy peeps.

After installing Eclipse 3.5.1 (SDK version) I installed all the plugins I usually use but when I installed the groovy v2 plugin all my existing plugins disappeared along with the groovy one. Eclipse insisted that they were installed but they weren't there.

It seems that this combination doesn't like "shared installs". My eclipse installation was in /usr/local/eclipse and I was installing the groovy plugin as my regular user. Fail.

There are two solutions.

1 - install a local copy of eclipse.

Simple but it means an eclipse install per user on your machine.

2 - install the groovy plugin as root.

Means that you have to run it as root every time you want to upgrade the plugin.

Choose whichever evil is lesser for you. (I chose #2).

BTW you may also need this magic shell script line

export GDK_NATIVE_WINDOWS=true

if you find that some of the buttons in eclipse don't like being clicked on. Apparently this is due to be fixed in eclipse 3.5.2.

Hope this saves someone a few frustrated hours.

Ham and Chips

Rob Fletcher's blog post on gMock inspired me to try out hamcrest, but I was surprised to find out there's no simple way to match collections in mocked method arguments - the best I could come up with was this...

mock(tagLib).fieldLabel(instanceOf(Map), instanceOf(Closure)).returns("<label></label>")

which doesn't tell me whether I'm calling fieldLabel with the expected attributes. After a bit of searching I found the MapContentMatcher in the Spring Integration library, but didn't want include the whole jar just for that. It also had dependencies other libraries and was heavily typed with generics. Here's the cut down groovy version...

import org.hamcrest.*
import static org.hamcrest.core.AllOf.*
import org.hamcrest.Factory as HamcrestFactory

class MapContentMatcher extends TypeSafeDiagnosingMatcher<Map> {

    Object key
    Object value
    
    MapContentMatcher(Object key, Object value) {
     this.key = key
        this.value = value
    }
    
    public boolean matchesSafely(Map actual, Description mismatchDescription) {
     return actual.containsKey(key) && value == actual[key]
    }
    
    public void describeTo(Description description) {
     description.appendText("$key:$value")
    }
    
    @HamcrestFactory
    public static Matcher hasAllEntries(Map expected) {
     return allOf(expected.collect { k, v ->
      new MapContentMatcher(k, v)
        })
    }    
}

Which can be used thus...

mock(tagLib).fieldLabel(hasAllEntries([id: expectedField, label:expectedHeading]), instanceOf(Closure)).returns("<label></label>")

Sitemesh layout applied to AJAX response (gotcha)

Grails supports multiple ways to specify a sitemesh layout for your responses. One way I wasn't familiar with is by convention, i.e. if you have a controller 'BookController' and a layout in /views/layouts called book.gsp all the actions in BookController will be rendered with the book layout by default - even if you do render("some text") or render(template: "/sometemplate").

So if you're wondering why all your AJAX responses are getting wrapped up in a sitemesh template this could be the reason. You can stop this by changing the content type to text/plain or by renaming the layout and specifying it explicitly in your gsps using <meta name="layout" content="layoutname"/>.

Testing grails scripts

Not sure if everyone caught the new functionality that Peter has put into 1.2M3. It's the ability to test grails scripts. Heres the link to his blog post:
http://www.cacoethes.co.uk/blog/groovyandgrails/testing-your-grails-scripts

Build Server Woes (mptscsi task abort)

I'm posting this here for two reasons:

• in the hope that someone out there will find this useful


• personal therapy

Scenario:

We've got a couple of build servers (x86_64 linux, openVZ) that have been having some disk I/O problems. These boxes (boxen) run various virtual machines related to our product builds - hudson masters & slaves, distribution servers, puppet master, test mail server, munin server... yada yada. You get the idea. They're kinda important.

The problem manifests itself by first reporting errors like the following:

Sep 16 14:28:51 hn3 mptscsih: ioc0: attempting task abort! (sc=ffff880422a348c0)
Sep 16 14:28:51 hn3 sd 0:1:2:0: [sda] CDB: cdb[0]=0x2a: 2a 00 12 b2 fc 9f 00 00 08 00
Sep 16 14:28:51 hn3 mptscsih: ioc0: Issue of TaskMgmt failed!

followed shortly by the volume in question getting offlined into readonly mode:

Sep 16 14:29:41 hn3 mptscsih: ioc0: host reset: SUCCESS (sc=ffff880422a348c0)
Sep 16 14:29:41 hn3 sd 0:1:2:0: Device offlined - not ready after error recovery

This ain't that helpful when you've got a whole load of hungry VMs wanting to write stuff to disk. A closer look at the disk controller yields the following:

>lspci
...
0b:00.0 SCSI storage controller: LSI Logic / Symbios Logic SAS1068E PCI-Express Fusion-MPT SAS (rev 08)
...

A quick search through our messages shows the following related information:

>dmesg | grep -i mpt
Fusion MPT base driver 3.04.07
Fusion MPT SPI Host driver 3.04.07
Fusion MPT FC Host driver 3.04.07
Fusion MPT SAS Host driver 3.04.07
mptsas 0000:0b:00.0: PCI INT A -> GSI 35 (level, low) -> IRQ 35
mptbase: ioc0: Initiating bringup
mptbase: ioc0: PCI-MSI enabled
mptsas 0000:0b:00.0: setting latency timer to 64
Fusion MPT misc device (ioctl) driver 3.04.07
mptctl: Registered with Fusion MPT base driver
mptctl: /dev/mptctl @ (major,minor=10,220)

A quick google turns up quite a few different issues with these controllers but no clear resolution (no surprise).

To cut a (very) long story short we appear to have a solution by using the drivers supplied by LSI rather than those shipped with the latest linux kernel. Patching the kernel (by replacing the drivers/message/fusion folder with the equivalent found in LSI's MPTLINUX_RHEL5_SLES10_PH16-4.18.00.00-1.zip distribution) with version 4.18 of the MPT drivers has yielded an (apparently) stable system, tested under reasonably high load (load average ~20).

Incidentally, for those of you who like acronyms, MPT stands for 'Message passing technology'.

My work here is done.