Gus and I will be presenting on "Effective Testing on Grails" at the London Groovy and Grails User Group next week Wed 1st of October, @18.30. Please feel free to attend if this seems relevant to you. A synopsis of the talk and registration for the event are all available at the following link.
http://skillsmatter.com/event/java-jee/ggug
Thursday, 25 September 2008
The UI decisions behind a typical web form
I stumbled across a very useful article recently and wanted to share it around. The article discusses the best placement of form submit/cancel buttons for a typical web form and backs this up with Heat Maps showing where the user spent time looking at elements on the page etc.
Check out the Primary & Secondary Actions in Web Forms blog post yourself.
There are a lot of very useful posts on that blog - all of them focused on useability and the "dark art" of UI decision making.
Inspecting Response Headers in the browser
I needed to inspect the headers being sent to various browsers today. Specifically we wanted to find out how Yahoo was able to deliver a "fresh" CAPTCHA image when you used the browser back button to return to the page with the CAPTCHA image on it.
We observed that this happened even with Javascript disabled, and so came to the conclusion that it must be something in the response headers.
In Firefox (v1.5+) there is a really useful extension that I have mentioned before (the Web Developer Toolbar extension) which can show the headers using Tools -> Web Developer -> Information -> View Response Headers.
When using Internet Explorer a solution is to use the IE HTTP Headers plugin (for IE v5.0+) which is available at http://www.blunck.se
For those interested, the solution to the problem was to add the following headers to the response:
static void doNotCacheResponse(HttpServletResponse response) {
response.setHeader('Pragma', 'no-cache')
response.setHeader('Cache-Control', 'no-cache,no-store')
response.setHeader('Expires', '-1')
}We've tested that this works to force a re-request using Firefox 2, Firefox 3, IE6, IE7, Chrome, Safari 3 and Opera 9 - which is nice!
Tuesday, 23 September 2008
Careful with what you Query.
The F5 BigIP load balancers have a feature called iRules. They allow you to manipulate and manage traffic in the Load Balancer.
They are really useful for checking or modifying cookies [HTTP::cookie], the host domain [HTTP::host], path [HTTP::path] and query parameters [HTTP::query] of URLs and requests.
A sligth word of warning though.... any checks made on the QUERY of a HTTP request actually search the URI of the referer as well. As an example, the following iRule would send a HTTP 200 response if you were either requesting a URL with a query parameter of "ThereIsAQuery" OR if you're refer page's URL included a query parameter of "ThereIsAQuery".
Other attributes such as [HTTP::host], [HTTP::path], and [HTTP::uri] are only valid for the requested URL. They are not checked in the referer header.
They are really useful for checking or modifying cookies [HTTP::cookie], the host domain [HTTP::host], path [HTTP::path] and query parameters [HTTP::query] of URLs and requests.
A sligth word of warning though.... any checks made on the QUERY of a HTTP request actually search the URI of the referer as well. As an example, the following iRule would send a HTTP 200 response if you were either requesting a URL with a query parameter of "ThereIsAQuery" OR if you're refer page's URL included a query parameter of "ThereIsAQuery".
when HTTP_REQUEST {
if {
([HTTP::QUERY] eq "ThereIsAQuery")
}
{
HTTP::respond 200 content "<html><body>There is a Query</body></html>"
}
}Other attributes such as [HTTP::host], [HTTP::path], and [HTTP::uri] are only valid for the requested URL. They are not checked in the referer header.
Monday, 15 September 2008
Command Objects Revisited
I posted yesterday about command objects but a conversation I had this morning made me realise I'd neglected to mention one of the interesting and less obvious ways to use them.
Imagine a scenario where you have a form that is basically for saving domain object instances but has a few other fields, not directly properties of the domain object in question but things like choices of how to set property values (e.g. a radio button to choose whether or not to update the timestamp on the object), workflow type instructions to the controller (e.g. preview this object after saving it), etc. This is a very common scenario.
You can handle this neatly with command objects by nesting an instance of the domain class inside the command object class, e.g.
Imagine a scenario where you have a form that is basically for saving domain object instances but has a few other fields, not directly properties of the domain object in question but things like choices of how to set property values (e.g. a radio button to choose whether or not to update the timestamp on the object), workflow type instructions to the controller (e.g. preview this object after saving it), etc. This is a very common scenario.
You can handle this neatly with command objects by nesting an instance of the domain class inside the command object class, e.g.
class MyCommand {
boolean updateTimestamp
boolean previewAfterSave
MyDomain domainObj
}Then name your form fields accordingly... <g:checkBox name="updateTimestamp" value="${command.updateTimestamp}" />
<g:checkbox name="previewAfterSave" value="${command.previewAfterSave}" />
<input type="hidden" name="domainObj.id" value="${command.domainObj.id}" />
<input type="text" name="domainObj.name" value="${command.domainObj.name}" />
<g:select name="domainObj.otherDomainObj" from="${OtherDomainClass.list()}" value="${command.domainObj.otherDomainObj.id}" optionKey="id" /> and so on.
Sunday, 14 September 2008
Command Objects
We have a bunch of horrible code in our codebase to do with binding form fields to objects. To be fair it dates from a time when we were a lot newer to Grails and Grails and its documentation was a lot less mature. The other day I had to add a date picker to one of our forms and write the binding code for it. The g:datePicker tag creates some select boxes and a hidden field. Unlike other implementations I've used in the past it doesn't use Javascript to compose the select box values into the hidden field on form submit. Instead the hidden field has the value "struct" which I guess is a hint to GrailsDataBinder. To cut a long story short the code I ended up writing is pretty nasty. Especially considering that GrailsDataBinder is perfectly capable of handling this kind of binding without any code being written.
Instead of all this tedious messing about with params Grails controllers can, as most of you probably know, bind form submissions to domain objects or command objects. Binding to domain objects works by doing domainObj.properties = params or using the bindData method. So much most of us are used to (even if you wouldn't think so from looking at some of our code).
To have your controller action use a command object you simply specify it as an argument to the action closure. e.g.
A form that contained...
Unit testing is easy because you can simply construct the command object and pass it in to the controller action. You don't have to worry about setting a bunch of esoteric key/value pairs on params to get your controller's hairy binding logic to work. In an integration test, if you really want to, you can set params and test that the command binding works as you expect.
The only non trivial thing in unit tests (in integration tests this is not a problem) is testing command error handling. In unit tests commands won't have an errors property or hasErrors method like they do in a running app or integration test. I've come up with the following which I've raised as an enhancement request for the experimental testing plugin:
Command objects can also be used to handle multipart upload forms...
The weak area in binding at the moment seems to be one-to-many domain class relationships (which is why I didn't jump into refactoring the code I mentioned at the start of this post). That does require you to write hairy binding code and is probably worth a post in and of itself.
Instead of all this tedious messing about with params Grails controllers can, as most of you probably know, bind form submissions to domain objects or command objects. Binding to domain objects works by doing domainObj.properties = params or using the bindData method. So much most of us are used to (even if you wouldn't think so from looking at some of our code).
To have your controller action use a command object you simply specify it as an argument to the action closure. e.g.
def save = { SaveCommand command ->
if (command.hasErrors()) {
render(view: 'index', model: [command: command])
}
// do some stuff
} The binding is done for you. As you may guess from the command.hasErrors() above, command objects can have constraints exactly as domain objects do. They can even have services and other Spring application context artefacts wired in automatically just like controllers and services can.A form that contained...
<input type="text" name="name" value="${command?.name}" />
<g:datePicker name="birthday" value="${command?.birthday}" />
<g:select name="sandwich.id" from="${Sandwich.list()}" value="${command?.sandwich?.id" optionKey="id" />Would bind to a command like this... class MyCommand {
String name // simple property
Date birthday // bound from multiple fields submitted by date picker
Sandwich sandwich // bound domain object
}Unit testing is easy because you can simply construct the command object and pass it in to the controller action. You don't have to worry about setting a bunch of esoteric key/value pairs on params to get your controller's hairy binding logic to work. In an integration test, if you really want to, you can set params and test that the command binding works as you expect.
The only non trivial thing in unit tests (in integration tests this is not a problem) is testing command error handling. In unit tests commands won't have an errors property or hasErrors method like they do in a running app or integration test. I've come up with the following which I've raised as an enhancement request for the experimental testing plugin:
def createCommand(Class clazz) {
def command = clazz.newInstance()
def commandErrors = new BeanPropertyBindingResult(command, GrailsClassUtils.getLogicalPropertyName(clazz.name, ''))
command.metaClass.getErrors = {-> commandErrors }
command.metaClass.hasErrors = {-> commandErrors.hasErrors() }
}
void testCommandWithBindingErrors() {
def command = createCommand(MyCommand)
command.errors.reject('birthday', 'nullable')
controller.action(command)
// assert the form is re-rendered with the command in the model, etc.
}Command objects can also be used to handle multipart upload forms...
<input type="file" name="avatar" />Maps to...
class MyCommand {
byte[] avatar
}If you need access to the uploaded file's metadata (content type, original filename, etc.) instead of declaring the property as type byte[] use MultipartFile. Either way the binding works seamlessly and allows you to apply constraints to the uploaded file.The weak area in binding at the moment seems to be one-to-many domain class relationships (which is why I didn't jump into refactoring the code I mentioned at the start of this post). That does require you to write hairy binding code and is probably worth a post in and of itself.
Friday, 12 September 2008
None of the rest of us can fathom CSS either.
Dave Minter has written a blog entry Float like a wasp that goes on about the inadequacies and problems faced with implementing CSS and the frustrations of HTML. I think that he's got a point when he says:
You are not alone. None of the rest of us can fathom CSS either.
He has a hit-list of things he'd like to see made more simple... and they include the usual bug-bears that we have all experienced at some stage (some more than others, obviously).
There are solutions to all the things he would like to see... but the key thing is that these solutions are invariably not intuitive and require arcane knowledge (and the patience of a saint) to ensure conformity.
I particularly liked this quote:
One of my tips for fixing all that irks with CSS would be to buy two or three of those books describing machiavellian ways to conspire against the deficiencies of CSS (anything with "hacks" or "tips" in the title) and then beat up the standard until the easiest approach to implementing everything in the book could be covered in a pamphlet.
I, too, would like to see a more structure simplification (rather that the current direction of CSS which seems to be adding to the mess).
Sunday, 7 September 2008
Pessimistic Locking With Grails
By default grails uses optimisic locking with versioning. In a nutshell this means that each of your domain objects is blessed with a version field and hibernate throws an exception if you ever try to update or lock an object with an old version number. For scenarios where you are happy to handle the exception, or just report the error to the user via a UI this is fine, however when this behaviour isn't ok you need to switch to pessimistic locking. This is where the fun begins.
Unfortunately the Grails pessimistic locking support is incomplete. There are scenarios when it will work, but scenarios when it wont. As of 1.03 the docs don't make this clear. For example, let's say you have a domain object called Account...
Obviously updates to the account balance need to be atomic. The Grails docs suggest the following...
The resulting SQL will look something like
The above will work fine PROVIDING the account entity is not already in the hibernate session. If the account entity was previously retrieved the resulting SQL only re-selects the id.
The db row is still locked, but because the account entity has not been refreshed your application could be working with stale data. The annoyance with the Account.lock(id) approach is that you always need to know the entity's id without obtaining the entity. So your code can never do
My initial workaround to the above is to duck into the hibernate API and do the following
Which appears ensures you have fresh data and a db lock. However this is ONLY true if your entity does not have any eager relationships (one-to-ones are eager by default). Incredibly hibernate retrieves the eager relationships before calling select for update on the main entity, meaning that the relationship data could be stale. I've tried fixing this with cascade: 'lock' but it did SFA. It now appears the best (and I use that term loosely) way to ensure fresh, locked data is to do the following
An alternative is to call "discard" on the object before you lock it
An interesting aside, which has nothing to do with grails but everything to do with concurrency is how you prepare to update multiple objects, e.g.
The above code might look OK at first glance, but what if two threads were to update an intersecting set of accounts?
Thread 1: accountA, accountB
Thread 2: accountB, accountA
If you were unlucky Thread 1 may obtain a lock on accountA at the same time that Thread 2 obtained a lock on accountB resulting in deadlock. The solution is to sort both sets of accounts using the same algorithm anywhere you attempt to locks multiple entites, e.g.
Another concurrency scenario is when you want to modify the many side of a one-to-many relationship, e.g.
In this case because a subscription is never shared between two people it may be ok to just lock the person object, e.g.
The reason I say 'may', is that this approach is dependent on all your other code taking the same approach when updating subscriptions and that it's not a performance bottleneck to lock the person. Choosing to lock the person is certainly an easy way to avoid creating duplicate subscriptions without the risk of a unique constraint violations.
Unfortunately the Grails pessimistic locking support is incomplete. There are scenarios when it will work, but scenarios when it wont. As of 1.03 the docs don't make this clear. For example, let's say you have a domain object called Account...
class Account {
String name
long balance
static mapping {
version false // Required to avoid stale object exceptions when hibernate attempts a lock
}
}Obviously updates to the account balance need to be atomic. The Grails docs suggest the following...
Account account = Account.lock(id)
account.balance += amount
account.save()
The resulting SQL will look something like
select id, name, balance from account where id = 123 for update;
update account set balance = 101 where id = 123;
The above will work fine PROVIDING the account entity is not already in the hibernate session. If the account entity was previously retrieved the resulting SQL only re-selects the id.
select id from acount where id = 123 for update;
update account set balance = 101 where id = 123;
The db row is still locked, but because the account entity has not been refreshed your application could be working with stale data. The annoyance with the Account.lock(id) approach is that you always need to know the entity's id without obtaining the entity. So your code can never do
Account account = Account.findByName(name) // Maybe in a controller or other service
...
account = Account.lock(account.id)
account.balance += amount
account.save()
My initial workaround to the above is to duck into the hibernate API and do the following
sessionFactory.currentSession.refresh(account, LockMode.UPGRADE)
account.balance += amount
account.save()
Which appears ensures you have fresh data and a db lock. However this is ONLY true if your entity does not have any eager relationships (one-to-ones are eager by default). Incredibly hibernate retrieves the eager relationships before calling select for update on the main entity, meaning that the relationship data could be stale. I've tried fixing this with cascade: 'lock' but it did SFA. It now appears the best (and I use that term loosely) way to ensure fresh, locked data is to do the following
account.lock() // versioning must be disabled
account.refresh()
account.balance += amount
An alternative is to call "discard" on the object before you lock it
account.discard()
account = Account.lock(account.id)
account.balance += amount
account.save()
An interesting aside, which has nothing to do with grails but everything to do with concurrency is how you prepare to update multiple objects, e.g.
accountsToBeUpdated.each { Account account ->
account.lock()
account.refresh()
account.balance += amount
account.save()
}The above code might look OK at first glance, but what if two threads were to update an intersecting set of accounts?
Thread 1: accountA, accountB
Thread 2: accountB, accountA
If you were unlucky Thread 1 may obtain a lock on accountA at the same time that Thread 2 obtained a lock on accountB resulting in deadlock. The solution is to sort both sets of accounts using the same algorithm anywhere you attempt to locks multiple entites, e.g.
def sortedAccounts = accountsToBeUpdated.sort { a, b -> a.name <=> b.name }
sortedAccounts.each { Account account ->
account.lock()
account.refresh()
account.balance += amount
account.save()
}Another concurrency scenario is when you want to modify the many side of a one-to-many relationship, e.g.
class Person {
Set subscriptions
} In this case because a subscription is never shared between two people it may be ok to just lock the person object, e.g.
person.lock()
person.refresh()
person.addToSubscriptions(subscription)
person.save()
The reason I say 'may', is that this approach is dependent on all your other code taking the same approach when updating subscriptions and that it's not a performance bottleneck to lock the person. Choosing to lock the person is certainly an easy way to avoid creating duplicate subscriptions without the risk of a unique constraint violations.
Subscribe to:
Posts (Atom)