Selenium IDE

Just a quick post to say that if any of you out there spend your day resizing and moving Selenium IDE after reopening it (it doesn't remember the previous window state), you can write a little script to do it for you using the handy 'wmctrl' :)

(on linux)
wmctrl -r 'Mozilla Firefox' -e 0,0,25,1330,1125
wmctrl -r "Mozilla Firefox" -b remove,maximized_vert,maximized_horz
wmctrl -a 'Mozilla Firefox'
wmctrl -r 'Selenium IDE' -e 0,1350,25,575,1125
wmctrl -a 'Selenium IDE'


The snippet above moves, unmaximises and raises firefox before moving and activiating the Selenium IDE.

Selenium CSS locators

We've got a project with quite a large set of tests, and we have both Firefox and IE7 continuous builds using selenium. The annoying thing is that something that takes 5 minutes in Firefox can take 30+ minutes with IE (I kid you not!).

It turns out this is a common problem and is due to the lack for native xpath support in IE. So for all the selenium tests that use xpath=//..... in IE the xpath is actually being evaluated using javascript ... ouch. An alternative is to use css locators, you can't do this everywhere but as well as improving the IE performance it can also provide some rather tidy rules. For example

xpath=//div[contains(@class,'balance')]
or
css=.balance

xpath=//div[@id,'topLeft')//span[contains(@class,'name')]  
or 
css=#topLeft .name

So even if you dont' care about IE using css selectors might be a lot nicer. (It might also improve your css skills!)

Timing Selenium

We finally figured a pain free way of profiling our selenium test suite. It could use some refinement, but we were only looking for a quick win.

Add the following test to the start of your suite

open|/|
storeEval|{lastTime = new Date().getTime(); 'x'}|x
storeEval|{LOG.info = function(message) { var now = new Date().getTime(); var duration = now - lastTime; this.warn(message + duration); lastTime = now}; 'x';}|x

When your run your suite the log file file contain lines like...


warn: Starting test /selenium-server/tests/suite-tests/conversation/SearchConversationTest.html119
warn: Executing: |open | /fixture/tearDownTestData | |47
warn: Executing: |assertTitle | Fixture | |239
warn: Executing: |invokeUrl | /fixture/reindexSearch | |28
warn: Executing: |invokeUrl | /fixture/setUpPersons | |121
warn: Executing: |invokeUrl | /fixture/setUpConversationsForSearch | |212
warn: Executing: |createCookie | splash=false | path=/ |16638


The duration for each step has been appended to the NEXT log line, i.e. setUpConversationsForSearch took 16638ms
The next step is to parse this file into something readable. We wrote a groovy script (see below), which generates output like:


===SELENIUM TEST TIMINGS===
69763=HomepageTest
54106=SubscribeToTribeConversationTest
50902=SubscribeToTagTest
48734=TagDedupingTest
44614=SubscribeToPrivateConversationTest

===INDIVIDUAL COMMAND TIMINGS===
153127=loginAndWait hits=291 avg=526.2096219931
103428=open /person/ hits=241 avg=429.1618257261
90431=open /logout hits=224 avg=403.7098214286
70059=invokeUrl /fixture/setUpConversationsForSearch hits=4 avg=17514.75
61926=open /login hits=246 avg=251.7317073171
61851=open / hits=133 avg=465.045112782
61034=createMessageAndWait hits=114 avg=535.3859649123


So now it's trivial to identify slow running tests / test steps when optimising our build. It's also useful to spot slow loading pages.


Here's the groovy script, it's tailored to our environment but a good starting point.

import org.codehaus.groovy.tools.LoaderConfiguration
import org.codehaus.groovy.tools.RootLoader
import org.codehaus.groovy.grails.commons.cfg.ConfigurationHelper
import org.codehaus.groovy.grails.commons.ConfigurationHolder
import groovy.text.GStringTemplateEngine
import org.apache.commons.lang.StringUtils

Ant.property(environment:'env')
grailsHome = Ant.antProject.properties.'env.GRAILS_HOME'
grailsEnv = System.properties.'grails.env';
pluginDir = this.binding['seleniumPluginDir']

includeTargets <<> timings = []

int nextId = 0
String currentTest = 'anonymous-test'
SeleniumTiming previousTiming
SeleniumTiming currentTiming

class SeleniumTiming {
  String id
  String test
  String command
  String target
  String value
  int duration = 0

  String getTargetKey() {

      def trimmedTarget = target?.trim()
      def trimmedCommand = command?.trim()

      if (trimmedCommand == 'open' && trimmedTarget ==~ '/person/\\w+') {
          return '/person/'
      } else if (trimmedCommand == 'open' && trimmedTarget ==~ '/tag/[\\w-]+') {
          return '/tag/'
      } else if (trimmedCommand == 'open' && trimmedTarget ==~ '/station/[\\w-]+') {
          return '/station/'
      } else if (trimmedCommand == 'open' && trimmedTarget ==~ '/line/[\\w-]+') {
          return '/line/'
      } else if (trimmedCommand == 'open' && trimmedTarget ==~ '/conversation/subscribe?.*') {
          return '/conversation/subscribe'
      } else if (trimmedCommand == 'open' && trimmedTarget ==~ '/conversation/unsubscribe?.*') {
          return '/conversation/unsubscribe'
      } else if (trimmedCommand == 'loginAndWait') {
          return ''
      } else if (trimmedCommand == 'sendMessageAndWait') {
          return ''
      } else if (trimmedCommand == 'createMessageAndWait') {
          return ''
      } else if (trimmedCommand == 'reply') {
          return ''
      } else if (trimmedCommand == 'replyAndWait') {
          return ''
      } else if (trimmedCommand == 'registerAndWait') {
          return ''
      } else if (trimmedCommand == 'reportThisAndWait') {
          return ''
      } else {
          return target
      }
  }

  String toString() {
      "[$test] $command($target,$value):$duration"
  }
}

def parseTestLine = { String input ->
  currentTest = input.substring(input.lastIndexOf(File.separator)+1, input.lastIndexOf('.html'))
}

def parseCommandLine = { String input ->
  List data = input.tokenize('|')
  if(data.size() == 5) {
      previousTiming = currentTiming
      currentTiming = new SeleniumTiming(id:nextId++, test: currentTest, command:data[1],
          target:data[2], value:data[3])
      timings << duration =" Integer.parseInt(data[4])" parsefile =" {">
  input.eachLine { String line ->
      if(line.startsWith('warn: Starting test')) {
          parseTestLine line          
      } else if(line.startsWith('warn: Executing: |')) {
          parseCommandLine line
      }
  }
}

target ('default': 'Generate Selenium Timings Report') {
loadSeleniumConfig seleniumConfig
new File(seleniumConfig.reportdir).eachFileMatch(~/.*\.html/, parseFile)

  def longestFirst = [ compare: { a, b -> a.equals(b) ? 0: Math.abs(a) < overalltimings =" new">
  overallTimings[(values*.duration).sum()] = key
}

  Map commandTimings = new TreeMap(longestFirst)
timings.groupBy({ "${it.command} ${it.targetKey}" }).each { key, values ->
   int totalDuration = (values*.duration).sum()
      int noOfValues = values.size()
      commandTimings[totalDuration] = "$key hits=${noOfValues} avg=${totalDuration / noOfValues}"

}

  new File(seleniumConfig.reportdir, 'timings.txt').withPrintWriter { writer ->
      writer.println '===SELENIUM TEST TIMINGS==='
      overallTimings.each { writer.println it }
      writer.println '===INDIVIDUAL COMMAND TIMINGS==='
      commandTimings.each { writer.println it }
  }
    
  commandTimings.each { println it }

}

Running Selenium Tests In Grails 1.1

Anyone experimenting with Grails 1.1 might be interested to know that I've got the Selenium plugin working. If you put the following entries in your project's grails-app/conf/BuildConfig.groovy:

grails.plugin.repos.discovery.energizedwork="https://svn.energizedwork.com/skunkworks/grails/plugins"
grails.plugin.repos.distribution.energizedwork="https://svn.energizedwork.com/skunkworks/grails/plugins"

Then type grails install-plugin ew-selenium you should be good to go.

The command to run tests is grails run-selenium. The script now runs in the test environment by default so you no longer need to use grails test run-selenium.

You may need to configure the browser Selenium uses. To do so edit test/selenium/selenese/conf/SeleniumConfig.groovy and set the selenium.browser property. For some reason Firefox 3 needs to be specified as *chrome. For example on a Mac I have to use the setting

selenium.browser = "*chrome /Applications/Firefox.app/Contents/MacOS/firefox-bin"

I have no idea why this is necessary. I have upgraded the Selenium Server version included in the plugin but I found this was also necessary with the old 0.9.2 version.

I changed the plugin name to ew-selenium as there is a plugin called selenium on the main Grails plugin repository and it looks like Grails will ignore conflicting names on other repositories.

On selenium testing...

I updated my blog to list some useful tips for writing more resilient Selenium XPath tests. 

Please feel free to add more suggestions or otherwise comment.

Selenium and Firefox 3

Those of you who've tried to run the selenium test suite using Firefox 3 as the browser have probably found that Firefox hangs trying to create the temporary profile. You could do some tedious mucking about with selenium.browser="*custom /usr/lib/firefox/firefox -no-remote -P selenium", manually configure the proxy, etc. However, it turns out there's a simpler solution.

Please note this is nothing to do with Selenium IDE, if you're having problems with that in conjunction with Firefox 3 this won't help.

Effective testing on Grails

Gus and I will be presenting on "Effective Testing on Grails" at the London Groovy and Grails User Group next week Wed 1st of October, @18.30. Please feel free to attend if this seems relevant to you. A synopsis of the talk and registration for the event are all available at the following link.

http://skillsmatter.com/event/java-jee/ggug