Running the ‘Mem Buster’ endurance test

I blogged a few weeks ago about how I was able to demonstrate improvements to the memory usage of Firefox using endurance tests. The test I was using was inspired by Stuart Parmenter’s Mem Buster test, and it has now been checked into the repository and available for anyone to run.

At the moment it’s only possible to run the Mem Buster test from the command line (hopefully Mozmill Crowd support won’t be too far off). The first thing you’ll need is the mozmill-automation repository checked out. If you already have this then you’ll need to do a pull and update to make sure you have the latest changes.

hg clone

Then, from the mozmill-automation directory, run the following command:

./ --reserved=membuster --delay=3 --iterations=2 --entities=100 --report= /Applications/

The reserved argument test the script to only run the Mem Buster test and not the general endurance tests. The test opens a site for each entity, so by specifying 100 entities and 2 iterations it will open a total of 200 sites. The delay of 3 seconds is from the original Mem Buster test. I would recommend including the report argument as this shares your results and allows you to see the visualisation of memory usage during the test. The final argument is the location of the version of Firefox you want to run the tests against.

Below is a screencast demonstrating the Mem Buster endurance test on Windows 7:

If you’re interested in following the progress of the endurance tests project, check out the project page. For further help you can find the documentation here, post a comment to this entry, or ask a question in the QMO forums.

Goodbye micro-iterations. Hello entities

Last month I blogged about the addition of micro-iterations in endurance tests. I was never 100% happy with the name for these, and although ‘micro-iteration’ is a good description of what’s happening (it’s a loop within a loop) it’s difficult to say, and can be difficult to clearly identify when you’d use them.

During a between session chat with Geo Mealer during the recent QA Automation Services work week, he suggested calling these ‘entities’. This is perfect, because the purpose of these inner loops is to allow an endurance test to interact with multiple entities rather than just one.

The simple example is the new tab test, which opens multiple new tabs. In this case, the tab is the entity. A more involved example is the in-progress app tab test, where a tab is still an entity but we interact with it in multiple ways. First, we open the specified number of tabs, then we pin them all, then we unpin them all.

The rename has now landed, and documentation has been updated. From the perspective of writing tests nothing has changed, the only changes are the name of the command line argument and the method names in the endurance.js shared module.