Since late last year I have been working on a prototype of an Endurance Testing project for Firefox. The idea is to use our existing Mozmill framework for automating UI testing of Firefox to write tests that stress and strain the browser over time. I’ve heard many times from people that Firefox needs to be restarted once in a while because it’s become sluggish, and indeed I’ve experienced this myself. The problem is that there are rarely clear steps to reproduce this issues as they normally are an accumulation of many actions over an extended period of time. What actions cause a degradation in performance, and why? This is what we hope to discover with the endurance tests project.
The initial implementation of endurance tests is rather simple: Create a test snippet that exercises a function of Firefox, and execute it repeatedly whilst gathering details of resources in use. Ultimately we may come up with more elaborate tests, and but it’s important to get a proof of concept.
There are several components to the endurance tests:
- Command line automation script
- Test snippets
- Resource gathering
The number of iterations each snippet repeats can be set on the command line, as well as an optional delay between each iteration. The normal command line options allow for logging, and reporting to a Mozmill Dashboard instance.
Triggering the endurance tests currently looks something like this:
./testrun_endurance.py --delay=1000 --iterations=50 --report=http://davehunt.couchone.com/mozmill /Applications/Minefield.app
This will then launch Firefox, run through all of the endurance tests (each one iterating over it’s test snippet 50 times), and then close Firefox. Because I’ve included a report parameter, the report will also be sent to the Mozmill Dashboard instance. These reports are currently available here.
Here’s a short screencast that demonstrates running the endurance tests: