Prioritizing and Filtering Tests
By default, the Eclipse JUnit integration always runs the tests in a
suite in the same order (although it is sometimes difficult in advance
of the first run to determine what that order will be). For continuous testing,
this can be frustrating. Consider our current failure, on
testTopJoke. As you try to fix the error, the most interesting thing
to you will be whether testTopJoke finally passes. However, every
time continuous testing restarts, it runs up to three passing tests before
trying testTopJoke. To verify this, open the Continuous Testing view,
and choose the Test Order tab:
Since these tests are
pretty fast, it doesn't seem too annoying at this point. However, you
can imagine that the frustration would build if each test required 5
or 10 seconds of time, or if you had to wait for 100 tests to run before
the one you're interested in.
To help alleviate this issue, continuous testing allows for several
kinds of test prioritization and filtering:
Open the properties dialog for topten,
and choose Continuous Testing Properties.
- On the
bottom left side of the resulting dialog box, you have the option of
choosing from six different test prioritization strategies. Select
"Most recent failures first": we have found that this is often the
most helpful strategy, since the most interesting tests are often
those that are currently failed, or that we've recently fixed.
- Press OK. The tests are re-run, in order to figure out which
tests are currently failing.
To see prioritization in action, leave the Continuous Testing view
open. Right-click the marker in the Problem view, and Rerun this
test. You'll see that testTopJoke, the
failing test, has moved up to the top of the test run order.
Along with prioritizing tests, filtering can also be done on test suites.
Some filters have been included with the plugin and are described
here. The prioritizers are described
here. Prioritizers and filters are also
known as test manipulators. These manipulators are
given information to help them order and filter the tests. They can use information
about previous results, and information about which files have changed.
Look here for more information about
developing new filters or prioritizers.
