If you have implemented many automated tests with Robotium, you will notice that Robotium is not the fastest tool when it comes to test execution time! Especially in big projects with more than 100 automated Robotium tests the execution time is really long (>1h). For nightly regression testing this is not a problem, but If you want to know if a simple commit from a developer break something, executing the whole test suite is not very efficient. To get a much faster impression of the quality of the last commits, you can build for example a smoke test suite. This suite covers the basic functionality like login, clicking on important entries but not all edge cases, like a full regression test suite will do.
In one of my last tutorials I described how to setup Robotium with Jenkins and Ant. Today, I want to show you, how to extend your Robotium test automation to have different test suites. These test suites can be executed from Jenkins again.
The secret behind the different test suites are annotations. These annotations are provided by the Android suite builder. If you want to have your test methods in different test suites you just have to annotate the test method with one of the following types (there are more annotations available):
Here is a simple example of a test method with a annotation:
@SmallTest
public void testSomething() throws Exception{
solo.searchText("TextHeadline");
solo.enterText(0, "MyName");
assertTrue("ERROR: Text not found", solo.searchText("Some Text"));
}
Thats basically all you have to do with your test methods. Don’t forget to import android.test.suitebuilder.annotation.SmallTest; in your test class.
If you want to execute all the @SmallTest methods you have to start the Robotium tests in the following way:
./adb shell am instrument -w -e size small com.XYZ.testcase.myRobotiumTests/android.test.InstrumentationTestRunner
By adding the parameters -e size small you tell the InstrumentationTestRunner to execute the tests with the @SmallTest annotation. If you want to execute medium or large tests just replace small with medium or large.
After choosing all the test methods that should be part of the smoke test suite, you can copy an existing Robotium Jenkins job and extend the ‘Execution Shell’ entry from
./adb shell am instrument -w com.XYZ.testcase.myRobotiumTests/android.test.InstrumentationTestRunner
to
./adb shell am instrument -w -e size small com.XYZ.testcase.myRobotiumTests/android.test.InstrumentationTestRunner
Save the new Jenkins job and execute your new Robotium smoke test suite.
Have fun and keep on testing!
Greets,
Daniel