iPhone test automation using KIF (Keep It Functional)

In my last posts I dealed a lot with Android and how to implement automated regression tests using tools like Robotium. This post describes how to implement automated tests using the tool KIF (Keep It Functional, http://corner.squareup.com/2011/07/ios-integration-testing.html). KIF is an open source test framework developed by the company square (https://squareup.com). It is an iOS integration test framework that allows you to implement test cases with objective C that can be executed currently only against the iPhone/ iPad simulator. The KIF tests are integrated to your workspace, there is no need for additional servers or services that must be started.

KIF is like a Grey Box Test Tool because you need knowledge of the structure of the App you want to test. KIF use the provided accessibility labels from iOS to interact with the UI elements in the App. The labels are normally used to make Apps available for people with visual disabilities.

With KIF you are able to simulate real user interaction like tap, click, drag, enter text and many more. Also you are able to verify elements on the screen.  It is really fast in executing the test cases on the simulator. Also it is really easy to integrate to your xCode and continuous integration environment.


KIF uses undocumented Apple APIs, but this is not a problem. That is true for the most iOS test frameworks that are currently available. It is really important that KIF is not a part of your production code, otherwise Apple will deny your submission because of the undocumented API.

To install and configure KIF please follow the provided information on the github page https://github.com/square/KIF.

Accessibility Inspector

If your project is configured with KIF you have also to activate the “Accessibility Inspector” on the simulator to have access to the labels. Do the following steps on the simulator:

  1. Open the Settings
  2. Click on General
  3. Click on Accessibility
  4. And activate the Inspector

Afterwards you see the inspector with a colorful layer within your simulator. You can drag the layer wherever you want. Clicking the small (x) on the left side opens the inspector. If you now do a single tap on an app you get information about the accessibility label and some more information (screenshot). If you do a double click you can e.g. enter the app. If you close the inspector layer with the (x) again, you have again single click on the simulator. NOTE: The inspector must be activated to use KIF for testing. Now you can start implementing your test classes.

iPhone Simulator with Accessibility Inspector


Code Example

KIF has three main classes that can be used to implement tests within your project. There is a test runner the KIFTestController, a scenario KIFTestScenario and a step KIFTestStep. The KIFTestController is like a test suite where all the test scenarios are listed that should be executed. A KIFTestScenario is like a container that includes several steps that should be done within a test. A step is the smallest and simplest action that represents user interaction like tap, enter text or wait for views.

The following listings show the Controller h-file and m-file. The m-file includes the list of possible scenarios.

MYTestController.h

#import <Foundation/Foundation.h>
#import "KIFTestController.h"
@interface MYTestController : KIFTestController {}
@end

MYTestController.m

#import "MYTestController.h"
@implementation MYTestController

– (void)initializeScenarios {

[self addScenario:[KIFTestScenario scenarioLoginWithWrongCredentials]];   // Add here additional scenarios
}

@end

Now you need to implement the scenario scenarioLoginWithWrongCredentials. You need again a h-file and m-file for that. The h-file includes the scenario definitions. The m-file is the class where the test methods are implemented.

KIFTestScenario+MYLogin.h

#import <Foundation/Foundation.h>
#import "KIFTestScenario.h"

@interface KIFTestScenario (MYAdditions)

+ (id) scenarioLoginWithWrongCredentials;

@end

KIFTestScenario+MYLogin.m

#import "KIFTestScenario+MYLogin.h"
#import "KIFTestStep.h"
#import "KIFTestStep+MYLogin.h"

@implementation KIFTestScenario (MYLogin)

+ (id)scenarioLoginWithWrongCredentials{
KIFTestScenario *scenario = [KIFTestScenario scenarioWithDescription:@”Wrong credentials”];

// enter wrong credentials in the username and password field
[scenario addStep:[KIFTestStep stepToEnterText:@”wrongusername” intoViewWithAccessibilityLabel:@”Username”]];

[scenario addStep:[KIFTestStep stepToEnterText:@”wrongpassword” intoViewWithAccessibilityLabel:@”password”]];

// click the done button on the keyboard
[scenario addStep:[KIFTestStep stepToTapViewWithAccessibilityLabel:@”done”]];

// verify that the error message is shown with a localized string
[scenario addStep:[KIFTestStep stepToWaitForViewWithAccessibilityLabel:LocalizedString (@”FAILED_MESSAGE”)]];

// click the button on the error message
[scenario addStep:[KIFTestStep stepToTapViewWithAccessibilityLabel:LocalizedString (@”OK_BUTTON”)]];

return scenario;

}

@end

Now you have a really simple test that tries to login with wrong credentials against an app. After pressing the done button on the keyboard KIF verifies with the stepToWaitForView… method that the right error message is shown.

Continuous Integration

It is highly recommended to integrate KIF to a CI server like Jenkins to build up a regression test suite to be sure that the existing functionality is still working after a commit. To integrate and execute your KIF tests to your CI server you must be able to start the simulator from the command line. To do this, you need an additional tool called WaxSim (https://github.com/square/waxsim). One Note from the KIF github page: “Note that the Square fork of WaxSim provides a number of bug fixes and some useful additional functionality. Your CI script should resemble something like the this:” Adapt the following script to your environment and add it to your Jenkins build configuration:

#!/bin/bash

killall “iPhone Simulator”

set -o errexit
set -o verbose

# Build the “Integration Tests” target to run in the simulator xcodebuild -target “Integration Tests” -configuration Release -sdk iphonesimulator build

# Run the app we just built in the simulator and send its output to a file

# /path/to/MyApp.app should be the relative or absolute path to the application bundle that was built in the previous step

/path/to/waxsim -f “ipad” “/path/to/MyApp.app” > /tmp/KIF-$$.out 2>&1

# WaxSim hides the return value from the app, so to determine success we search for a “no failures” line

grep -q “TESTING FINISHED: 0 failures” /tmp/KIF-$$.out

That’s it, I hope this post gives you an impression on how to build up a test automation suite for iPhone or iPad apps! You should now be able to build your own iPhone regression test suite.

Further information can be found in this really good google group about KIF: https://groups.google.com/group/kif-framework?pli=1

This post based on sources from:

http://corner.squareup.com/2011/07/ios-integration-testing.html
https://github.com/square/KIF
https://github.com/square/waxsim

Have Fun!