Home » iPhone

Category: iPhone

XCTest and OCUnit on real devices in the cloud

Yesterday, I read the latest blog post from AppThwack, that they now support real iOS devices on their test cloud. Developers and testers are now able to run their automated XCTest or OCUnit test suites against the provided cloud with more than 46 iOS devices, from iPhone to iPad.

iOS devices in the cloud

The big advantage of having the OCUnit support in the AppThwack cloud, is that a number of popular iOS testing frameworks are built on top of OCUnit. There are tools like KIF (Keep It Functional), Kiwi, Spectra or OCMock. If you already have an automated test suite with one of those tools, you can easily execute them now against the cloud. All you have to do is upload your bundle either with the provided UI or API.

See the short AppThwack tutorial, on how to upload and test your project against the cloud:
http://blog.appthwack.com/xctest-and-ocunit-on-real-devices/


At the end of the test run, a really nice report is generated. A sample report can be found here.

AppThwack Sample Report

If you want to try the cloud testing service, just register and get 100 free device minutes for testing.

What kind of cloud testing services are you using?

Have a nice friday and weekend.

Keep It Functional 2.0

Nearly 2 years ago I wrote the post “iPhone test automation using KIF (Keep It Functional)“. The fact that I am not working with KIF in one of my current projects, I missed the new KIF 2.0 release. The new version was released in september 2013 with a major rewrite of the framework.
KIF 2.0 is built on top of ocunit and is able to execute the tests sequentially as they appear in your code. In the old version the test execution based on steps and scenarios.

If you check the example on the KIF github page you will see the whole new structure of KIF. In the old version you had to create a KIFTestScenario object. Then you had to add several KIFTestSteps to that object to interact with your application. In the new version you have two main classes KIFTestCase and KIFUITestActor. The ocunit test runner is loading the test cases from KIFTestCase. Inside the tests the tester object is interacting with your application under test. The tester object offers several actions that can be used for testing. The most used ones are:

  • tap this view
  • enter text into this view
  • wait for this view

See a sample login test method with KIF 2.0

...
- (void)testLogin
{
    [tester enterText:@"testUser" intoViewWithAccessibilityLabel:@"Login User Name"];
    [tester enterText:@"testPassword" intoViewWithAccessibilityLabel:@"Login Password"];
    [tester tapViewWithAccessibilityLabel:@"Log In"];

    [tester waitForTappableViewWithAccessibilityLabel:@"Welcome to your account"];
}
...

Compared to the old version

...
- (id)scenarioLogin{
   KIFTestScenario *scenario = [KIFTestScenario  
      scenarioWithDescription:@"Login with credentials"];

   [scenario addStep:[KIFTestStep  
      stepToEnterText:@"testUser" intoViewWithAccessibilityLabel:@"Username"]];

   [scenario addStep:[KIFTestStep 
      stepToEnterText:@"testPassword" intoViewWithAccessibilityLabel:@"password"]];

   [scenario addStep:[KIFTestStep 
      stepToTapViewWithAccessibilityLabel:@"Log in"]];

   [scenario addStep:[KIFTestStep
      stepToWaitForViewWithAccessibilityLabel:LocalizedString (@"Welcome")]];

   return scenario;

}
...

Also the new version of KIF relies on accessibility labels, so be sure to provide a full accessible app. I really like the new version of KIF because it is much shorter and clearer.

Besides the new structure, KIF also offers some more functionality:

  • Use breakpoints between your test steps.
  • Test failures are shown in Xcode.
  • No need to create a duplicate target.
  • It is working with Xcode 5 command line tools, test navigator, and Bots.
  • And you can combine it BDD frameworks like Specta.

Have fun with new version of KIF!

Source: https://github.com/kif-framework/KIF

Appium on Sauce – A new tool for testing your iOS apps in the cloud

Two days ago Sauce Labs introduced a new tool for iOS test automation: Appium on Sauce. Appium on Sauce is able to automate hybrid or native iOS apps. The new tool based on the open source tool Appium written in Node.js. Currently the tool support iOS devices only but Android support is on its way.
Appium uses Selenium commands and map these commands into a format for UIAutomation. It uses the WebDriver JSON Wire Protocol to drive UIAutomation.
Sauce Labs says, the tool requires no recompiling or modifications on the app you want to test. Tests can be written in any programming language and testing framework using the Selenium API. Appium on Sauce needs no setup and no maintenance. Tests can be run in parallel across several machines in the cloud. CI support is also included.

Appium on Sauce
Appium on Sauce

Currently the tool is only available by invitation to see if the environment is stable enough to handle many users and apps in parallel. For me this tool looks very promising and we will see how well it can be used.

See Appium on Sauce in action:

Greets

Mobile App Update Testing

During the development process of mobile apps manual and automated testing is highly recommended. But there is another test you should do before releasing or submitting your app to a store. Testing the update procedure! Testing the update procedure from the current store version to the new release candidate is really important and it is often not done by developers or QA experts. With this test you verify, that the new app is not affecting the old installed version. Here are three facts that will be covered with that test.

Test that the update…

  • … will not logout a user from the app e.g. before the update he was logged in.
  • … will not affect the local database e.g. existing data will not be modified or deleted.
  • … will be installed correctly from the store (simulation).

You can also test the update from far older versions to the latest one, to see what is happening.
For the two big players Android and iPhone I will describe, how to test the update from one version to another using different tools.

iOS
On the iOS side there are two ways of testing the update procedure. The first one can be done with iTunes. See also the Technical Note TN2285 from Apple.

  1. Build an adhoc app version of the app that is currently live in iTunes store. HINT: This version must have the same bundle identifier as the new app.
  2. Be sure that no older versions of the app are installed within iTunes and on your test iOS device (Sync with iTunes to be sure).
  3. Drag the app from step 1 into iTunes and sync the version to your test device.
  4. Launch the app and do some manual testing to see that the app is working.
  5. Build the new release candidate version of the app and drag it to iTunes and sync to the device. iTunes should confirm, that the older version will be removed. HINT: DO NOT delete the old build! In the next step iTunes will install the new app over the old one and will simulate the update from the app store.
  6. Launch the new version and see if everything is ok.

The second way of testing the update procedure for iOS is using the iPhone Configuration Utility. Using this tool is more comfortable, especially if you want to test the update procedure on more iOS test devices like iPhone 3G, iPhone 3GS, iPhone 4(S), iPhone 5 or iPads.

  1. Build an adhoc app version of the app that is currently live in iTunes store. HINT: This version must have the same bundle identifier as the new app.
  2. Be sure that no older versions of the app are installed on the test device.
  3. Install the app from step 1 to the devices you want to check.
  4. Launch the app and do some manual testing to see that the app is working.
  5. Build the new release candidate version of the app and install it with the tool. The update procedure will be simulated.
  6. Launch the new version and see if everything is ok.


Android
The same can be done also on the Android side. To test the update for Android you can use adb to simulate the procedure.

  1. Install the current Google Play store version of your app to your phone.
  2. Check that the version is working.
  3. Build a release candidate of the Android app. HINT: Be sure to sign the release candidate with the Play store keystore.
  4. Use the following command to test the update procedure:
    ./adb install -r APPNAME.apk. The option -r means reinstall the app and keeping its data.
  5. The new version is now installed and can be tested.

That’s it! Try it for your next release!

Greets!

Overview of Mobile Test Automation Frameworks

I found a really nice overview about Open Source Mobile Testing Frameworks for Test Automation. The overview was created by Dominik Dary (eBay). In his table he showed how many test automation frameworks are currently available for Android or iOS testing. The complete blog post can be found here:

http://www.dary.de/2012/03/open-source-mobile-test-automation-frameworks

A big thank you to Dominik, who provided this overview!