Mobile Device Applications Testing Tips - Security Testing

General considerations

There are a few things to consider around security at the device level when comparing mobile applications to desktop or web applications:

1. Most mobile devices assume one user; they do not have multiple accounts. (Except in the case of multiple email addresses configured per device, which should be tested especially if your application deals with the device’s Contacts or Email functions). But in general there is no concept of user switching, different profiles, or permissions based on user level.

2. It is up to the user whether or not they configure a password/unlock pattern for their device at all.

3. All necessary encryption needs to occur at the application level because the device will not necessarily handle this.

4. Outside communication of any sensitive data should be done over SSL/TLS because it can’t be assumed that the user will configure their Wi-Fi to be secure.

Web applications

While running most web security testing tools on a mobile device is virtually impossible, you can run your mobile website on a PC itself. Then you will be able to test the application with common web security testing tools, such as Firefox Web Developer add-on, SQL Inject Me, XXS Me, Web Scarab, or Brutus. A web proxy can also be used to intercept all mobile device traffic to monitor data and test for security issues.

Application and device permissions

During installation of your application on a mobile device you are likely to encounter options for application execution permissions. For example, on Windows Mobile devices there are three levels: Privileged, Normal, and Blocked, which are specified by the application’s certificate, and define how much access the application will have to the device itself. On Blackberry, you may be asked to allow, prompt for, or deny your application access to various connection types, interactions, or user data on the device.

Research the permissions configuration on your device and design your testing to ensure that your application can get access to the device areas it needs in order to function properly, and it provides useful error messages directing the user to set the permissions properly if it cannot. The device itself could also have different security configurations, such as security off, various levels of prompting, required third-party signing, or locked. It is important to verify that your application can be successfully installed at all expected device security level settings. For example, on Windows Mobile a high security level may not allow an unsigned CAB file to be installed, even if it is wrapped in an MSI file. This kind of behavior must be discovered and mitigated. Even after installation, it is possible to change the security settings of your device or application permissions. You should include some test cases to cover these scenarios as well.

Mobile Device Applications Testing Tips - Stress Testing

Although mobile device’s memory and power have improved over the past few years, mobile applications still have much more memory and CPU constraints than desktop applications. With users expecting applications to hold up even under high traffic conditions with many applications running, stress testing is a must to find exceptions, hangs, and deadlocks that may go unnoticed during functional and user interface testing.

Techniques

The following are some stress testing techniques to try:

1. Load your application with as much data as possible to try to reach its breaking point. For instance,

Windows Mobile devices can accomplish this with either ActiveSync (Windows XP) or Windows Mobile Device Center (Windows Vista and newer). This will give the user full access to the Windows file system on the device to add needed testing files where they are required.

2. Perform the same operations over and over again, particularly those that load large amounts of data repeatedly.

3. Perform the repeated operations at varying speeds – very quickly or very slowly.

4. Leave your application running for a long period of time, both interacting with the device and just letting it sit idle, or performing some automatic task that takes a long time (like displaying a 200-photo slideshow).

5. Randomly send screen taps and keystrokes to your application.

6. Have multiple applications running on your device so you can switch between your application and other device applications often, and at different states within your application:

a. On Android or Blackberry you can press and hold the hard Home or Menu key, respectively, to quickly switch between running applications.

b. Use all means to launch your application, for example on Windows Mobile use the Touch Flo, the file explorer, and if the application is already launched, maximize and minimize the application.

Non-reproducible issues

Another general tip related to stress testing – do not be afraid to report bugs that you haven’t yet found a way to reproduce 100% of the time. If you can at least provide crash logs and a best guess as to how to trigger the problem, that is usually enough to give the developer some ideas on narrowing down the root cause.