Mobile testing isn’t easy. 

It poses many different challenges to you as a software tester or mobile developer. For example, there are the different devices that you need to handle with the different hardware and software combinations. On top of that, the devices offer a plethora of sensors that your mobile application can interact with, and these built-in sensors can have a huge impact on your mobile application. In the worst case scenario, your app can crash and will stop working because the integration with the sensor has a bug.

Next to the device challenges - there are many different mobile testing scenarios to consider when testing mobile apps. For example, you should be testing for interrupts, app permissions, accessibility, usability, gestures, 3rd party app integration, local storage, mobile networks and battery consumption just to name a few (for more information see the Mobile Testing Cheat Sheet). All these scenarios are in addition to regular feature testing. And last but not least, mobile users have high expectations when it comes to mobile apps and their quality, which adds even more pressure on mobile development teams.

This all sounds stressful when working in a mobile development team, right? And I haven't even mentioned mobile test automation yet.

Mobile test automation adds another challenge to your long list of things to cover during the day to day work in a mobile development team. Finding the right test automation tool, setting up the test automation solution, integrating it into a CI/CD pipeline as well as writing and maintaining the test scripts is more than a full time job.

Existing Mobile Test Automation Solutions Are Not Enough

‍

The existing mobile test automation tools on the market such as Appium, Maestro, Detox, Espresso or XCUItest offer a great set of features. And the list of the mentioned tools is by no means complete - there are many mobile testing tools available to choose from. The mentioned tools will help teams to implement a good set of test scripts that cover important parts of the product. Each tool has its pros and cons, and mobile development teams should take their time to pick the tool that fits best into their development and testing pipeline. At the same time, mobile development teams must consider a combination of tools, to cover most of the testing scenarios.

For those of you working already in mobile development or testing, know that there are features that are hard or nearly impossible to automate. 

For example, leaving the app context while running an automated script to test something on the operating system level (e.g. the settings) is not possible or easy to do for the aforementioned tools. Usually the mobile automation tools can only run the automated scripts to a modified app file. The app file must be signed with a development provisioning profile or certificate to execute the tests. However, if you want to submit and download an app to the app stores, you need to sign the app with a distribution profile, and all apps (e.g. settings app) is not signed with your development profile but with the one from the other development party.

There are workarounds and dirty hacks that would make it possible to leave the app context and to test outside of the app sandbox. However, the investment a mobile development team has to incur is likely not worth it. 

Furthermore, the workarounds will impact the operating system and change the running apps just to make it possible to run the automation, e.g. to test something inside the settings app. And more importantly, you can’t guarantee that the modifications will act in the same way as on the customers’ devices - and that is why we test! In the worst case, it will hide critical issues from you. Mobile development teams usually end up incurring a large amount of maintenance time to keep these workarounds up and running.

{{blog-cta}}

What Are Important Mobile Tests To Automate?

If you develop and test a mobile app, I bet your app relies on platform specific features to provide the best possible user experience. There are features such as:

‍

• Push notifications
• In-app purchases
• Deeplinks from other apps
• 3rd party app integrations
• 2-factor authentication

Just to name a few! But have you ever tried to automate these features with one of the mentioned tools from above? As mentioned, it’s somehow possible but very challenging, and not something that will simulate a real user. 

And there are more things to cover with automation that have a direct impact on your app and the overall quality. 

For example, what will happen if you change the language of the device? Is your app able to handle the language change and is the app working as expected? Can you automate the setup or communication to other connected devices such as smartwatches? Again just to name a view scenario and there are more depending on the purpose and use case of your mobile app. And cases like this are impossible to automate.

Mobot Is Here To Help

I hope you are not too concerned that you are missing out on important parts of your app with automation. 

There is a solution for your challenges. And the solution is called Mobot. Mobot is a robot-assisted automation solution and is using real robots, with mechanical arms, to interact with the app on a real device like a real user would do.

The really interesting part of Mobot is that you don’t need to write the code for those difficult parts. All you need to do is to record a video of the scenario you want to test on real devices. Then upload this video, alongside with your device wishes, to the mobot service. A team of mobot engineers will prepare the robots for you with the test scripts. They will prepare the selected devices and the mobot team will handle the execution and maintenance of the tests. Once the test run is over, you will receive a detailed report of the execution results.

With Mobot you are able to automate the impossible test scenarios, such as

• Push notifications
• In-app purchases
• Deeplinks from other apps
• 3rd party app integrations
• Changing the device settings
• Communication between two devices e.g. for messaging apps
• And many more cases

Want to see the robots in action? Then take a look at the video that I created on my YouTube channel about Mobot and how they can help you automate challenging mobile tests and improve your test & device coverage.

Author Bio

Daniel Knott loves digital products with high quality being web or native mobile applications. Currently, he is working as Head of Product Quality Engineering at MaibornWolff in Germany, where he shapes the future of testing for his clients. In the past 14 years, he worked as Lead Software Test Engineer for different native mobile apps and products. 

Daniel wrote two books - Hands-On Mobile App Testing and Smartwatch App Testing and is a frequent blogger at www.adventuresinqa.com and conference speaker. Since 2022 he also created his YouTube Channel about Software Testing.

Get in touch with him via

- LinkedIn: Profile (https://www.linkedin.com/in/daniel-knott/)

- Twitter: @dnlkntt (https://twitter.com/dnlkntt)

- YouTube: https://www.youtube.com/c/DanielKnott/

‍