Mobile App testing with Kobiton
Sandarekha Ramanayake
Associate QA Specialist at TechLabs |BSc (Hons)| |MBA| |ISTQB®
How many Operating systems do we have today for mobiles? How many mobile device varieties do we have in market? This may be a wow experience for a customer who expects to buy or use a mobile device with latest technologies. But this comes as a nightmare for a mobile testing QA engineer!!
According to seven testing principles, Exhaustive testing is impossible (ISTQB Version 2018 V3.1) We cannot have all the possible scenarios to be tested and verified. But this is not going to be the answer for a angry customer of our mobile app OR burning face team lead who got the customer complaint in the morning for a different mobile device we could not tested. Even if the developers may not able to fix soon - it is better to know where the problem is and which devices may go wrong with the new features.
When it comes to mobile app testing all these apps used to behave with QA engineers' usual device in a nice manner. But it wears the naughty face when we tries it in different mobile versions and different devices. So we see barriers in mobile testing as
- We do not have resources to get all the mobile OS versions and devices to test.
- Even we get the device issues from customers we need to reproduce and get developers to check in order to fix it for the same device
The solution comes with device farms.
Device farms provides access to various types of real mobile devices in current market. We can upload our mobile apps to the device farm. When we select a device, they direct us to install the app in that device and we can perform our functionalities. If we are keen in automation, it also allows us to set up the scripts and execute it several devices parallelly. We can take screenshot evidences - video of the test execution - device logs - execution report related to our test.
There are number of device farms to serve for mobile testing now. We can list them as
It is better to take trial periods to do a POC and move ahead for the purchase if it fits to your product expectations. Among these device farms , below explains the experience with Kobiton device farm testing flow and pros - cons with it.
Kobiton Login offers you to start with a trial period by registering. You can get the login and explore.
First you have to get a release build of the app from your mobile developers. You can upload the apk (android) or ipa (iOS) file to Kobiton from here. After a successful upload you can see the app with App ID, App Package Version, Package, and if you use automation, app capability to use also provided. If you have multiple versions of the same app, it is easy to mark the App ID to avoid confusions.
Then we can check the device pool from devices menu.
Kobiton offers more than 300+ devices in android and iOS. You can simply search for the desired device here. It displays the device is available for now, or is it using by someone now or is it offline now. Then "Launch" the device to install your app.
Now the device get connected. While it is loading, you will see guiding messages to use the device. You can see set of menu in left side to use to handle device. In right side there is the device log. In top right side you can see the app installation guide menu.
You can simply use the mouse to control the device. Typing from key board, copy paste also help you to continue your testing. After the test, you can exit from your session and check the reports.
Sessions are saved as video, log and screenshots as well. You can integrate JIRA with these sessions and it smooth the reporting flow to your development team considering device issues.
Kobiton also provides you to maintain team members login to device farm. Uploading apps can control as per the user. And also Kobiton allows you to automate testing with Appium and Katalon which is much smooth to work with.
When it comes to pricing, Kobiton charges per usage minutes. How much time you spend with a particular device from launch to exit the session will be counted and billed. You can check their pricing from here. Their minimum package for 50$ (in 2021) serves 500 device minutes much worth for a average QA team. And their support team is quick in responses to communicate upon any device issues and requests.
Besides Kobiton, it is worth to have a idea on some general facts about device farm usage.
- Make sure to cover your target in POC of the device farm. Do your home work about your app and make sure to have exact purpose to use the device farm.
- It is better to go for a automation than clicking one by one device. Then you can check the recorded video of the sessions to get your point.
- Device minutes are billed for you. So make sure to launch the device and use as it connected and to exit the session after your work. Usually they wait for about 15 minutes and exit it automatically when there is no response, but do not waste device minutes.
- Try to cover exact test case with target devices. Most of the time OS versions may matter. Then you can have the map to device where you should test.
- Access to phone contacts, access to settings , access to phone gallery may blocked by device farms sometimes. So pay your attention to when design the test cases with it.
Device farms helps to manage our apps from the issues comes with variety of devices. Mobile testing with Kobiton device farm gave much flexible experience so far. It may depend on your app requirement but better to have a look with current device farm solutions. You can try for mobile automation combined with device farms to make your QA life easier.