iOS UI testing: Getting started with Xcode
UI Tests for your iOS apps guarantee your users are getting your best experience, and give you confidence in your code
So your iOS app is comprehensively unit tested, meaning you can guarantee your business logic functions as you expect. But what about the views that your mobile user experiences? How do you guarantee you're showing the information you think you are? Unit testing views just isn't appropriate, but isn't UI testing fragile and difficult to get started for iOS apps? It is true that UI testing does take longer, and uses more resources, compared to unit tests. (which is precisely why you should be using unit tests to test all your code -- they are good at it.) But automated UI testing takes so much less time and resources than manually testing each of your app's experiences, and getting started with iOS UI testing in Xcode is probably simpler than you expect.
You don't need to do a big bang, consider adding some happy path tests to protect against regressions. Then, as you gain confidence in your new tests, gradually add to them as you create new features. In this article I am going to cover how to get started writing your first test and how to let Xcode help you while you gain knowledge and confidence. I’ll also give you some points to consider when introducing more tests, such as the type of tests you can add and what they are good at, and how best to architect your testing.
Writing your first UI test
In Xcode's navigator pane open the test navigator. Then, in the bottom left, click the + button and choose 'New UI Test target'.
The default values here are most likely correct, but if your project has multiple targets, check that you're adding the tests to the right one. In the file inspector you'll see Xcode has created a new UITests group with a sample tests file inside.
To keep things simple, lets delete everything inside the class definition so you have something that looks like this:
import XCTest
class My_Sample_AppUITests: XCTestCase {
}
Now to create our test. As with unit tests, test functions must start with the word test, so let's create a new function.
func test_myFirstTest() {
}
The first thing we need to do is to tell our application to launch. We do this with XCUIApplication().launch() but we'll want to keep the application reference to use later, so let's go ahead and add this to our new test function.
let app = XCUIApplication()
app.launch()
If you run this test now you'll see your app is launched, and the test passes. The next stage is to interact with our app.
How to make your test navigate your app
The next step is to navigate our app. The best way to first learn this is to let Xcode do it for us. Place your cursor inside your new test function below the call to launch the app. You'll see in Xcode's bottom bar a red 'record' button. Click this button. Xcode executes your test up to the point where your cursor is and pauses. Now any actions you perform in your app are recorded in code by Xcode.
As an example I have created a very simple main/detail app with a table view that pushes to a detail view displaying the value the user tapped on the table. The app has four items, labeled 1 - 4.
For my test I want to tap on item 1 and check '1st Item' is shown on the detail screen. I'll navigate back, then do the same for item 4. So I'll perform those navigation actions in the simulator. Here's the full test function, including the code Xcode has written for me.
func test_myFirstTest() {
let app = XCUIApplication()
app.launch()
let app = XCUIApplication()
let tablesQuery = app.tables
tablesQuery.staticTexts["Item 1"].tap()
let myAppButton = app.navigationBars["UIView"].buttons["My App"]
myAppButton.tap()
tablesQuery.staticTexts["Item 4"].tap()
myAppButton.tap()
}
Clearly this needs tidying up a little, but if we remove the second let app = XCUIApplication() Xcode added for us, we do now have a functioning test. The test repeats the actions we performed a moment earlier, but it still doesn't assert on any content on the screen. So lets add those next.
Checking your screens with your test
Lets drill down into some of the navigation code that Xcode has just written for us:
let tablesQuery = app.tables
tablesQuery.staticTexts["Item 1"].tap()
The majority of this code, everything before .tap() is an element query. A request in our code for the test runner to find an element on screen. This element doesn't exist in our test until we do something with it. This could be interacting with the element, as above, or it could be checking a property on the element. We know there should be four cells labeled Item 1 - 4, so let's write an assertion for these. We do this in the same way we would assert something in a unit test.
XCTAssert(tablesQuery.staticTexts["Item 1"].exists)
XCTAssert(tablesQuery.staticTexts["Item 2"].exists)
XCTAssert(tablesQuery.staticTexts["Item 3"].exists)
XCTAssert(tablesQuery.staticTexts["Item 4"].exists)
Let's add some assertions for the labels on the detail screens and tidy up the code, and this is our finished test.
func test_myFirstTest() {
// element queries
let app = XCUIApplication()
let mainTable = app.tables
let item1Cell = mainTable.staticTexts["Item 1"]
let item2Cell = mainTable.staticTexts["Item 2"]
let item3Cell = mainTable.staticTexts["Item 3"]
let item4Cell = mainTable.staticTexts["Item 4"]
let item1Label = app.staticTexts["1st Item"]
let item4Label = app.staticTexts["4th Item"]
let backButton = app.navigationBars["UIView"].buttons["My App"]
// test
app.launch()
// check cells
XCTAssert(item1Cell.exists)
XCTAssert(item2Cell.exists)
XCTAssert(item3Cell.exists)
XCTAssert(item4Cell.exists)
// navigate to detail
item1Cell.tap()
// check screen
XCTAssert(item1Label.exists)
// navigate to another detail
backButton.tap()
item4Cell.tap()
// check screen
XCTAssert(item4Label.exists)
}
How to architect your tests for future expansion
One thing you might notice in the test above is the large number of item queries we have at the start of the test. As we add more tests we'll likely want to re-use these, and probably add to them.
And the actions we perform navigating to and from screens, it seems there would be a lot of duplication throughout tests. The robot testing pattern is a great answer to creating reusable building blocks for UI tests that not only cuts down on repetition, but breaks down tests into steps that are easy to follow, even for those not familiar with reading code.
Types of UI testing
There are many different flavours of UI tests. Commonly we often refer to them all as 'UI Tests', but in reality they are different in important ways since they have different functions and different complexities. I think at this stage it's important to know the differences between types of UI test before getting started on your project. This way you can build tests with a specific use in mind, meaning more consistent, more reliable tests.
Stubbed UI tests
Stubbed UI tests have a similar principle to a well written unit test: extracting anything external to what is being tested. Things like network requests, user defaults, stored files and databases -- anything external to the app code itself. This is where I would advise starting out for any project that is new to UI tests. The stubbing part does mean there is a higher cost to setup these tests, but the resulting tests will be more reliable and failures easier to track down. Extracting any external dependencies allows you to isolate any failures to the app code. The exact scope for these tests can be up to you - you could keep the scope extremely narrow, similar to a unit test, and check an individual screen or perhaps a single component. But consider that an important part of UI is the way your user interacts with your screen. I would recommend tests somewhat larger in scope that navigate through a path, checking screens as they go. Almost as if it were an end-to-end test for a feature.
I won't tell you here exactly how to stub your external items, as that really depends on how your app is architected. But here are some options you could consider:
- Integrate a server into your app to return stubbed network responses.
- Setting up a server external to your app that returns stubbed responses controllable by sending network requests from your test code.
- Stubbing out certain code that provides network responses or other data in your app and controlling those stubs with launch arguments.
E2E tests
End to end tests allow you to check your app's full integration, not just your iOS code, but your servers’ code too. E2E tests really reach out to real servers that return real data. While it might sound as though there is no setup overhead for this you will need to ensure the data your app receives from any services you reach out to is reliable given the test. Perhaps you will need to provision certain account states, and you'll need to ensure any data stored within your app is consistent too. Depending on your app this could be significant and will likely need continuous maintenance that stubbing removes. As you're making real network requests these tests will also be the longest to run and the least reliable. So use them wisely. The benefit however is that these tests are as close as you can get to what your user will experience when they use your app.
Snapshot/Screenshot tests
Snapshot tests are great for preventing regressions. UI testing cares if information is shown on screen, but it doesn't care if that information is displayed in the right place, in the right order, or in the right colour. With snapshot testing your test navigates your app and takes a screenshot. If this screenshot differs from a known-good reference, then your test fails.
Screenshot testing is not a feature built into XCTest, but can be added using the SnapshotTesting library from Point Free.
Other applications
Xcode has the built in ability to assess your app’s performance including measures such as launch time, memory usage and animation performance. UI Tests assess your app's screen in the same way as assistive technology does, so any well written test is also an accessibility test, but if you want to go further with this the A11yUITests library can help.
Tips for UI testing
To help you continue to build your app's UI test coverage, there are a few hints that will likely help.
Disabling animations
Disabling animations does mean that the app you are testing won't be exactly the experience your customers will see, but it will still be very close, and the benefit is far faster and much more reliable UI tests. In your app code set UIView.setAnimationsEnabled(false). I'd recommend putting this in a compiler macro to enable this to be compiled out of your release builds.
Continuing after failures
Generally we would want tests to fail fast but at times, such as when you are writing your tests or debugging a failure, continuing after the failure is desirable. XCTestCase has the property continueAfterFailure. Set this to true to continue your test once a failure has been found, but I'd suggest in your pipeline test runs, keep this set to false. By default Apple sets this to true.
Waiting for elements
If you're waiting for an animation or for a network service to return before something is displayed on screen, then XCTAssert(item1Cell.exists) isn't going to work. Instead you can use expectations that are fulfilled once the item appears on the screen. Using these, our expectation for our item1Label, would look like the following:
let predicate = NSPredicate(format: "exists == true")
let expectation = XCTNSPredicateExpectation(predicate: predicate, object: item1Label)
let result = XCTWaiter.wait(for: [expectation], timeout: 5)
XCTAssertEqual(result, .completed)
Finding elements by identifier
In your app’s code every view has a, confusingly named, accessibilityIdentifier property. Setting this allows you to find views by this ID instead of searching for the whole text. This can make element queries faster and more reliable. And if your app is localised means your tests will work in any of your localised languages. If you do use this technique, remember to assert on your element’s label to ensure you’re presenting the string you think you are. For the example test code below we have set itemLabel.accessibilityIdentifier = "Item1" in our app code.
let item1Label = app.staticTexts["Item1"]
XCTAssertEqual(item1Label.label, "1st Item")
Build confidence in your app through testing
UI tests are a powerful way to ensure you are providing your users with the experience you think you are, all while avoiding time-consuming and repetitive manual testing. As with any project, there is a cost to starting, but Xcode helps make it as seamless as possible. I'm convinced once you have gotten this far you'll be sold on the benefits of UI testing. Your users, as well as your product owners, will thank you for it. And you'll have the confidence to make changes to your app, knowing that your tests will catch any issues before they become bugs.
Technology vector created by vectorjuice - www.freepik.com