Lesson 7: QA and Testing
Creating apps is only the first part of getting them into user’s hands and the market. Before that, apps need to go through a testing process on their way to deployment. The goal of this process is not only to ensure that the app is working as expected (happy-happy scenario), but also to check if it will still work in some less predictable scenarios. No matter how thorough this process, things will always get through, and then it is a matter of troubleshooting and figuring out what is happening in order to debug.
Unit, Feature and Regression Testing
There are different types of testing depending on the need and stage of development:
Unit:
Testing only a small block of code to ensure it works properly
Test behaviours of the application and not just functions
For example check if some variable was initialized or check if the controller has loaded all users in a certain time or stage
Both Xcode and Android Studio have tools to handle unit tests
Should be done constantly as you develop
Is the basis for Test Driven Development methods
Feature:
Testing all of the different functions of a given feature and anything it could affect
Usually done manually
Performed when the completed feature is live (usually in a staging environment)
Often the type of testing that is done either by the entire development team, or can be dedicated to one or a group of developers
This is the type of testing that many companies will have new developers start with in order to gain knowledge of the application functions
Regression:
Test all features/functions of an application after a change
Some or all of the regression test can be automated
If automated, can be performed every time there is an update
Should always be done after a major change to the application or data model
Main goal is to make sure recent additions/changes do not break anything else in the app that may seem unrelated
Local, Staging and Production Environments
Most modern development teams will have multiple environments (as outlined in the Version Control lesson) which play a role in the testing and deployment process. These environments usually consist of a local environment on the developer's machine, staging server and production server, and features will need to pass through all of the environments, being tested along the way, before getting to an end user.
Local:
Complete the code on your local machine first
Should be doing some form of unit testing on each feature/object as you develop them
Test all functions and scenarios of the completed feature
Code should only be pushed to staging server if it passes all tests locally
Staging:
Fully functioning application with access limited to invited testers
Testers could only be internal to the company or could be a group of end users with 'Beta' privileges
Should perform a full test of new features and a regression test of the application to make sure new features do not affect old ones
Once the application passes a full regression or feature test, the code can be deployed to Production (usually through a merge of the development/staging branch with the Master)
Production:
Fully functioning app that is live to end users
For mobile, this is the app that is currently in the store
Errors still occur from unexpected user behaviours or other things not caught in testing
These errors are reported by users some times or caught by different tools (Sentry, Crashlytics, etc)
Tickets are created for current bugs, which are either fixed through the same process and included in the next deployment (typical) or else or 'hot fixed' if it is a major issue affecting user experience, where the fixed code is pushed directly to Production
Troubleshooting
Whether it is a bug that makes it through the testing process, or some other issue a user is having, it is important to understand how to properly troubleshoot in order to fix it and determine the cause. If it is not a bug in the code itself, it could be a combination or one of: unexpected user behaviour, user or network error, or version cache or storage issues (like a mobile device being full). It is important to ask the right questions and be able to give the user some basic steps that could solve some of these other problems. There are also many tools you can include in your application (and built into browsers) that help catch errors and give you information about their cause, allowing you to be more proactive than reactive at fixing issues.
Questions
When speaking with a user, a couple basic questions can help provide a better picture of the issue:
What device or you using?
What screen in the application is the issue occurring on?
What is happening when the issue is triggered?
Troubleshoot
While you diagnose and debug the issue, there are a few basic things you can ask the user to do in order to help fix some of the more user related issues:
Close and reopen the application
Sign out and sign in
Delete and reinstall
Console Messages
For web applications the browser provides developer tools that can help you diagnose issues and figure out what errors are occurring.
Tools
Preparing Tickets
Description
User info
Steps to recreate
Screenshots/captures
Console
Assignee and severity
Resolving
Last updated