🙋 Who Takes Part in QA at Stormotion and What are They Responsible For?
🔗 Steps of Quality Assurance Processes at Stormotion
🔒 What are the challenges in Quality Assurance?
⌛ How much time does QA usually take?
📈 How can you make Quality Assurance more effective as a stakeholder?
❓ What to do if a customer has their own QA Engineer on the Team?
*“The bitterness of poor quality remains long after the sweetness of meeting the schedule has been forgotten”, — Karl Wiegers, an American software engineer*
We believe that this quote hits home for almost every competent IT specialist, and Stormotion is not an exception at all. Obviously, it’s impossible to achieve high quality without testing the product and that’s where Quality Assurance comes to help.
Quality Assurance is a method to determine whether the software is matching the requirements of the stakeholders at the current stage, as well as to “polish” the product to the point of being as close to defect-free as possible. Although exhaustive testing is impossible, its conduction is hugely beneficial in terms of cost-efficiency, product quality, and customer satisfaction.
The clients are usually also interested in proper testing of their product. In general, QA Engineer finds and confirms around 80-90 bugs for a project of medium complexity. In bigger projects it’s rational to expect more than a hundred bugs appearing. Imagine releasing a product of such quality, keeping in mind that there are also smaller bugs that aren’t making it to documentation. We don’t think that such a project is going to fulfill the customer's business needs, so Quality Assurance is without a doubt vital for success.
Unfortunately, a completely bug-free project is an impossible task for anyone (image by Ana Beverin)
In general, the bugs found throughout the testing are not the result of developers being incompetent or them making wrong decisions. Usually, they appear in situations that can’t be predicted and are only seen after implementation. For example, it may turn out that some requirements are absolutely incompatible in a single project, or the level of required customization supposes configurational changes that may lead to unforeseen results. Also, we can’t forget about the bugs caused by the appearing technical debts and the team’s ways to handle them.
If you are interested in Stormotion’s way of solving technical debt issues and some general tips from our experts, please make sure to check the article down below.
How to Deal With Technical Debt?
In this article, we’ll try to get acquainted with Quality Assurance in general, its processes, challenges, participants, and also open the curtains of Stormotion’s inner QA flows along with sharing our best practices from our experience. Let’s get to the point!
The main specialist in Quality Assurance in our team is without a doubt our QA engineer. However, there are people who usually also contribute to the process, leaving no place for bugs to escape 😼
So let's acquaint ourselves with everyone responsible for the Quality Assurance stage at Stormotion!
Developers. Our developers responsible for the project are the ones who start the “testing cycle”. Their task is to provide our QA engineer with a crash-free product version after conducting Smoke testing.
Smoke Testing is a software testing stage usually conducted by the developers to approve the product version’s readiness for the QA. Typically it’s done to check for the installation and database connectivity errors by running small tests.
Project Manager. In our team, the QA cycle is a so-called safety net. In case our developers are unable to conduct Smoke Testing, our PM is the one to provide the crash-free build — code pieces compiled into a program which is executable yet remains unreleased — to the QA engineer. Also after our QA specialist fills in the documentation, the PM always needs to verify it.
Although only QA specialists are usually called “testers” they’re not the only ones responsible for the project’s testings (image by Ana Beverin)
**Designers. Even though their role isn’t directly associated with the QA cycle, the design they create needs to be maximally convenient for the user. **To achieve the best UX, they should conduct usability testing in the process of creation, before delivering the app to the QA.
QA Engineer. The person whose role is to make sure that the delivered software build is of the highest quality. In brief, their responsibilities include:
Sanity Testing is a way to test a build’s sanity before taking the next step into more advanced testing. For example, it makes no sense to test complex features unless you are sure that every basic one is working correctly.
A common misconception is to think that Quality Assurance (or it’s also called “Software Testing”) is a process described as a single activity. Vice versa, QA is a set of activities and methods that aim to complete the process of validating the product’s stability and verifying its usability. As Software Testing is a complex process, it’s usually referred to as Software Testing Life Cycle (STLC) for specification.
In this section, we’re about to present Stormotion’s internal processes for STLC, taking our Norsk Guardian project as an example.
Norsk is an IoT companion app that our team managed to develop in about 30 weeks. Based in the USA, Norsk Lithium develops and sells batteries for outdoor activities including open-water fishing, ice fishing, and camping. In this project, the development itself along with the testing stage took about 18 weeks.
The Software Testing Life Cycle at Stormotion consists of 5 phases instead of the common 6, as in our team the Environment Setup Stage is not included as a separate one. Here are the phases of STLC our QA Engineer always goes through:
_Although Quality Assurance is a parallel process with the project development itself, it doesn’t usually start right away. The best approach is to involve a QA engineer as early as possible to gather more useful information on the project’s planning stages. _
Let’s take a look at each of them separately!
Requirement Analysis — also known as Requirement Phase Testing — is a stage when the project requirements are reviewed through the prism of testing. This phase helps our team to understand testable requirements as well as to determine automation feasibility for testing.
Automation Feasibility is a process of determining test cases that may be automated based on the given functional requirements and availability.
As Norsk Guardian is a project more inclined to manual testing as some main features such as Bluetooth can’t be automated. Thus, after discussing with the team, we decided to save time by sticking up to manual testing instead of an automated one.
Common activities for the Requirement Analysis stage are:
One of the Norsk user stories stated that users should be offered to set up a notification reaching a custom charge level. When verifying the backlog and user stories, the question was where to locate the required notification block in the UI.
Shot from the Norsk Guardian app
A "Set Custom Charging Level Notification" button should be found after proceeding to the "More Options" section in client’s "Batteries Group". Its location between the “Delete Boat” and the “Rename Boat” features is necessary in order to perform regression tests in a row. A QA engineer doesn't need to move to another section and spend extra time on these actions.
Test Planning is a stage for QA specialists to determine the strategy for a test plan, testing schedule, cost estimates, test environment, and limitations. All the risks should be also determined as well as The Test Plan should be finished by the end of this phase.
Test plannings define the main methodologies and estimates that a QA Engineer will use throughout the tests (image by Outline2Design)
Common activities for the Test Planning stage are:
Among the testing tools for Norsk Guardian our QA Engineer used Slack, Google Sheets, Active Collab, TestFlight, Figma, Android Studio, and others.
On the Norsk Guardian project, we identified the main functionality, which we would cover through tests. Sometimes, in the case of a low functionality project, we would cover by tests 70-80% of all functions, but on large projects, the test cases cover only 20-30%.
Test Case Development stage is about creating new and modifying already existing test cases and scripts after finishing the Test Plan.
A test case example
Test Case is a testing artifact that contains a set of actions and/or conditions required to test the functionality of the software part
Common Activities for Test Case Development:
At Stormotion the RTM matrix is used to verify that all our requirements are covered by test cases. This document we present along with the Master Test Plan to the stakeholders.
Test Execution supposes exactly the process of test script implementation and bug reporting after the test plan is finished. The bugs reported to a development team are getting fixed for the further iteration of test execution.
Common activities for Test Cycle Closure are:
On the Norsk Guardian project, our QA decided to include localization testing as the project will focus not only on the English-speaking market.
Regression Testing is a testing method used to determine whether the new code modifications have affected the results of previous tests. Usually, it makes QA engineers retest new builds using previous successfully implemented iterations.
For convenience, the document for QA logs was created to track the time of verifying the version and understanding whether the build is stable. The report should include the build version, a number of issues, fixed bugs, and the list of “Open” bugs.
Test cycle closure is a phase where obviously the testing cycle and its execution are coming to an end. With filling the test completion reports, this is when the QA team gathers and analyzes information regarding the processes of this project. After the discussion, it becomes clear what aspects weren’t as good as they could be and what to do in the next projects to enhance efficiency.
Test Cycle Closure is the final part in a QA cycle consisted mostly of evaluations reports and analysis of the conducted testings (image by KiraSvitlana)
Common activities for Test Cycle Closure are:
Generally, customers that aren’t deeply aware of the inner flows of QA, tend to underestimate its importance and the amount of work threat the QA Engineer is facing. Such misconceptions usually lead to making it harder for the specialist to give their best results.
We'd like to share the main challenges a QA Engineer may face during Quality Assurance based on Stormotion’s experience:
Frequent changes in customer’s project requirements make a Team rewrite documentation and choose a different testing approach.
We recommend you to spend more time thoroughly understanding and prioritizing your requirements before the development starts. It may not deflect all the needs in their changing in the future, but will definitely make them less crucial for the Team.
Clients require the highest product quality without willing to spend more hours on testing it.
The quality of your product has its own price. Thus, be ready to allocate extra resources if you have high technical requirements.
Cases, when Quality Assurance estimates are not high enough to stick to without losing in a single aspect. Thus, QA specialists have to test in a fuss to keep up, increasing the risks of missing out on something.
To avoid such challenges it’s better to structure your budget with a complete understanding of the QA level you require as a stakeholder of the project. As well as to be ready for having to decide the need of raising the estimates during the development.
Projects, where stakeholders have no clue about the importance and basics of the Quality Assurance stage.
As you are already acquainting yourself through reading this article, this challenge is less likely to hit your Team. However, keep in mind that the more you understand this topic, the easier it’ll be to establish great communication with your Tech Partner and achieve your goals.
Vacations or delays from either a client or Team members. As Quality Assurance requires developers and QA to work in synergy, every gear in this mechanism is extremely important for productivity. And vice versa, in case the Team needs any approvals while the customer is unavailable — the development and testing may have to be stopped.
To avoid this challenge’s appearance we recommend you to be more precise in sharing your working schedule with a Team. The earlier you notify about your vacations or days off the easier it’ll be to adjust the testing cycles for the times you are present.
The main challenge for a QA Engineer is to reach the closest point to the ideal product version after Quality Assurance. As the testing principle states that an absolutely bug-free project is utopian, QA’s task is to get it to the highest possible level.
As every single project we build is a unique experience, there is no precise answer to this question. In general, the QA stage starts almost from the very beginning of development. And till the product release Quality Assurance becomes its inseparable part.
There are always risks that are not present in every single case that, however, should be considered. The time required for the QA stage depends on the unique methodology our QA engineer chooses before the conduction. This methodology includes an exclusive Testing Plan based on the project type, its risks, and the customer’s behavior as a stakeholder. If a client is constantly making changes in his requirements for the project, they are also making its life cycle significantly longer.
Based on our experience at Stormotion, the QA cycle usually takes up to 20-30% of the whole project’s lifecycle, making it quite a noticeable part of the whole development in the bigger picture.
We mentioned in previous sections that stakeholder behavior requires consideration while creating Testing Plans for the product. But what should a customer consider to make the QA effective and time-saving?
You as a stakeholder can also assist your Tech Team in raising your product’s quality (image by Anna Magura)
Here are some tips from our team on how to achieve a maximum of QA productivity:
There are cases when customers already have the QA engineer in their team and are willing to take control of the process. Generally, we see three potential scenarios of how it can be done:
This is the most common scenario from our experience. Usually, a client doesn’t have any objection to letting our QA engineer do their job. We highly recommend entrusting the Quality Assurance to the team responsible for the development, as it is more convenient in such ways:
This scenario is certainly less common and the risks it brings compared with the first case clearly show why. Letting a customer’s side to conduct a Quality Assurance stage usually implies such risks:
It may sound like nonsense, however, unlike the previous one, this type of QA stage even has a way to be efficient. This case may be used in larger projects, where the scope of work is noticeably bigger. With the compromised QA conduction on both sides of the development, it becomes possible to find bugs and problems way faster.
However, the risks are not leaving anywhere, increasing the estimates and taking way more time for QA engineers to adapt to paired testing.
Although, the combined QA conduction is also a real case, we still recommend to entrust its processes to your Tech Team (image by Jourdan Rahschulte)
Obviously, the less risky and more productive QA is always when it is inseparable from the Tech Team. Therefore, we recommend not to reinvent the wheel, wasting time trying to make it efficient, and completely trust the Team you are hiring in every aspect.
In conclusion, we’d like to emphasize one more time that the Quality Assurance stage, despite being assumed as an easy process, is quite complex and significant for the software quality. At Stormotion almost every team member takes part in Quality Assurance in one way or another. Developers, designers, project managers and of course QA engineers are the ones responsible for creating a defect-free product.
It’s also important to understand the inner flows of the Software Testing Life Cycle to take a deeper look at how the process of polishing is taking place. At Stormotion Quality Assurance supposes the conduction of such phases:
It’s also possible for you as a customer to discuss the involvement of your own QA specialist, however, after the comparison of such scenarios we highly recommend letting the Team take care of it to avoid extra risks.
If you have any questions or need help with creating quality software for your IoT solution let us know. We will be happy to help you meet your business needs with an outstanding product!
Was it helpful?
All About Project Development at Stormotion in 3 minutes
How Much Does It Cost to Develop IoT Software?
The Stormotion Team: What Makes Us Special?
Our clients say
They understand what it takes to be a great service provider, prioritizing our success over money. I think their approach to addressing ambiguity is their biggest strength. It definitely sets them apart from other remote developers.
Max Scheidlock, Product Manager