Editor’s Note: In the spirit of the famous advice columns of the past, one of our Tosca Testsuite experts has started their our own column: Dear Dr. Tosca. Have questions for Dr. Tosca? Send them our way in the comments!
Dear Dr. Tosca,
I am part of a large Dev team that is currently developing the next big IoT mobile app. It’s a really young and dynamic team, so the development is moving along really fast, but we just aren’t able to keep up with the testing. We have tried virtualization, but it seems like every time we fix a bug or add a new feature, the synchronization just breaks our tests, making us start from the beginning.
Is there a way of simulating all of the parts of the app that are missing as well as third party services, without ending up with broken test cases?
Yours sincerely,
Frustrated Developer
Dear Frustrated Developer,
Thank you for your question and congratulations for using Service Virtualization – you are definitely on the right path! But I can also see that you are missing some important pieces on the road to success so let me start right from the beginning.
As you say you are a young and dynamic team, let me guess – you are working in an agile approach, having developers and testers in one team?
The best approach is to set up GUI and (depending on your APP architecture) API tests in parallel to your development progress. This lets your developer and testers work closely together. If a task is technically demanding the developers can give their input, while your testers optimize the test execution and variants to cover your app’s full functionality.
I assume as well that you are using a code versioning system combined with a build server. The idea is that any time a colleague checks in a code change, your build server will compile the source code, deploy your app to a test environment, and trigger the test case execution of the test cases against your application. This is an ideal setup – fully automatic and running in the background, while always giving an accurate overview of your application functionality.
Most likely your mobile Application will have a lot of service calls – have you decided to use REST? In fact, the technique or format does not matter at this point, what matters is how you deal with your system dependencies in development and test. Service Virtualization is the best option to release all the dependencies to any 3rd party systems your mobile app relies on. For testing purposes you can focus on your app, while all service calls are handled by Service Virtualization. Of course these virtualized services will also handle all your test variants, including complex scenarios like adding, modifying or deleting data (i.e.: creating a new customer login, adding or redeeming bonus points, etc.), and will be setup in full dimension by your testers.
The missing link you described has a name: Combined test data management. You could update your virtualized services based on real underlying production data – but be careful! Do your test cases in the GUI and API reflect these changes? No! And this is the reason why your test cases break.
The solution is to use combined test data for executing GUI- and API-test cases as well as for the virtualization. You can use a combined test data set, including all relevant test data for the full test & risk coverage.
Imagine you run a test case in your GUI with the login “drtocsa”. You will use the same data field for the virtualization request – inbound and outbound – so of course the data fits. Any enumerations or mapped code fields will be included in your test data sheet. You will not even have to bother with IDs in foreign systems – with your combined test data even IDs can be handled. And the good news: Your total effort for any test data related issue will drop by more than 60%!
So again, congratulations for using Service Virtualization. The missing link lies in not using combined test data management to cover GUI, API- and Virtualization. Change this and you will stay on-deadline instead of doing nightshifts…
Yours Sincerely,
Dr. Tosca
Want to learn more about Service Virtualization?