Should you test the same software build if only the data has changed?

Question

We have a .net web based software application that is a user guide for popular world cities. The application is installed on a standalone device (surface tablet) The Software can handle multiple types of data about different cities dependant on what 'city' xml file is in the system. The customer e.g Canadian tourism only has 1 city loaded into the application at a time on their tablet.

So the software remains constant, but the data changes depending on what country is using the software.

Currently the baseline software build is tested (manual smoke testing and automated regression) against a set of data e.g. Canadian Tourism v1.2.0.123 Dataset v1.0. My question is that if Germany Tourism, UK Tourism, India Tourism now want the application does their release build need testing with new data if the baseline build is passing tests and working fine? Obviously I want to cut down on testing with this amount of continuous deployment, but my instinct is that the application must still be tested (full regression?) before release with the new data, even if the underlying software is the same and 'should' be working fine.

Hope this makes sense!

Thank you.

Answer 1

You must test before releasing it to new clients. See the below scenarios to cover:

1. You have installed and done testing on Surface tablet. But if release for new client it can be in another device with a different configuration than the one you have used to test. So, go for compatibility testing in different versions.
2. As the data changes for new client, you must test the data. Check for the images which are displayed correctly or not, if any.
3. Check for Optimized page loading because if pages are loading with big size images, pages will be lazy to load.
4. Use test data like 'lorem ipsum' and do some Volume testing on data.

You must go through all the pages at least one time even if there are no updates for new release because there will have design(UI) issues as the device of the end user may have changes in version or configuration.

Also, check for the spelling mistake in real data just after released to the client, if needed.

Answer 2

In my opinion, if the data has the same parameter, we only need to do smoke testing or even a quick functional verification.

Unless, the tourism data from different country has different the data size then yes, I agree with Faizal to do Load Testing. Or if they will use different devices, then it would be better to do compatibility testing.