View Full Version : Distributed Development, Testing and UAT

04-13-2018, 08:41 PM
I work as QC staff.
I've watched your presentation "Architecture of Testing for Complexity, A Managerís Approach" and really appreciate it.

However, I have some problem which I believe your knowledge and experience could help me.

In my company, we have development team from 3rd party vendor and testing team from another 3rd party vendor(it may be strange but, yes, my organization chose this model). These 2 vendor don't work together. And because of the contract, we have a straight schedule of development and testing like traditional waterfall model which is :
Dev(1st vendor) -> Testing(2nd vendor) -> UAT(my company's testing team).
I and my testing team has to perform a testing activity on UAT to verify the result of development and testing from vendor(the testing activity might be duplicated but we have to simulate ourselves to be a business user to test on UAT instead).

The problem is, I've just seen the schedule and I think it's quite late to have any feedback from my company(UAT phase) and start with development again to fix it. But I can't change the process or have a daily meeting with development(1st vendor) either.
Fortunately, I still have a rights to take participate in testing activity of Testing team(2nd vendor).
So I would like to make this opportunity to prevent too late feedback. But I'm not sure how and when I can improve the testing quality and time.

Could you please share your idea on how to manage this Testing vendor to prevent the problem(about quality and time)as much as possible before it's coming to UAT?

04-14-2018, 11:35 PM
This is truly between the rock and the hard place. I have a few questions and then some thoughts.

Are you able to see the fault or defect reports from the test group as the defects are discovered? Can the developers see these defects as they are reported also?
Is there another way to get testing feedback to the development staff as the developers are working, or is the development considered ďdoneĒ after it is delivered to the next supplier to perform testing? That should not be the case, but I do not see the plan or the business agreements. Even if you UAT testing is too late to give feedback to the developers, it could be possible to have the testing supplier to provide some feedback via their fault reports (ideally) directly to the developers.
Can or does UAT and supplier testing happen simultaneously? That would free up some time at the end but comes with some extra work in coordinating fault reports (no duplicates)
Why can you not discuss with the vendor? Can you discuss daily with testing supplier? It could help some to have the developers have these daily discussions once an iteration is delivered for the testers to start their work.

I donít suppose there is any chance to adjust the schedule? In truth, you may end up adjusting the schedule anyway if the product has sufficient defects that will necessitate moving the introduction date. If this comes to a re-plan Ė I would suggest breaking the deliveries down into increments to go through the process and perhaps have testing and UAT happen congruently but that collective feedback would need to be coordinated (no duplication of fault reports sort of thing).

04-14-2018, 11:41 PM
Yes, I have a permission to acess to see their raised defect system and I requested them to report us a progress daily.
The testing activity still not yet start. And I have a request that vendor testing staff should be working in our work space so that we can have well communication(right now, vendor developer staff are working in their office). When testing activity start, we will communicate them daily. Actually, I have also requested that there should be some developers to work on our office during testing phase as well and I hope this will be approved(but it is too late to make them relocate on this development phase).
We cannot let testing vendor staff to start their testing before developing vendor has done their job. When dev finished their coding, there will be some of developers remain for a maintenance phase(which is to fix bug from testing vendor staff).
Actually, I (as a user) will also take participate in this testing phase instead of waiting for UAT date. And then when it come to UAT, there shouldn't be much defect left on that phase. I hope.
4. Yes, I will try as much as possible to make a good communication with vendor tester. I don't have much influence with developer vendor team but hope that my request about developers who are standby on testing phase to fix bug will relocate to work with us so that we can at least have a full team face to face communication on some period of time.

P.S. I'm new in this current company, around 1 week joined. However, during this 1 week, I've noticed some communication problem between these 3 companies and I would like to improve it. However, in the worst case, if I cannot change the way to communication, I would like to have a backup plan to prevent project delayed and/or quality drop.

Therefore, I would ask your advise so that I can improve a testing activity.

04-14-2018, 11:51 PM
I would also make sure you have a common view of the severity scale for the fault reporting work. This can be by class (such as A, B, C) or by points, but there must be a way to articulate the severity of the defects found, whether you find them in UAT or the defects are found in testing. This will allow for prioritization of those defects most damaging given the end of UAT and the start of production has little gap for rework, retest, and another UAT. This may come in handy when you are near the end of the schedule and as you have noted, there is insufficient time to feedback, correct and test again another iteration.
I understand, there is one development release to the test entity. This is less than optimum the better way is to have small increments that go through testing with the results feedback to the developers, incremental and iterative. This one pass makes determining the quality difficult, however, you can look at the number and types of defects found to glean the level of quality (volume and severity - for example if you run 10 test cases, and find 3 faults, that may provide you with some way to assess the product quality). Many defects and high severity defects will tell you something about that particular iteration anyway. I would keep track of the arrival rate and the percent of testing completed (based on requirements perhaps).
Good that you are included in the testing phase, that makes number 2 above look very interesting.
What is needed is good communication between you and the other two vendors (testing and development). Things found in testing, prioritized and severity considered, need to be sent back to the developers as quickly as possible so that the little time remaining the defects can be corrected. It is possible to find defects in testing before the UAT begins or even as UAT is underway. If possible, it would be good to have daily meetings with the developers, testers, and the UAT to discuss what was found and the severity. This meeting should have a fixed and very short time sort of like at daily sprint meeting. The testers show what was found, the UAT folks will likely have a better feel for the nature of the test failure, that is the impact on the end user or customer. In that way they may be in a better place to assess the severity of the defect. I hope you have a tool to manage the defects found, that includes the important information required to solve and track between all parties.

Prioritize the portions of the project scope that are the most critical, that is, what parts of the product, if there were a defect, would cause the most damage to the customer (safety, security things like that) and do that testing and UAT work first. That also gives what little time there may be remaining before launch to be used to solve any critical problems.

Does this help?