In Visual Testing, highlighted differences can be the result of either intended change in the UI of the application, which is a natural evolution of the product, or as a result of a bug. The first type of differences is handled by updating the baseline with new captured screenshots. If the differences are a result of a UI bug, there are two use cases that should be addressed:
a) In addition to the highlighted UI bug, there is also an intended change that should be included in the baseline for future test runs. In this case, ideally, we would like to have the option to accept the new screenshot as the baseline and still have that bug fail our future tests.
b) The UI bug is the only highlighted difference and is a known bug, which is currently being worked on. This situation causes unnecessary time consumption of testers, who are required to manually validate the failed Applitools results due to that bug. In this case, ideally, we would like to have a "snooze" button on the failure caused by the bug, which could be set until the estimated time for fixing that bug.
In order to handle the aforementioned situations, Applitools added the option to add a Bug Region in the Test Result Editor.
Bug Region enables marking regions as bugs, addressing the two scenarios above, and allowing collaboration between all relevant team members (testers, developers, product managers, etc.).
How to add a Bug Region:
1. Click on the exclamation mark.
2. Drag the mouse over the region of the bug to select it, as seen in the figure below:
3. Insert a title for this region, describing the bug.
4. Click the "Create" button.
As can be seen in the figure above, all collaboration regarding the bug, between the different team members, appears in the chat box.
5. In the first scenario (a) in which in addition to the highlighted UI bug there is also an intended change, adding a bug region and checking the option 'Fail tests' in the checkbox prior to accepting the baseline will update the new screenshot as the baseline image. It will also cause all future runs to fail until the bug region is removed or the "Fail tests" checkbox is unchecked.
In the second scenario (b) in which the UI bug is the only highlighted difference and is a known bug, adding a bug region, checking the Fail tests and setting the Snooze option, as seen below, will not cause future tests to fail due to that bug until the selected date. (Other visual discrepancies may still fail the test.)
Note that when checking the failed checkbox, the test will fail only if there are no differences between the baseline and the checkpoint. Therefore, if there is a known bug, you should:
1) Accept the checkpoint to be the new baseline which will include the bug
2) Add a bug region with the failed checkbox enabled.
3) Additional issues that come up result in an unresolved test result, in order to make them easily noticeable and prompt a review.
Removing, copying, and sharing the bug region is done using the three icons on the top right corner of the region.
In the dashboard page, filtering Batches containing Bug Regions or Remark Regions can be done using the "steps annotation" view mode as shown below.