When integrating Applitools Eyes with your test environment, Applitools Eyes can trigger a few types of errors/exceptions. Let’s review the possible results and explain each one of them:
● NewTestError/NewTestException - When a test runs for the first time in an environment, the test does not have an existing baseline to compare, and the initial run will be used to create a baseline (expected result) for future runs of this test in the same environment. In this case, the test will be saved automatically and no action is required from the operator, there is a link to the test that mentions the exception/error.
● TestFailedError/TestFailedException - There are 2 main reasons for this error:
- The failure was caused by a real difference between the baseline and the actual results - in this case you need to follow the link to the test result (the link is included in the exception details), review the change in Applitools Eyes web application, and approve or reject the change and save it.
- If new steps were added or removed from the test, that will also result in a failed test. In this case you also need to follow the link to the test result (the link is included in the exception/error details), review the new steps (or missing steps) in Applitools Eyes web application. For new or missing steps, in case these steps were added/removed on purpose, you should accept these steps and click on save.