“NewTest” error/exception happens when there is no existing baseline found with such combination of OS, Application name, Test name and Window-size, and therefore, a new baseline is created.
We should distinguish between two situations:
● The desired behavior - I either created a a new test or decided to run an existing test on a new environment (browser/device/screen size). In this case the desired behavior is creation of a new baseline for this environment.
Please note that the predicted exception can be avoided if desired: You can add "false" in eyes.close to avoid the exception. Click here to review additional advanced test results details.
● The undesired behavior - I ran the same test again without making any notable changes, but for some reason this run resulted in a new baseline. A common reason for this issue is in case the viewport size is not defined explicitly in the eyes.open call, when the test runs on a different machine or on the same machine with a different browser window size, the new run will result in a new baseline. In order to prevent this issue, it is recommended to explicitly set the desired viewport size in eyes.open.
To check how to set the viewport size, you can check this video.
Other reasons for this behavior can be that other params of OS, Application-name or Test-name changed and thus result in creation of a new baseline. An important tip regarding baseline creation is that in eyes-selenium SDKs it's recommended to use the eyes.open overload which takes a viewportSize and set it to a desired size. In such case our SDK will take care of resizing the browser to adjust the viewport size to the specified size and ensure that the results are consistent even if the test is run on different machines/screen sizes.