A baseline is defined by 5 parameters as can be seen in this article:
1) Application name
2) Test name
3) OS
4) Viewport Size
5) Browser
The first two are the test parameters, and parameters 3-5 are the environment parameters.
When running a test, Applitools Eyes checks if there is an existing baseline. If there is, then it compares the test to it, and if not it creates one.
A baseline is always saved within its environment (combination of OS, Viewport Size and Browser).
There is an option to compare tests across environments - for that you set a Baseline Name.
Setting a Baseline name allows you to compare and validate your application across different OS, Browsers and Viewport sizes.In order to run test across environments, we will need to define a Baseline Name, that will link between the different environments, using the following syntax:
Java
eyes.setBaselineEnvName("BaselineName");
eyes.baseline_name="BaselineName"
eyes.baseline_name='BaselineName'
eyes.setBaselineName("BaselineName");
C#
eyes.BaselineName = "BaselineName";
When running a test for the first time with a given Baseline Name, the Baseline Name "points" to the environment of that test, and any future tests that will run with this Baseline Name will be compared to the corresponding test within the environment to which the Baseline Name is "pointing" to.
Accepting and saving in cross environment test, changes the environment to which the Baseline Name is pointing to the new environment, and overrides the baseline of that test in the new environment.
Note: When specifying a Baseline Name, a test (defined by an App Name and a Test Name) will always be compared to its corresponding baseline in the last environment in which it was updated.
The following example demonstrates a common scenario for cross browser testing:
After setting our Baseline name to "BaselineName" we ran the test in our chosen environment (in this example we used Mac as OS, Chrome as browser, 700x700 as Viewport size), and the following baseline was created:
After setting our Baseline, we ran the test again, but this time on Firefox instead of Chrome:
As you can see, after replacing the browser, which is one of the Baseline's parameters, the new test was still be compared to our existing Chrome Baseline, as a result of running it with the same Baseline Name.
Accepting and Saving this test will not override the original Chrome Baseline, but will create a new baseline defined by the Firefox browser, and set the Baseline Name to point to the new environment.
The following test we ran, was a test with Chrome again, but this time with a different image (to emphasize it's a new test):
As you can see the test was compared to the last environment in which the baseline was updated (Firefox).
Next, we accepted and saved the last baseline which saves the results only in the Chrome Baseline and not in the Firefox Baseline. To demonstrate it, we ran another test, this time WITHOUT setting a Baseline Name, using Firefox.
As you can see, the original Firefox Baseline wasn't changed due to the last accept and save.
The last test we ran show how the last accept and save, updated the baseline of the Chrome environment.
This test was ranwith a different image (only to emphasize that it's a new test):
To sum up:
BaselineName is used for Cross environment testing.
Test results can only override baselines in their environment.