This document describes how you can run a test that implements Applitools Eyes visual testing from within the VSTS build environment, and then see the results in the VSTS Build result window.
The screenshot below shows a VSTS Built results window after the user has clicked the "Applitools" tab. Note that you only see current batch (the batch related to the current VSTS buid). If you enter the step or test viewers then they will open in a new browser panel in the usual way. From there you can also navigate to the full Test Manager window.
Preparing your Eyes code
To run Eyes enabled code in the VSTS environment you it in the same way as you do when not in the VSTS environment. The only thing you need to do is to explicitly create a batch for your test and set its Id from an environment variable as shown in the following code :
Eyes eyes = new Eyes(); String apiKey = System.getenv("APPLITOOLS_API_KEY"); eyes.setApiKey(apiKey); // obtain the batch name and ID from the environment variables String batchName = System.getenv("APPLITOOLS_BATCH_NAME"); String batchId = System.getenv("APPLITOOLS_BATCH_ID"); // set the batch BatchInfo batchInfo = new BatchInfo(batchName); batchInfo.setId(batchId); eyes.setBatch(batchInfo);
Var eyes = new Eyes(); apiKey = process.env.APPLITOOLS_API_KEY; eyes.setApiKey(apikey); // obtain the batch name and ID from the environment variables var batchName = process.env.APPLITOOLS_BATCH_NAME; var batchID = process.env.APPLITOOLS_BATCH_ID; // set the batch eyes.setBatch(batchName,batchID,0);
eyes = Applitools::Selenium::Eyes.new() api_key = ENV['APPLITOOLS_API_KEY'] eyes.api_key = api_key # obtain the batch name and ID from the environment variables batch_name = ENV['APPLITOOLS_BATCH_NAME'] batch_id = ENV['APPLITOOLS_BATCH_ID'] # set the batch batch_info = Applitools::BatchInfo.new(batch_name) batch_info.id = batch_id eyes.batch = batch_info
eyes = Eyes() api_key = os.environ.get('APPLITOOLS_API_KEY') eyes.api_key = api_key # obtain the batch name and ID from the environment variables batch_name = os.environ.get('APPLITOOLS_BATCH_NAME') batch_id = os.environ.get('APPLITOOLS_BATCH_ID') # set the batch batch_info = BatchInfo('Batch By ID python') batch_info.id = batch_id eyes.batch = batch_info
var eyes = new Eyes(); eyes.ApiKey = Environment.GetEnvironmentVariable("APPLITOOLS_API_KEY"); // obtain the batch name and ID from the environment variables var batchName = Environment.GetEnvironmentVariable("APPLITOOLS_BATCH_NAME"); var batchId = Environment.GetEnvironmentVariable("APPLITOOLS_BATCH_ID"); // set the batch var batchInfo = new Applitools.BatchInfo(batchName); batchInfo.Id = batchId eyes.Batch = batchInfo;
$eyes = new Eyes(); $apiKey = getenv ('APPLITOOLS_API_KEY'); $eyes->setApiKey($apiKey); // obtain the batch name and ID from the environment variables $batchName = getenv ('APPLITOOLS_BATCH_NAME') $batchId = getenv ('APPLITOOLS_BATCH_ID') // set the batch $batchInfo = new BatchInfo(batchName); $batchInfo->setId(bartchId) $eyes->setBatch($batchInfo);
The batch name defined by the Eyes Extension is composed of the team project and the build number as defined in the VSTS environment.
Note that the
APPLITOOLS_API_KEY is an environment variable that you need to define and set as explained in the next section, whereas
APPLITOOLS_BATCH_ID are environment variables defined and set by the Eyes SDK VSTS should not define the batch related variables in your operating system.
Installing the Eyes task
The following description assumes that you already have VSTS installed on your system.
- The Eyes extension needs access to your Eyes API key. It obtains the value of the key from an environment variable . You can set the environment key externally via your operating system, or you can do it internally in VSTS in the "Build Definitions => [some build definition] => Variables" tab. To obtain your API key see How to obtain your API key.
- Add the Applitools Eyes build task. In VSTS, click on the "Build and Release" tab, then click on the build definition to which you want to add the Eyes .
- Click on the edit button
- Click on Add Task
- Click on "Check out our Marketplace".
- Type "applitools", click the search icon and you should see the Applitools Eyes task. Click on the Applitools Eyes task.
- Click on the install button.
- Wait a bit....
- and then press "Confirm"
- wait a little bit more
- Click the "Close" button
- Refresh the browser
- Click "Add Task"
- In the search box start typing applitools, when the Applitools Build Task appears, put your cursor over it until the "Add" button is displayed and then click the button.
- The Applitools task appears at the end of the list of tasks.
- Drag it up so that it is before the "Build Solution".
- Click "Save & Queue", click "Save", add a comment in the popup and click "Save".
Running your tests
You run your test in VSTS in the usual way :
- click the "Queue" button
- click the "Queue" button on the popup
- Click on the build number of the current job:
- This will the VSTS Build results window. Click the Applitools tab, and then, when the test is complete you will see the main pane of the Applitools Test manager with the results of the current build loaded.
You will see the results for the current job. To see the results of other builds you can select them from the VSTS "Recently Completed" list. You can also enter full Test manager mode where you can see all results of all the tests runs. To do this do as follows:
- Click any test to see the steps of that test.
- Click on the step thumbnail to enter the test editor.
- If you click the button the display will return to VSTS. If you click the Applitools logo the display will show the main Test Manager panel where you can see all the runs.