Articles in this section

See more

Match Levels

Avatar
Yarden Naveh
  • Updated

The next article will explain how to adjust your test's Match Level in your code.
For your convenience please find the video guide below:

When writing an automated visual test, sometimes we will want to change the comparison method between our test and its baseline, especially when dealing with applications that consist of dynamic content.

Applitools Eyes provides you the option to set different match levels within the test setup:


There are a few match levels that can be selected. All the options can be found under the MatchLevel enumerable.  When not specified, the default match level is "STRICT" which is designed to mimic the human eye.


 

The match level is defined at the test level (in future versions you will be able to set it in the step level or set it directly from the UI for specific regions). To set the comparison level use the following syntax:

C# (set before eyes.open)

eyes.MatchLevel = MatchLevel.<selected_level>

Java (set before eyes.open)​

eyes.setMatchLevel(MatchLevel.<selected_level>)​

Javascript (set before eyes.open)

eyes.setMatchLevel("<selected_level>");

Python

from applitools.eyes import MatchLevel

eyes.match_level = MatchLevel.<selected_level>

Ruby

eyes.match_level = :

eyes.match_level = Applitools::MATCH_LEVEL<selected_level>

UFT\QTP (Before open, either one of the lines according to the desired match level)

eyes.MatchLevel(eyes.Match<selected_level>)

 

Applitools Eyes can test the UI in 4 different comparison levels:
      1. Exact (MatchLevel.EXACT) - pixel-to-pixel comparison, not recommended. Exact performs a pixel-to-pixel comparison with predefined threshold. This mode was added mainly for demos and is likely to cause false positive results, therefore, we wouldn't recommend using it.
         
      2. Strict (MatchLevel.STRICT) - Strict compares everything including content (text), fonts, layout, colors and position of each of the elements.  Strict knows to ignore rendering changes that are not visible to the human (anti-aliasing changes, small pixel movements and various other changes that are typically caused when running tests on different machines with different graphic cards, etc.).  Strict is the recommended match level when running regression tests on the same browser/OS (strict is not designed to do cross browser comparison).

        Strict comparison will detect differences in the content/text, so although it does not do OCR by default, it can still detect character changes.  There is an option to activate OCR on elements in the .Net SDK, but in most cases we don't recommend using it.

      3. Content (MatchLevel.CONTENT) - Content works in a similar way to Strict except for the fact that it ignores colors (the content level can be useful if your website includes different colors which are not relevant for your tests).

      4. Layout (MatchLevel.LAYOUT) - Layout, as its name implies, compares the layouts (i.e. structure) of the baseline and actual images. It validates the alignment and relative position of all elements on the page, such as: buttons, menus, text areas, paragraphs, images, and columns. It ignores the content, colour and other style changes between the pages.

        The Layout algorithm reverse engineers the structure of the compared images by applying complex image processing algorithms resulting with the logical page structure of each image that includes all the components of the page. Then, it proceeds to map each component with its counter part in the other image which allows detection of missing or new elements as well as misplaced ones. All such elements are considered different and are highlighted as such in the comparison result.
         
        Layout will have greater benefits than other match level in certain scenarios, which most notably include: dynamic content, language localisation, responsive design, and cross-environment testing (including: operating systems, browsers, devices, and form factors) since it is more flexible approach of visual testing will have greater benefits.
         

        Note: As when using a floating region, a content comparison is made to validate that the same element is shared between the baseline and current images (with some movement tolerance) - a floating region has no relevance while comparing with the Layout match level, which ignores the content and focuses only on validating the structure of the page. 
         

Applitools dashboard allows you to examine what the result of the test would have been had you run it using the different match levels. 





For a full match level explanation video, please click here. 
 

 
 

Related articles