Mesmer Best Practices

Introduction

When using a powerful product like Mesmer it can be difficult to know the best way to get things done. In order to simplify this, here are a series of learnings that will help you navigate best use of all the features. With these tools you will become a Mesmer genius in no time!
Let's go through some best practices.

Recommended Crawl Time

Mesmer recommends that crawls on your app run for 4 to 6 hours to get good coverage of the app. As you see the crawl results the time can be adjusted to best fit your situation.

When to use assertions to stop a test

"Mark as Defect - Continue Test" should be the preferred solution for handling defects. "Mark as Defect - Stop Test" should only be used when the state of the test has to be a certain way or there is no point is continuing. An example might be a banking app which is testing a withdrawl from an account. If there is not enough money is not in the account, there is no reason to continue replaying the test. This is a good time to use "Mark as Defect - Stop Test". Otherwise it makes more sense to us "Mark as Defect - Continue Test". Mesmer will still mark the defect for your review, but the test itself will go forward and not look like a replay failure.
We have found many customers believing Mesmer had failed to replay their test when in reality the assertion in a previous screen has failed. Look for this if you see the following

Use "contains" not "equals"

Sometimes an assertion is being used to test that a specific text exists in an element, such as the word "login" on a button. It might seem intuitive to use the condition equals, but over time we have seen the test will be more resilient to change with the condition contains and recommend that be used instead.
When I tap the element matching condition that its text contains text "Items to cart"
instead of
When I tap the element matching condition that its text equals text "Items to cart"

Image compare assertions

There are two discussion points around creating assertions with Image Compare.
The first is a warning: currently this feature will not work if you record on a physical device and replay on a virtual one, or visa versa. Image Compare is not transitive between device types.
The second is to watch out for dynamic data. We frequently see customers set an assertion with content that changes frequently. For example think about the Amazon mobile app. We might see this set on the top product on the Daily Deal list. The test will run fine today. Tomorrow the top product will change, and the test will fail! The lesson here is to make sure you make your assertions on immutable characteristics.

Dynamic data and Gherkin

Gherkin snippets that handle dynamic data can require some thought. The issue is that the data is changing. Maybe the DOM resource ids change build to build (or run to run!). Here are some approaches that we've found to help in these situations.

Always use Gherkin to select an item in a dynamic list

If you have a list of items that changes from time to time, prefer a Gherkin for your assertion. It provides you more flexibility and tends to allow for an assertion that is much more resistant to change.

Select list items with ordinals

Preferentially use the Gherkin syntax for ordinal numbers to select an item in the list, even if the list does not continually change. Many applications that change their DOM internally will otherwise fail. Ordinal numbers are numbers such as '3rd' as in the following expression
When I tap inside 2nd list item containing text "Buy now"

When ordinals won't work

Sometimes this will not work. Maybe you need a specific item on the list, but it's position changes from run to run. In that case see if you can reference something unique to the item. In this example we are using the text "Refridgerator".
Given I tap on element where its text contains "Refridgerator"

Using ordinals inside a container

When looking at the DOM for your product to select from a dynamic list, sometimes you'll see your app puts the list inside a container. In that case you can use an ordinal number inside a container as in this example
When I tap 3rd tag "actionable_image" inside matching condition that its resource id is "resource ID: com.example.app:id/productListLayout"

Using resource IDs

Be careful using these. Many apps these days are built on frameworks where the resource ids in the DOM change every build. but if you have an app where they stay the same you can likely create some very targetted Gherkin snippets that will work replay after replay.
When I tap element where its resource id is "android:id/text1" where text contains "Click this"
In this use case the more specific the snippet, the better.
When I tap on element where its resource id is "com.example.app:id/featureSpinner" inside matching condition that its class is "android.widget.LinearLayout" containing label "No click here!"

How Swipe works in Gherkin

When using the Gherkin swipe feature you need to be aware of its implementation. Here are the details you need to know.
  1. 1.
    Swipe down means scrolling towards the top of the screen.
  2. 2.
    Swipe up means scrolling towards the bottom of the screen.
  3. 3.
    Do not use Rapid Scroll feature today. Use Scroll Until instead. There is currently a bug that will be fixed in the future. At that time this recommendation will be removed.
  4. 4.
    By default, Gherkin based swipes start in the middle of the screen.
    1. 1.
      Hey! What if we want a full screen swipe? To accomplish this you need to find an element at the bottom of the page (something that will always be there) and swipe off that element a number of pixels that gets you to the top of the screen. Given I swipe 2000px on element where it's text is "Swipe launch pad!"

Be familiar with the Gherkin syntax

The Mesmer Gherkin syntax is documented [here](https://help.mesmerhq.com/guides/gherkin-syntax). Give it a read -- there are many, many useful features there.

Test Data in Crawl

When doing a crawl of your mobile app without a journey you sometimes want to specify different pieces of data, such as what phone number Mesmer should fill in while crawling your app. To do this, on the Project Test Data page you would add the data with the prefix of "p.". For example adding a phone number would look like this:
Here is a full list of all the data you can currently provide
  • password
  • creditCardNumber
  • creditCardExpiry
  • crediCardCVV
  • email
  • name
  • username
  • address
  • zipCode
  • search
  • countryCode
  • phoneNumber
Last modified 1mo ago