Scheduling Automated Tests

You can run an automated test in one of two ways:

  • Interactive mode: Use the Sofy Device Lab to execute and observe a test on a real device. When you create a new test, we recommend watching the test first before scheduling and running the test on multiple devices. To learn more, visit Executing Automated Tests.  
  • Scheduled runs (batch mode): Schedule and run multiple test cases or test suites across multiple devices without waiting for a device to become available. You can set these scheduled tests to run immediately, or on a daily/weekly schedule. You’ll receive a notification once your test runs are complete. 

For the purpose of this article, we’ll discuss how to schedule automated test runs on multiple devices. 

Creating a Scheduled Automated Test

  1. Log in to your Sofy account.
  2. Select Automation from the left navigation bar. 
    Sofy home page with a callout over the Automation option in the left navigation bar.
  3. Click the Runs & Results tab:
  4. Select Add New Run at the far right. The Create New Automated Test Run modal window will open:
  1. Name your test run and select an app version. Choose to Run Test either Now or Later. If you select Later, you’ll define your schedule at the end. 
Create New Automated Test Run window with a callout over the Run Test options.
  1. Choose either the Test Suites or Test Cases tab.
  2. Scroll through or search for the desired test suites or test cases. Under Test Cases, you can also use the filter button to the right of the search bar to filter your test cases by the following parameters:
    • By Tags: View test cases with a specific tag (you can select multiple tags). 
    • By User: View test cases created by a specific user.
    • By Priority: View test cases of a specific priority. 
    • Created Between: View the test cases created between two specific dates. 
      Test cases filter options.
  3. Select all test cases or test suites you want to add to the scheduled run. Selected tests will appear in the right panel.
  4. Click Next. 
  5. Select what devices you want your test to run on:
    • Any Available Device: Run the automated test on the first available device, or any available device with a specific OS version or from a specific manufacturer. 
      1. Check the box Select first available device if you’re fine running the automated test on the device that is first available
        OR
      2. Specify the Manufacturer and OSVersion you want to prioritize when the automated test is executed.
        Any Available Device settings window.
    • Specific Device: Run the automated test on one or more specific devices. 
      1. Select which specific devices you want to run your test on. Filter the list of devices by the Manufacturer, OSVersion, and RAM.
        Specific Device settings window.
  6. Click Next. 
  7. If you chose to run your test later, set the test run's schedule:
    1. Run test: Choose to run the schedule either Daily or Weekly.
    2. Time: Click the clock icon and specify a time of day to run the test.
    3. Until: Click the calendar icon and specify what day you want to stop running the scheduled test. The scheduled test run will continue to execute within the specified parameters until that date is reached.
      Schedule Time window for scheduling a test run for later.
  8. By default, the Clean Device Cache option is turned on to enable Sofy to reset and restart your app with a clean cache after each run. Toggle it off if you’d like Sofy to retain the device’s cache during each run, which may be useful if you want to retain certain information, such as login info. 
  9. Review your scheduled test run selections, then click Schedule Test.

You can review all scheduled test runs and their results in the Runs & Results section:

Sofy recommends limiting your schedule execution to 2 hours. For schedules that continue to execute beyond 2 hours, the schedule may be paused temporarily to cool down the device, before it continues from its previous state.

How did we do?