Debugging Test Failures - Best Practices
When a test case fails, it's crucial to identify the root cause of the failure. Effective debugging requires a thorough understanding of both the test case logic and the application under test.
Best Practices for Debugging
- Reproduce the Issue
- Rerun the test to confirm that the failure is consistent.
- Observe exactly where and how the test fails.
- Review the Failed Step
- Identify the specific step in which the failure occurred.
- Check for missing inputs, incorrect selectors, or timing issues.
- Analyze the Root Cause
- Understand the intended behavior of the test step.
- Compare the expected vs. actual results.
- Check logs, screenshots, or recordings for clues.
- Make use of Sofy's Comparison Tools
- Compare Baseline vs. Current Run to identify UI or flow changes.
- Use Element Explorer to compare attributes like XPath, resource ID, or visibility.
- Verify Test Data
- Ensure correct and up-to-date test data is being used.
- Double-check that variables are properly mapped.
- Check Build or Environment Changes
- Validate that the app build hasn’t changed unexpectedly.
- Ensure the environment or backend services are functioning correctly.
- Review Step Timing and Waits
- Look for steps failing due to timing issues.
- Add or adjust dynamic waits where necessary.
- Use Logs and Performance Insights
- Review system logs or step execution times.
- Identify delays or errors not visible in the UI.
- Make Targeted Edits
- Use Offline Edit or Lab Session to modify and test the failing step directly.
- Apply changes incrementally to isolate the fix.
- Revalidate and Save
- After making changes, re-run the test.
- If successful, document the root cause and update the test version with a clear description.
Clear cache
Cache stores temporary data to speed up application performance. However, outdated or corrupted cache can cause test failures or unexpected behavior.
When to clear cache in testing/debugging:
- If the test case is failing due to outdated data or UI not reflecting recent changes.
- When updates to the app aren't loading as expected during execution.
Clearing Cache in Sofy Lab
During a device session, users have the option to clear cache for the app being tested.
- When Clear Cache is used, the app is relaunched with cached data removed.
- This helps resolve issues caused by outdated or corrupted cache without requiring a full reinstallation.
- Once Clear Cache is performed, the green dot indicator—located next to the app name—is removed.
- This green dot signifies that cache is present. Its absence confirms that the cache has been successfully cleared.
Conditionals
Conditionals allow users to create dynamic and flexible test flows based on specific scenarios or outcomes. They help test cases adapt to different application states during execution.
Common Use Cases for Conditions
Example:
If a pop-up appears, then close it;
Else, continue to the next step.
