The AppMap Test Failure Report directs you to the root cause of test failures.
Opening a pull request triggers AppMap to start observing your CI run and analyzing your application’s runtime behavior for flaws. If one or more tests fail, AppMap will issue a comment telling you why the test failed and where in the code you need to go to fix it.
The AppMap Test Failure Report
This is easier than digging through logs, or parsing cryptic system messages to find out why individual tests failed. This example Pull Request introduces a change to the correct_user method in the UsersController to flash a message about the 24-hour time limit in which a user can edit their profile.
Example Pull Request with failing tests
The build didn’t complete because the users_controller_test is failing. The AppMap Test Failure Report shows me that on line 38 the test expected a redirect, but was instead returned a 200 ‘ok’ status.
AppMap Test Failure Report showing the line that caused the test to fail
Related to that, line 45 was expecting a false response but instead got true.
AppMap Test Failure Report showing the expected response
The AppMap Test Failure Report contains a preview of the code change that is related to the failing test. I can see that the redirect was removed. The report also includes a link to the full source of the failing test.
AppMap Test Failure Report showing the code change that led to the test failure
This is a relatively simple change, but if I needed to dig deeper the AppMap Test Failure Report also includes links to a sequence diagram showing a visual diff of the test-breaking behavior.
A visual diff featuring a sequence diagram
If I go to the ‘Files changed’ tab , and look at the diff, I see that AppMap has added annotations to show me files related to the check that are throwing a test failure.
AppMap adding annotations to the ‘Files changed’ tab in a GitHub Pull Request
It becomes really hard to miss the fact that this test is expecting a redirect. At this point I’m having questions about whether or not this 24-hour check was added in the right way.
The code needs to be updated to handle the redirect, and the test needs to be expanded to handle both cases for the 24-hour timer.
Updating the fix to handle cases expected by the test
Once this is done, the tests pass, the build finishes, and now AppMap can go on to complete its analysis of the application’s full runtime behavior.
AppMap runtime code review
Because we added two new test cases when updating the code, the runtime code review shows that two new AppMap files visualizing this new code behavior have been added. Using the AppMap Test Failure Report means that I don’t have to dig through logs to manually analyze test failures. I make a change, run one or two tests locally, push to AppMap to see the downstream impact of that change, make necessary adjustments, and repeat.
AppMap is available on the GitHub Marketplace. Once installed it provides a guide for the setup and configuration of the AppMap GitHub Action.
Get AppMap