Understanding the Runtime Code Review Report

AppMap Analysis works with your Continuous Integration (CI) system to collect, store, analyze, and report on the behavioral changes within each Pull Request. AppMap analyzes the changes in your application on each pushed commit or pull request. AppMap performs a thorough analysis of the runtime differences, giving you:

• Root cause analysis of failed tests.
• Web service API changes - breaking and non-breaking - via comparison of generated OpenAPI definitions.
• New and resolved security vulnerabilities.
• New and resolved performance issues.
• New and resolved flaws in other categories - maintainability, reliability, and user-defined rules.
• Code Behavior diffs represented as sequence diagrams showing changed runtime behavior within the PR.

• The AppMap Change Report
• Summary and Status
  • Failed Tests
  • API Changes
  • Security Flaws
  • Performance Problems
  • Code Anti-Patterns
  • New AppMaps

The AppMap Change Report

Once a Pull Request is opened, reopened, or changed, an AppMap Analysis command will run to build a comparison report between the head revision and the most recent base revision for which an AppMap archive is available. This ensures AppMap is only analyzing the code which changed in the pull request. This report compares structural differences observed at runtime between the feature and the origin point of that feature branch. The same action can also add comments and annotations to the source code changes involved in the Pull Request. The report data is also available as a JSON file.

Key insights include:

• How the new code will impact the system.
• Any major architectural changes introduced.
• Changes to the code interactions from API to Database

Here is an example of a report from AppMap in CI:

This report shows the runtime analysis done by AppMap, which records code execution behavior before those changes are approved for deployment to production.

Summary and Status

The summary gives you an overview of the following code-related flaws, problems, and anti-patterns.

• Failed Tests
• API Changes
• Security Flaws
• Performance Problems
• Code Anti-Patterns
• New AppMaps

Failed Tests

AppMap will report on any tests that failed as part of your test suite. For each test that failed to complete successfully, AppMap will analyze the failed test, showing the full details of the error and the specific part of the test that failed.

API Changes

AppMap will report on any changes seen with your API routes at runtime. AppMap will identify changes to routes themselves, notifying you when a new route appears or an existing route is deleted. Additionally, it will identify changes to the response body, content, descriptions, and other attributes.

Security Flaws

AppMap analyzes your application’s behavior to identify new security flaws in how the new code changes execute. AppMap can identify issues such as:

• Deprecated cryptography algorithm
• Logout without Session Reset
• Deserialization of untrusted data
• and others.

For a full list of all the flaws and issues which AppMap can detect, refer to the Analysis Rules reference section.

Performance Problems

AppMap analyzes software behavior changes in your code base to identify performance problems before you merge your code changes. AppMap identifies a variety of software performance problems such as:

• N Plus One SQL Query
• Slow Function Calls
• Slow HTTP Server Requests
• and others.

For a full list of all the flaws and issues which AppMap can detect, refer to the Analysis Rules reference section.

Code Anti-Patterns

AppMap can identify any major or potential structural flaws in the architecture, or logical flow that could be introduced due to merging the feature into your application. Some examples of these code anti-patterns that AppMap can identify are:

• Circular Dependencies
• Too Many SQL or RPC Updates
• SQL Queries from View Layer
• and others.

For a full list of all the flaws and issues which AppMap can detect, refer to the Analysis Rules reference section.

New AppMaps

New test cases added to a pull request will lead to new AppMaps being created, with one AppMap created for each new test. AppMap will list all of the new AppMaps (and therefore, each new test) created in each pull request.

---