In CI

AppMap Analysis can work with your Continuous Integration (CI) system to collect, store, analyze, and report on the behavioral changes within each Pull Request. AppMap will analyze the changes in your application on each pushed commit or pull request. AppMap performs a thorough analysis of the runtime differences, giving you:

  • Root cause analysis of failed tests.
  • Web service API changes - breaking and non-breaking - via comparison of generated OpenAPI definitions.
  • New and resolved security findings.
  • New and resolved performance findings.
  • New and resolved findings in other categories - maintainability, reliability, and user-defined rules.
  • “Behavioral diffs” as sequence diagrams showing changed runtime behavior within the PR.

Request early access to our GitHub integration

The AppMap Analysis GitHub actions can run cooperatively or independently of your existing CI job. For example, AppMap Analysis can run as a GitHub Action even if you do not use GitHub Actions as your CI system.

AppMap Analysis runs entirely within your existing GitHub account and does not send code or data to an external or 3rd-party server. For more information, refer to the AppMap Security FAQ.

How It Works

Once you create a pull request, AppMap data is recorded as your test cases run. As AppMaps are generated, an AppMap archive file is created that includes all the AppMap data, plus some metadata about the job. As code is pushed to a branch, AppMap Analysis create an archive file for that code revision. This archive file is automatically saved as a CI artifact. Once an archive has been created, it can be compared to the “base” revision (i.e. your mainline or production branch).

Once a Pull Request is opened, reopened, or changed, an AppMap Analysis command will run to build a comparison report between the head revision and the most recent base revision for which an AppMap archive is available. This ensures AppMap is only analyzing the code which changed in the pull request. This report compares structural differences between the feature and the origin point of that feature branch. The same action can also add comments and annotations to the source code changes involved in the Pull Request. The report data is also available as a JSON file.

Key insights include:

  • How the new code will impact the system.
  • Any major architectural changes introduced.
  • No fluff to sift through, only the results of the code to be reviewed.

Here is an example of the AppMap in CI report:

This may look like a simple test runner integrated into your CI system, but the difference is runtime analysis, recording your code as it behaves at runtime before it is approved for production.

The Summary & Status

The summary gives you an overview:

  • Any tests that failed or failed to run.
  • Any changes made to your application APIs.
  • Findings
    • New: Any major or potential flaws in the architecture, or logical flow that could be introduced due to merging the feature into your application.
    • Resolved: Any flaw that existed at the origin point of the branch that no longer exists as a result of the feature branch.

Analysis Findings

This is where the real power of AppMap Runtime Analysis shines. Here you get a list of all of the newly created code quality issues that will be introduced to your application if this code is merged. With details like:

  • An explanation of the flaw.
  • A link to the specific code where the flaw is detected.
  • A link to the difference view which shows the changes between the map at the time the branch was created (code at the moment the feature branch was created) and the map at the point the pull request was created.
  • A link to the specific rule that detected the flaw.
  • An option to ignore this finding in this branch.

The value of AppMap in CI

We designed AppMap’s findings as a comment in the PR itself to enable the Development, QA, Architecture, and Security teams to review code faster, evaluate code independently, and catch deep-rooted code issues easier. This helps you determine the stability of new code in the context of where the code actually lives and allows for faster and more reliable code delivery.

By simply browsing the PR comments, a DevOps team can reliably deploy code with the assurance that it meets all needed code quality requirements, reducing the amount of unnecessary communication between teams just to get the code live long before it is ever pushed to production.

Requirements

If you have already generated AppMaps outside of CI (for example, by running your test cases locally), you can quickly deploy AppMap in CI using the same commands used to execute your test cases. If you already have an existing CI job that builds an environment to execute your test cases, AppMap Analysis can be added to that job.

Next steps

Was this page helpful? thumb_up Yes thumb_down No
Thank you for your feedback!