In GitHub Actions

Overview

AppMap Analysis can work within GitHub Actions to collect, store, analyze, and report on the behavioral changes within each Pull Request. AppMap will analyze the changes in your application on each pushed commit or pull request. AppMap performs a thorough analysis of the runtime differences, giving you:

  • Root cause analysis of failed tests.
  • Web service API changes, both breaking and non-breaking.
  • New and resolved security findings.
  • New and resolved performance findings.
  • New and resolved findings in other categories: maintainability, reliability, and user-defined rules.
  • “Behavioral diffs” as sequence diagrams showing changed runtime behavior within the PR.

Prerequisites

Add AppMap to GitHub Allowed Actions

If your organization limits which GitHub Actions can be used, update your organization settings to allow the specific AppMap actions required for this integration to work. All AppMap Actions are published on the GitHub Marketplace under the getappmap owner namespace.

In the top right corner of GitHub.com, click your profile photo, then click Your organizations.

Next to the organization, click Settings.

In the left sidebar, click Actions, then click General.

Under “Policies”, select Allow OWNER, and select non-OWNER, actions and reusable workflows and add the following AppMap required actions. This will ensure that current and future actions will be supported.

getappmap/*

Alternatively, if you would like to restrict to only the current list of actions further you can list them individually. You will need to keep this list updated as new features and functionality are added.

getappmap/install-action@*,getappmap/archive-action@*,getappmap/analyze-action@*

Install or Request the AppMap GitHub App

The AppMap Analysis application authorizes your account to run AppMap Analysis in CI. It also enables you to open AppMaps in your browser.

The AppMap Analysis application:

  • DOES NOT transfer any of your repository code, data, or AppMaps to an external server.
  • DOES confirm that you have access via GitHub permissions to the AppMap data stored in your repository action.
  • DOES open a browser tab with a signed URL that enables your browser to securely download the AppMap data from your GitHub and display it.
  • DOES directly transfer data from YOUR GitHub to YOUR browser, without going through any other other 3rd party services.

Organization owners can install GitHub Apps on their organization. If you are not an organization owner, you can still initiate the install process. GitHub will then send a notification to the organization owner to request them to approve the installation. Ask your organization owner to approve that request.

If you’re installing into your personal account, you will already have permissions to install this app there.

1) Click Configure on the AppMap Analysis GitHub page.

2) Select the GitHub Organization in which you are using AppMap.

3) Select All Repositories OR select the specific repositories that you have already configured the AppMap GitHub Action into.

Step-by-step walkthrough

Configuration of the AppMap GitHub Action happens inside a branch and can be easily tested in a Pull Request before merging any code changes to the mainline branch. This allows users to easily test AppMap in the environment before deploying across the repository.

Follow the steps below for your project. If you need additional assistance contact AppMap at support@appmap.io or join us in our Community Slack

Step 1: Create an initial baseline AppMap archive

Start by cloning your project to your local machine. Next, create a branch called appmap-analysis, and commit the code listed below to that branch.

File Name: .github/workflows/appmap-analysis.yml

name: appmap-analysis

on:
  pull_request:

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Set up Ruby
        uses: ruby/setup-ruby@v1
        with:
          bundler-cache: true

      # Install AppMap library and tools
      - name: Enable appmap gem update
        run: bundle config unset deployment      
      - name: Install AppMap tools
        id: install-appmap
        uses: getappmap/install-action@v1
        with:
          project-type: bundler # Choose the type of your project here:
                        # bundler, maven, gradle, pip, pipenv, poetry,
                        # yarn, npm, etc.
 
      ####################################################################
      # Put the commands required to run your application test suite.
      # For example, create and setup the test database, commands 
      # to run your test suite.
      - name: Rails test setup
        run: ./bin/rails db:migrate

      - name: Run tests
        run: ./bin/rails test

      - name: Archive AppMaps
        uses: getappmap/archive-action@v1
        with:
          revision: ${{ github.event.pull_request.base.sha }}

Commit the change:

$ git add .
$ git commit -m "ci: Bootstrap initial AppMap analysis archive"

Push the changes upstream to your branch and open a new Pull Request which will trigger the GitHub action. Once the build of your latest commit completes, verify that the AppMap Analysis build artifact has been created and stored in GitHub by navigating to the summary page for your GitHub Action run and looking for the “artifacts” list below the test results.

You should see a file named appmap-archive-full_<git hash> as well as an appmap-install.patch file, which you will use in the (optional) next step.

Step 2: (Optional) Apply the AppMap Configuration to the Project

Note If your repo is already configured for AppMap, you don’t need to do this step. This section describes how the apply AppMap configuration generated in CI to your project.

Download the appmap-install.patch file that’s in the Action summary to your local machine.

Unzip the patch file archive and apply the patch file to your local repository:

$ cat <path to patch file> | git apply

Example:

$ cat appmap-install.patch | git apply

Delete the patch file after applying it:

$ rm appmap-install.patch

Commit the changes:

$ git add .
$ git commit -m "ci: Configure the project for appmap"

Push the changes upstream to your branch to update your Pull Request.

$ git push

Step 3: Add the Build Triggers and Analysis Step

Next, update the GitHub Action configuration file, created back in Step 1 of this document and located in your repository folder at .github/workflows/appmap-analysis.yml.

In the on: section of the configuration, we will add additional configuration to trigger this action on pull requests and also on a schedule to ensure the archive stays fresh in the GitHub asset repository. If your mainline branch name is not called main, update it accordingly.

on:
  pull_request:
  push:
    branches:
      - main # Change this to the name of the mainline branch
  schedule:
    - cron: '0 0 * * 0'

Now that we have the initial baseline of AppMaps, replace the Archive AppMaps action at end of your GitHub Action configuration with a Save AppMaps job which saves AppMaps to the GitHub cache for later use. Finally, add a workflow call to the reference AppMap Analysis workflow file. It will be nested at the same level as the test: job.

Replace this block

- name: Archive AppMaps
  uses: getappmap/archive-action@v1
  with:
    revision: ${{ github.event.pull_request.base.sha }}

with this block.

- name: Save AppMaps
  uses: actions/cache/save@v3
  if: always()
  with:
    path: ${{ steps.install-appmap.outputs.appmap-dir }}
    key: appmaps-${{ github.sha }}-${{ github.run_attempt }}

Finally, add this block at the same nesting level as the test: job.

appmap-analysis:
  if: always()
  needs: [test] # You may need to change this to match the name of the step that runs your tests.
  uses: getappmap/analyze-action/.github/workflows/appmap-analysis.yml@v1
  permissions:
    actions: read
    contents: read
    checks: write
    pull-requests: write

The commit diff should look similar to this:

The complete updated configuration now looks like this:

name: appmap-analysis

on:
  pull_request:
  push:
    branches:
      - main
  schedule:
    - cron: '0 0 * * 0'

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Set up Ruby
        uses: ruby/setup-ruby@v1
        with:
          bundler-cache: true
      - name: Enable appmap gem update
        run: bundle config unset deployment      
      - name: Install AppMap tools
        id: install-appmap
        uses: getappmap/install-action@v1
        with:
          project-type: bundler
      - name: Rails test setup
        run: ./bin/rails db:migrate
      - name: Run tests
        run: ./bin/rails test
      
      - name: Save AppMaps
        uses: actions/cache/save@v3
        if: always()
        with:
          path: ${{ steps.install-appmap.outputs.appmap-dir }}
          key: appmaps-${{ github.sha }}-${{ github.run_attempt }}

  appmap-analysis:
    if: always()
    needs: [test]
    uses: getappmap/analyze-action/.github/workflows/appmap-analysis.yml@v1
    permissions:
      actions: read
      contents: read
      checks: write
      pull-requests: write

Commit the changes:

$ git add .
$ git commit -m "ci: Add build triggers and analysis step"

Push the changes upstream to your branch and create a new Pull Request for this branch.

$ git push

After the build completes, AppMap will post details of the Analysis build into your pull request.

If your pull request report shows changes to code behavior or new AppMaps (when no changes are made), you may have some non-deterministic tests. For additional help resolving this, please contact the AppMap engineering team via email or join the AppMap Community Slack.

Step 4: Configure Additional AppMap Analysis Rules

AppMap Analysis comes with a comprehensive set of rules that are categorized by their impact on applications: Performance, Reliability, Maintainability, Stability, and Security.

You can refer to the AppMap Documentation for more information about all the rules that are available within AppMap Analysis.

To enable additional rules simply add them to an appmap-scanner.yml file in the root of your project directory and commit it to your project.

This is a sample appmap-scanner.yml file which you can use to enable or disable certain AppMap Analysis rules. Rules can be disabled by commenting them out with the # character.

checks:
  - rule: authz-before-authn
  # - rule: circular-dependency
  - rule: deprecated-crypto-algorithm
  - rule: deserialization-of-untrusted-data
  - rule: exec-of-untrusted-command
  - rule: http-500
  # - rule: illegal-package-dependency
  #   properties:
  #     callerPackages:
  #       - equal: actionpack
  #     calleePackage:
  #       equal: app/controllers
  # - rule: incompatible-http-client-request
  # - rule: insecure-compare
  # - rule: job-not-cancelled
  - rule: logout-without-session-reset
  # - rule: missing-authentication
  - rule: missing-content-type
  - rule: n-plus-one-query
  # - rule: query-from-invalid-package
  # - rule: query-from-view
  # - rule: rpc-without-circuit-breaker
  # - rule: save-without-validation
  - rule: secret-in-log
  # - rule: slow-function-call
  #   properties:
  #     timeAllowed: 0.2
  #     functions:
  #       - match: Controller#create$
  # - rule: slow-http-server-request
  #   properties:
  #     timeAllowed: 0.5
  # - rule: slow-query
  #   properties:
  #     timeAllowed: 0.05
  - rule: too-many-joins
  - rule: too-many-updates
  # - rule: unbatched-materialized-query
  - rule: unauthenticated-encryption
  - rule: update-in-get-request

Add these changes to Git and commit and put them into the PR branch.

$ git add .
$ git commit -m "ci: Add customized scanner configuration"

Push the changes upstream to your branch which updates the Pull Request.

$ git push

The AppMap Analysis report will be updated on the completion of the build and a new report will be displayed.

Step 5: Merge this PR to deploy AppMap Analysis

Congratulations! You’ve successfully set up the AppMap Analysis Github Action and can now merge this into your project to make it available for every other developer to use on each of their subsequent pull requests.

Workflow examples

Reference implementations of the AppMap GitHUb Action are available for the following languages and frameworks:

Matrix builds

If your project uses matrix builds in order to split your test runs across multiple runners, you will need some additional configuration for AppMap to properly save and merge AppMaps from across all these runs.

From the GitHub documents on the matrix strategies:

A matrix strategy lets you use variables in a single job definition to automatically create multiple job runs that are based on the combinations of the variables. For example, you can use a matrix strategy to test your code in multiple versions of a language or on multiple operating systems. Use jobs.<job_id>.strategy.matrix to define a matrix of different job configurations. Within your matrix, define one or more variables followed by an array of values.

For example, to split tests across two runners you could set a matrix strategy like the following:

strategy:
  fail-fast: false
  matrix:
    ci_node_total: [2]
    ci_node_index: [0, 1]

For AppMap to analyze any failed test cases, you need to include the configuration flag fail-fast: false in your matrix strategy.

In Step 1, you’ll need to add an archive-id: to the getappmap/archive-action@v1 set to the unique index of the runner.

For example:

- name: Save AppMaps
  if: always()
  uses: getappmap/archive-action@v1
  with:
    archive-id: ${{ matrix.ci_node_index }} # Set this equal to the unique index of the runner

You will then need to create another task dependent on the previous task which which merges and archives the AppMaps.

appmap-analysis:
  if: always()
  runs-on: ubuntu-latest
  needs: [ record-appmaps ]
  permissions:
    contents: read
    actions: read
    checks: write
    pull-requests: write
  
  steps:
    - uses: actions/checkout@v3
    
    - name: Install AppMap tools
      uses: getappmap/install-action@v1
      with:
        install-appmap-library: false
    
    - name: Merge and Archive AppMaps
      uses: getappmap/archive-action/merge@v1
      with:
        revision: ${{ github.event.pull_request.base.sha }}
        archive-count: 2 # Set this to the total number of workers from the previous task

View an example workflow file here.

After the test cases complete, you will use the appmap-analysis-matrix reusable workflow with an additional configuration option archive-count.

The archive-count is the total number of archives created in the previous jobs. For example, if you have split your test runner across 2 hosts, you need to set the archive-count equal to 2.

After merging, you’ll configure your AppMap with the subsequent Archive and/or Analysis actions as in Step 3. Except you will use the appmap-analysis-matrix.yml reusable workflow and pass the variable archive-count equal to the total number of runners in the previous job.

For example:

appmap-analysis:
    if: always()
    needs: [record-appmaps]
    uses: getappmap/analyze-action/.github/workflows/appmap-analysis-matrix.yml@v1
    with:
      archive-count: 2
    permissions:
      actions: read
      contents: read
      checks: write
      pull-requests: write

See an example GitHub Action configuration with AppMap Analysis working with a multi-runner matrix build.

Permissions

permissions:
  contents: read
  pull-requests: write
  actions: write
  checks: write

Configures the permissions granted to the $GITHUB_TOKEN. This is the minimum required level of permissions that will allow the GitHub action to execute. Refer to the GitHub documentation for more details about action permission levels.

contents: read Grants the action permission to read the artifacts created by the GitHub action, such as the AppMap archive tarballs. This is used to fetch the “base” and “head” archive data for comparing changes between a mainline branch and changes in a pull request.

pull-requests: write Grants the action permission to comment on the Pull Request with a detailed report of code behavior changes.

actions: write Grants the GitHub action permission to interact with the artifacts and caches of the job.

checks: write Allows the action to annotate code inline in a Pull Request workflow.

Actions reference

Install AppMap tools

getappmap/install-action Prepares a repository to record AppMaps and to run AppMap CLI commands.

This action will run on both commits to the mainline and pull requests. This action will install the AppMap binary into the GitHub action environment in the /usr/local/bin/directory

Configuration Options

Archive AppMaps

getappmap/archive-action Archives AppMaps which have been built in the current project checkout.

This action runs when a new branch or Pull Request is merged to the mainline branch. This action will generate a .tar.gz archive of the AppMaps created by your test cases and store it as a GitHub artifact. The stored file includes the archive JSON and the AppMaps archive.

Configuration Options

Analyze AppMaps

getappmap/analyze-action

This action runs when a pull request (or draft) is opened, reopened, or a new commit is added. This action generates the AppMap Analysis report for the head revision, compared to the base branch. A report in Markdown format is generated and is stored as a GitHub artifact within the GitHub action. This report can optionally be pushed to the active Pull Request as a comment.

Configuration Options

Reusable Workflow

GitHub supports calling a reusable workflow in another project to simplify and avoid duplication. AppMap provides two reusable workflows for single runner and full matrix builds.

appmap-analysis.yml@v1

Inputs

inputs:
  runner-name:
    required: false
    type: string
    default: ubuntu-latest

For example you can customize your runner with:

appmap-analysis:
  if: always()
  needs: [record-appmaps]
  uses: getappmap/analyze-action/.github/workflows/appmap-analysis.yml@v1
  with:
    runner-name: My-Custom-4-Core-Runner
  permissions:
      actions: read
      contents: read
      checks: write
      pull-requests: write

appmap-analysis-matrix.yml@v1

inputs:
  archive-count:
    required: true
    type: number
  runner-name:
    required: false
    type: string
    default: ubuntu-latest

The matrix runner supports a custom runner name as well as requires an archive-count which needs to be equal to the total number of runners in your matrix build.

For example:

appmap-analysis:
  if: always()
  needs: [record-appmaps]
  uses: getappmap/analyze-action/.github/workflows/appmap-analysis-matrix.yml@v1
  with:
    archive-count: 2
  permissions:
      actions: read
      contents: read
      checks: write
      pull-requests: write

Multiple options can be passed to the reusable workflows.

appmap-analysis:
  if: always()
  needs: [record-appmaps]
  uses: getappmap/analyze-action/.github/workflows/appmap-analysis-matrix.yml@v1
  with:
    archive-count: 2
    runner-name: My-Custom-4-Core-Runner
  permissions:
      actions: read
      contents: read
      checks: write
      pull-requests: write
Was this page helpful? thumb_up Yes thumb_down No
Thank you for your feedback!