Visualizing Mastodon Internals

Visualizing Mastodon Internals

Mastodon is free and open-source software for self-hosting social networking services. It most closely resembles Twitter, but where Twitter is a centralized service for all its users, Mastodon consists of thousands of individual Mastodon hosts that are run by individuals and organizations.

Each of these instances are part of the greater Fediverse, a decentralized group of systems that can all communicate with each other. There can be a lot to learn when getting started as a Mastodon user, but luckily there are some fantastic blog posts and sites with tips out there to help you get started.

I’ve been checking out Mastodon as an end user (follow me!) and I wanted to get to know the project better and see if there is an opportunity to contribute. Mastodon is written in Ruby on Rails, which is an easy to learn web application framework.

Joining the Open Source Development Community

Whether you are running your own federated instance or not, you may decide to contribute back to the open source community. I don’t run my own instance (yet?) but Mastodon is an approachable project for me because of my experience with Ruby on Rails. There is definitely no shortage of help needed with any open source project and as its popularity grows so can new requests and issues.

But Mastodon is a large Ruby on Rails application, and for even the most experienced Rails devs out there, onboarding to new codebases can be time consuming. As a former Ops person I find the best way to learn a project is to get it running and see how it works.

Mastodon is a very well tested application with wide test case coverage, but it can be challenging to get up and running for folks with limited systems experience. Mastodon has a few system dependencies such as Redis, Elasticsearch and Postgresql and this can add to the complexity in operating it.

Lucky for us, this project ships with a devcontainer.json configuration file, which will dramatically reduce the time it takes to run Mastodon locally.

What are Dev Containers?

Dev Containers are a feature of VS Code (and also supported on GitHub Codespaces). They enable a developer to use a Docker container as a fully featured development environment. Why is this so amazingly helpful? One of the most difficult parts of running an application locally (or even running tests locally) is requiring very specific tools, runtimes, or libraries to successfully work with a codebase. There are MANY tools out there that aim to solve this problem (tools like asdf or pyenv and others), but installing the entire code editor with all the necessary dependencies is close to a foolproof solution.

Dev Containers and Mastodon

Mastodon already includes a devcontainer.json which points to the appropriate docker-compose.yml file, configures various VS Code extensions to be installed in the container, sets up port forwarding, and defines the command to run after the containers are created. In this case, after the container starts, all the commands detailed in the Developer Docs are executed so our container will be ready to run Mastodon. You’ll need Docker Desktop setup and working before you get started.

Open Mastodon in a Dev Container

Launching Mastodon inside a Dev Container is simple with VS Code. When you open a supported project, VS Code will notify you with a popup to relaunch inside a container.

visualizing mastodon internals

Clicking “Reopen in Container” launches a new VS Code window and starts the deployment process. When the postinstall command completes you’ll be prompted to Press any key to close the terminal.

visualizing mastodon internals

Opening Docker Desktop, you’ll see all the containers involved in operating Mastodon.

visualizing mastodon internals

We’ll now be dropped directly into the command line inside our Docker container with the environment perfectly configured to support running this app.

visualizing mastodon internals

Visualize How Mastodon Works

As an employee with AppMap I use AppMap every day to explore new and interesting code projects. AppMap is an open source runtime code analysis tool which helps developers visually understand how their code actually operates at runtime.

AppMap records your applications interactions by watching your test cases running, or by seeing how your application runs locally, in Docker, or in development. The AppMap Diagrams will include data about your application like function calls, HTTP request attributes, authN/authZ, SQL queries and more.

The result is a complete picture showing exactly the path the code takes as it executes. From API request to the database - you can see all the packages, classes, and functions involved in the request.

visualizing mastodon internals

Install AppMap

You can skip the entire installation by cloning Mastodon from the AppMap GitHub repository which is pre-configured and ready to run. But you can check out our last post to learn how you can add AppMap to Mastodon or any other project.

Record AppMap Diagrams of Mastodon

Now we can simply launch the rails application by running bundle exec rails server. Since the AppMap gem is installed at the top of our Gemfile, AppMap will enable request recording by default. This will generate a new AppMap for each request we make into Mastodon and help us better understand what is happening behind each of those API requests.

visualizing mastodon internals

Because Dev Containers are awesome, VS Code will automatically notice when one of our forwarded ports is available with a connection on the other side. When the application starts, simply click on “Open in Browser” and see your local Mastodon server running.

visualizing mastodon internals visualizing mastodon internals

Generate AppMap Diagrams

With Mastodon running and AppMap recording enabled, we can start to interact with the app. When running, the only seed data that gets created is the default admin account. You can login with:

Username: admin@localhost:3000 Password: mastodonadmin

After logging in you can interact with the application and AppMap Diagrams will appear instantly in VS Code.

visualizing mastodon internals visualizing mastodon internals

Click on any of them to open them in VS Code to see the HTTP requests, function calls, SQL queries and more.

visualizing mastodon internals

As more AppMap Diagrams are generated, you’ll start to see AppMap annotating the code functions. Clicking on any of these will show you a list of all the maps that code is used within. This can be incredibly helpful as you are onboarding with Mastodon to better understand where and how certain functions are used.

visualizing mastodon internals

Summary

At this point you should have been able to get Mastodon up and running locally with the power of VS Code and Dev Containers. Once the Dev Containers are deployed, adding AppMap is as easy as clicking a single box to install. After installation, you can generate maps and learn how the Mastodon internals work!

In part 2 of this post, we’ll dive into how to run rspec tests locally on Mastodon using Dev Containers. With AppMap installed, we’ll automatically record those test cases running and generate maps. From there we’ll modify some code and show how AppMap can re-run only the specific tests required for the changes you made. This can save a lot of time when doing local development by only running a handful of tests vs the entire test suite.

If you are interested in learning more, you can clone the Mastodon project in the AppMap repository. You can add AppMap to your VS Code or JetBrains code editor. And finally, you can join the conversation and chat with other AppMap engineers and users by joining the AppMap community Slack


AppMap logo Get AppMap