DogOps: Dogfooding OpsLevel @ OpsLevel

OpsLevel engineers aren't so different from engineers at other companies: we face similar challenges, triumphs, and puzzles. It's probably no surprise that, like any good team lead, myself and the rest of the Platform team use OpsLevel ourselves to help improve the security, reliability, and maturity of OpsLevel services.

With that in mind, I'd like to introduce a new blog series, covering an insider's view into how we dogfood at OpsLevel. In sharing, we hope that we might help others, make some friends, or at the very least, let you know that you're not alone in your toughest, most head-scratching engineering challenges.

What is Dogfooding

Let’s talk about “dogfooding” in case that’s not a term you’ve heard before. Dogfooding refers to the saying “eat your own dogfood”. With technology you create, the sentiment is that you should be using your own product so you can experience the trials and tribulations your customers will face in order to build a better product. You can read more about it here on wikipedia.

Over the past year, OpsLevel has built our own Platform team to do just that. This team is responsible for building tools like our CLI and Terraform provider, as well as driving campaigns and maturity improvements across the company.

OpsLevel Circular

To do this we have our own OpsLevel account we call Circular. This is the term we use to refer to our own account in OpsLevel (the product).  This is where we register our services, track our ownership and drive our own maturity improvements just like our customers.

What is a Service

First off, let's get on the same page about what exactly a "service" entails.

After all, it's often the first question we get from customers who are early in their "services" journey. Until recently, Circular had only 1 service registered. That was our monolithic Ruby on Rails application, which powers OpsLevel.

Our array of services has since expanded, but some of those components still might not fit snugly into your definition of what a "service" actually is. Here's the framework for how we defined what counts as a service.

• Our Ruby on Rails application - https://app.opslevel.com
• Our Jekyll website - https://www.opslevel.com
• Our CLI
• Our API client library - opslevel-go
• Our CI system - Gitlab runner
• Our ingress controller - Nginx
• Our certificate controller - cert-manager
• Our deployment controller - Flux
  

Each of these may or may not fit into your idea of a service, but that is ok. The important part is that a service is anything you want to keep track of, know who the owner is, and drive improvements upon.

In some circumstances, that may mean viewing the client library and the API application as one service. That's totally fine, and may vary from company to company. In other cases, it may also mean avoiding the registration of products for which you don't control the code, like GitLab Runner and Nginx.

In our case the above list are things we want to keep track of, know who the owner is, and drive improvements upon, even though we don't control the code for some of them.

We hope that, given these simple parameters, it becomes easy for you to identify what you will register as a service.

Taking Inventory

Now that you've identified what you will register as a "service" in OpsLevel, the problem space of taking an inventory of everything your company has should be more clear and you can start directing your efforts toward the most valuable ones.

Most customers initially struggle with this stage. After all, it's a monumental task when you have 100+ services. It can feel daunting.

But take comfort in knowing that incremental progress is perfectly okay - maybe even ideal. Careful, accurate progress looks a lot better than no progress at all, or barging ahead without knowing what you have or who owns it.

OpsLevel has developed easy, clever ways to ingest large amounts of services into your catalog. Whether you use Kubernetes, Terraform, AWS, Google Cloud, Azure, or something else, you can bulk ingest services with minimal effort through OpsLevel.

In a future post we’ll cover some of the different ways we register our services and the reasons why we did it this way.

Call for Questions

With our ever-growing feature set of tools and services with which we integrate, there's a lot more to say. If you've made it this far, please reach out and let us know which topics you'd like us to cover in the near future. Have you ever had a burning question, like "How do I bring my AWS ECS services into OpsLevel?" Consider this an exploratory space where we can help you by sharing how we did it ourselves. If it's not something we've done before, we'll find a customer who has, and we'll share the solution we developed.

Until Next Time

We look forward to continuing this series and we hope you enjoy and find the content useful as you continue your service maturity journey at your company.