What is a service catalog?

A service catalog is the lifeblood of a successful internal developer portal (IDP). They are a centralized inventory of software services and components used within an organization. A catalog provides Platform Engineers, SREs, and development teams with a comprehensive view of their assets, facilitating efficient management, resource utilization, software standard optimization, and thoughtful decision-making. 

A truly good and effective catalog should possess these qualities:

1. Completeness: The catalog should contain all services in production and associated data.
2. Accuracy: Regular updates should be made to ensure the catalog reflects the latest information and can be trusted to be accurate.
3. Accessibility: The catalog should be easily accessible for everyone who needs it and user-friendly, allowing for effortless navigation.

However, many organizations struggle to establish effective catalogs, falling short of meeting the necessary criteria. 

In the past, catalogs were typically managed by using spreadsheets or importing manually compiled configuration files into a solution. These methods are resource-intensive and time-consuming, burdening teams that are already stretched, leading to stalled catalog creation and IDPs that never get off the ground. 

To expedite catalog creation and streamline maintenance, OpsLevel offers Catalog Engine, a solution for fast catalog creation and maintenance that teams can trust.

What is the Catalog Engine?

OpsLevel’s Catalog Engine, is a combination of automated, AI-enhanced features that support fast, accurate catalog creation and lightweight, continuous maintenance to keep catalogs accurate and up-to-date while reducing the burden for engineering teams.

How does it work?

Catalog Engine has five main components:

• Automated catalog creation
• Recommended service owners & notification
• Grouping services by aliases
• Service page actions
• AI-derived service descriptions

Automated catalog creation

Automated catalog creation is the cornerstone of our Catalog Engine. Without the Catalog Engine, building out your catalog can be a tedious and time-consuming process. When first starting out, teams often attempt to build their catalogs manually or track everything in a spreadsheet. This can quickly become an unmanageable process. To create a truly accurate catalog takes multiple people across various teams. The “cat(alog) herding” alone often leads to stalled roll outs and catalogs that are never completed. Additionally, when a team can focus their attention on catalog building, that means precious development time is lost, sometimes for months. Another challenge is even knowing ahead of time what all of your services are. You might think you do, but it’s common for organizations to have services that they are unaware of, and you can’t manage what you can’t see. 

Automated catalog creation gathers data through integrations with a user’s toolset to discover attributes (see table below). Through these integrations, all running services, including services that users may not be aware of, are discovered and attributes are attached to each service. 

You can think of this in two phases:

1. Entry collection: We discover what services exist across data sources

2. Metadata enrichment: We associate and attach data about those services automatically

‍

As part of this automated process, Catalog Engine groups newly detected services by associated aliases to eliminate potential duplicates and leverages AI to generate context-rich service descriptions to help teams to start accurately managing those services right away.

Automated catalog creation greatly expedites the catalog formation process. This reduces the time-to-value from (6 - 12) months to a matter of weeks so development teams can quickly and easily start getting the information they need to begin setting and improving standards right away. By using automated catalog creation, then supplementing with YAML files after the initial framework is created, teams can achieve value much more quickly without sacrificing high-fidelity or granularity in their data. Additionally, automated catalog creation will discover the presence of services that teams may not be aware of, ensuring that the catalog is truly accurate and that everything is accounted for.

‍

Recommended service owners

If a complete catalog is the heart of an IDP, then the next more important component is service ownership. Service owners are tasked with managing every aspect of the service to ensure that everything is functioning properly and efficiently. When a service is left orphaned, you don’t know who to contact when something breaks. This will eventually lead to a drop in service levels and ultimately the overall performance of your systems and business. But having owners assigned to everything isn’t always as easy as it may seem. In large, heavily-matrixed organizations figuring out who should own what isn’t always clear. Add to that frequent reorgs that require ownership to change hands. Having to hunt this information down and ping individuals creates quite a hassle for engineering teams, and the burden of that upkeep alone is enough to leave them unmanaged.

In response to these challenges, recommended service owners are automatically detected in OpsLevel so owners can easily be assigned when services are added to the catalog. Teams and/or individuals who are identified as potential owners are also notified via email when there is an action to take. So even if you are not in OpsLevel everyday, you will still be alerted that there is an action you need to take. In addition to email notifications, users are also alerted via a sidebar in the app of the top three services that need ownership assigned. The recommended service owner feature meets users where they are, eliminating the time spent tracking down owners and reducing the likelihood that services will be unmanaged.

Grouping by aliases 

Catalogs are ever-evolving. Just as your system evolves and changes, your catalog should too and that process should be as effortless as possible. As you integrate with more and more solutions over time, you’ll find that some detected services have already been registered in your catalog but may appear to be new because of different naming conventions. This can create a huge backlog of services to sort through, manually registering them as new or merging them with already existing services individually. This creates a huge time sink for engineers, or even worse, it just doesn’t get done altogether, allowing new services to slip through the cracks and a loss in accuracy for the catalog.

Catalog Engine takes the guesswork out of trying to figure out what services are or are not duplicates and removes any additional time teams would otherwise spend trying to figure out the answers and the mundane work of taking action on each individually. Additionally, some tools force users to adhere to strict naming conventions to avoid having newly detected services appear as duplicates. With this feature, teams can continue to use their own naming conventions… another element that encourages teams to keep their catalogs up-to-date by facilitating lightweight maintenance.

AI-derived service descriptions

You will occasionally come across services that are so old no one knows what they are anymore, you may have even forgotten that they exist. Without any info, where do you start? That’s where AI-derived service descriptions come in. Catalog Engine uses large-language models (LLM), a type of artificial intelligence (AI) to process natural language and pull information from documentation to create service descriptions. This gives teams a jumping off point with these “mystery” services and saves time that would otherwise be spent writing and entering these descriptions manually. 

Service page actions

When first building out your catalog, you will most likely have a high number of detected services to sort through and register. This can appear overwhelming at first, but Catalog Engine makes it quite manageable. 

Bulk actions allow users to select multiple detected services at once and accept or ignore everything with one click. This may seem like a small thing, but when you’re first creating your catalog or if you’ve recently integrated with a new service, the ability to quickly take action on ten, twenty, or thirty services at once saves a tremendous amount of time and prevents creating a daunting backlog that will eventually lead to outdated catalog. 

You can also filter by services to quickly identify newly detected services based on their status, source, and team to narrow down the list and just focus on the services that you’re looking for.

Additionally, you can toggle your detected services list by last detected, so you can view services by an integration date or surface old services that have been lingering and need to be addressed.  

A catalog you can trust

Having a complete, data-rich, and accurate catalog doesn’t have to take months to build or require a lot of tedious, on-going maintenance. The Catalog Engine offers fast, accurate catalog creation and lightweight continuous updates to reduce the burden on engineering teams and get you up and running with an Internal Developer Portal in days, not months.

‍

Ready to learn more? Schedule a call with our team today to see how the OpsLevel Catalog Engine can improve visibility and increase efficiency for your entire engineering team.

‍

‍