Build vs Buy: Why Prodigy Chose OpsLevel to Solve Service Ownership

The Problem

What Does Ownership Mean?

Software Architect Ben Johnson had a good grasp of all the services that existed in Prodigy’s environment. But Johnson and Prodigy knew that all this knowledge needed to be documented and shared widely across the organization.

They quickly discovered this was easier said than done. They ran through many attempted solutions, in a variety of formats, including spreadsheets, Confluence pages, internal wikis, and readme files. The biggest issue: no solution was ever comprehensive or up to date.

The many iterations did help the team realize a larger problem:

“We never really defined institutionally what ownership entailed. There wasn’t a clear sense of like, Hey, when you own this thing, these are your responsibilities towards the thing that you own.” – Ben Johnson, Software Architect at Prodigy

Around the same time, Prodigy recognized that their engineering teams needed a north star, synthetic metric that could be used within their OKR planning framework.

They got to work building their own tooling for a scorecard system. But completing the scorecard was tedious and manual. And engineering leaders who filled out the scorecards were too far removed from service owners who actually owned services and would need to take corrective action.

The Decision

Is an Open Source Framework Worth the Investment?

By their own admission, Prodigy is a strong proponent of CNCF. So they were interested in open source project Backstage and turned to it to improve their catalog and scorecard tooling.

“But it really seemed to us like it was a framework for building a service catalog and not a service catalog,” said Johnson.

On top of being more framework than tool, Prodigy learned Backstage was operationally challenging and lacked any specific functionality for tracking and improving service quality.

They also considered the opportunity cost: how much engineering time would it cost them to customize and maintain the Backstage framework for their service catalog?

“Honestly, we found OpsLevel makes this really freaking easy and having the internal time to work on the scorecard is hard enough,” said Johnson. “Trying to deal with fixing Backstage is just not something we want to spend time doing.”

Ultimately they concluded Backstage wasn’t mature enough for their needs, especially given the investment required.

The Outcome

Keeping the Main Thing the Main Thing with OpsLevel

Selecting OpsLevel gave Prodigy an accurate registry of all their services and the ability to define and track their synthetic engineering quality metric.

Importantly, it also allowed Prodigy engineers to spend time on the actual work of owning their services–improving their reliability, security, and overall maturity–instead of needing to spend cycles building internal tooling.

There were a variety of quick wins, including identifying and tracking library versions across their architecture and supporting ISO compliance efforts through maturity checks.

OpsLevel has also driven bigger-picture cultural changes. Engineering management now uses their definition of ownership, codified in OpsLevel, before green-lighting new services, ensuring teams aren’t overburdened and services aren’t neglected or orphaned.

“I think about it kind of like pressure testing… you’re adding some friction and you’re adding some pressure, but pressure makes diamonds, right,” said Johnson.

Reporting in OpsLevel has also proved to be a value-add for Prodigy, especially thanks to Groups which allows leaders to see–and then act on–granular data on the maturity of the exact services their teams own.

Individual service owning teams have embraced service maturity data too. It’s easier for them to spend time on non-functional work when they can point to concrete progress they’ve made, using a language shared across the organization. No one spends time justifying or explaining their non-functional work. They just say We leveled-up from a C to a B this quarter.