Cutting through the dev portal noise: How to find the right IDP for your needs

Engineering organizations are under more pressure to move faster than ever before—but you need to ensure your team can be agile without breaking things along the way. Gartner estimates that organizations lose $6M per year on outages and downtime. And if you move too slow, they also estimate businesses lose out on $99M each year due to delays in release cycles. So how do you balance speed and standards?

Enter: The internal developer portal (IDP). But before you line up calls with vendors, you must determine what’s important, what's nice to have, and what’s mission-critical in a new tool. We’ve got your back. In this resource, we’ve laid out all the nuances of an IDP and give you the tools to build your own set of criteria while venturing down the path of developer portals.

What is an Internal Developer Portal? 

First, let’s start with the basics. An internal developer portal (IDP) provides a central place for developers to discover, use, and manage.

To be clear, it’s not simply an API catalog. It comprises more than a list of API specs and documentation. It’s a central place for developers to go and learn about the systems that they work on and interact with. It provides self-service tools to get devs integrated quickly and easily. And it gives folks a place to go when they have questions.

And while sometimes the terms are used interchangeably, a developer portal is not the same as a developer platform. Your developer platform might consist of tools like PagerDuty, Sentry, Datadog, Honeycomb, etc. And while those tools might be connected, your developers likely have to hop from one to the other or hunt down access to these tools to get complete service information. A developer portal brings all those things together so that if you're working on a service, you have a unified view of all of the tooling connected to that service and can easily access the info you need.

In truth, there’s a lot of overlap between these two concepts and we’ve covered them in depth in previous posts.

A developer portal helps engineers across the org answer questions like:

• How do I get started developing for my service? 
• What other services exist in the company and who owns them? 
• Where do I go to find help?

While the name is “developer portal” the users who benefit from this technology span DevOps, platform engineering, and site reliability engineers as well as the product developers.

Use cases for Internal Developer Portals 

The benefit of an IDP is that the use cases are broad. And with most vendors allowing custom integrations, the platform engineer or DevOps owner has a lot of room to customize their portal.

Some common use cases include:

• Discover and catalog your software ecosystem
• Find internal APIs and document your own 
• Find service owners and log service issues 
• See status of your services + other services 
• Track service maturity over time 
• Perform self-service operations against running infrastructure 
• Create new services with templates
• Manage org-wide changes such as an upgrade 
• See how standards are being adopted by teams and the org as a whole over time
• Improve your developer experience (DevEx)

What an Internal Developer Portal is not 

Since developer portals are fairly new and since use cases can be broad, there’s been some confusion about the extent of capabilities. So, we thought we’d clarify a bit and share what an IDP is not:

An API Developer Portal

• There’s some overlap with an Internal Developer Portal in that you can find API endpoints for services
• API Developer Portals are skewed to developing and testing APIs (think Postman or MuleSoft) and enabling other dev consumers to test-drive them
• Internal Developer Portals document your software at the service/package level, plus provide more functionality for onboarding new engineers and managing software quality in-team and cross-team

An Internal Developer Platform 

• Internal Developer Platform: “a set of tools that work together to provide flexible abstractions for developers to deliver software faster”
• Brings productivity and standardization to the internal dev process
• Think: source control, infra provisioning, CI/CD tooling 
• Not a hard line between the two - usually, you are building both and they work together in harmony
• The developer portal becomes a unified pane of glass tying platform tools together

A CI/CD deployment system

• A system for verifying changes and deploying them, safely and in an automated fashion, to production
• CI/CD systems should integrate with your portal - e.g., to show # of recent builds, status, test results, kick off build requests from the UI, etc. 

“Just” a service catalog

• Service catalogs were one of the big early use cases for internal developer portals - but they’ve become much more 
• Meant to serve as the single place to understand development and gauge ongoing quality and progress of your team’s SDLC 
• Limited to visibility and typically a homegrown or manual system of record

Should you build or buy your dev portal?

The age-old debate of building vs buying. Many engineering teams get excited about the prospect of building internal tools to aid their software development processes. Creating slick internal tools seems like a fun engineering challenge and a way to show off your technical chops. However, developing internal tools comes with many hidden costs that are often underestimated. 

A common problem that teams face is that they want to save the budget or think that it will be less expensive to build their own portal either through a homegrown solution or on top of Spotify’s Backstage framework. In both cases, a serious Total Cost of Ownership (TCO) analysis needs to be done. Often what people forget is that it’s not just about building, it’s also about maintaining and more importantly—making sure your team actually adopts the tool. 

There are also teams who think they have very unique needs that a SaaS solution can’t cover. If you have truly novel needs not addressed by vendors, then custom tools may be your only option. Sometimes internal tools can provide competitive differentiation. And occasionally there are no vendor solutions for your specific use case. But these situations are less common than we admit. Carefully evaluate if custom tools are absolutely necessary before deciding to incur the costs.

‍

Features of Internal Developer Portals 

An IDP needs to be as self-sustaining as possible. This means automation should be the tie that binds all the features and surface areas together. Without automation, you’re creating more toil for the service owners and platform teams to maintain the portal. The three key areas of a developer portal are:

• Software catalog
• Service maturity
• Self-service

Let’s dig into each of these a little more to understand the nuances.

Software catalog

As we mentioned earlier, developer portals are often confused as being “just a service catalog” but a true IDP goes beyond cataloging services and acts as the central source of information about your entire software ecosystem. The issue is the motto of "if you build it they will come and update it frequently" does not apply to a spreadsheet. Your IDPs service catalog should remove the administrative burden from your developers and update your services automatically via integrations. It should do things like tie together the same services with different aliases, etc, etc. Because if you're relying on manual updates, you're going to be very disappointed. A software catalog should pull together documentation, services, components like libraries, systems, and domains. We believe a software catalog should be:

Complete: Every service, component, and system is tracked.

• Automatically suggest service metadata
• Automatically surface new and existing services and components

Rich: All the metadata and knowledge, across all sources.

• Custom properties to track things outside of the service or component
• API and Tech Docs for every service
• Ownership of the service

Up-to-date: Not missing any additions or changes over time.

• Service change history
• Automation driving software and service detection

Service Maturity

Maturity in this context simply means the health and quality of your software and services. This surface area of an IDP should  help you do things like:

• Measure the health of all your services and components
• Check services for vulnerabilities and that standards are applied
• Run a time-bound campaign to do work like upgrades
• View team-specific service health alongside global health standards

Developer self-service

Within an IDP, developer self-service means unblocking development teams so they can get more done, faster. It’s great for developers because they have fewer operational roadblocks making their SDLC smoother.

Developer self-service features within a developer portal should help engineers:

• Execute operational tasks on their own, within standards and guidelines
• Create new services from scratch with pre-defined templates
• Gain visibility into their service ecosystem and what steps they need to take to improve service health

Each IDP is created a little differently, some who require a YAML-or-bust approach (creating more hands-on time for the admins and the service owners), and others who focus on automation to help reduce time spent on ownership. It’s important to understand your organization’s needs before choosing a vendor. 

‍

Team considerations when choosing an IDP

If you’ve ever rolled out a team-wide tool, you understand how critical it is to get your stakeholders involved in the procurement process. Not only does it make your job of finding the right tool easier, it’ll help with rollout and adoption if you have more internal champions at your back.

We typically see these roles involved in the vendor discovery and decision making process:

DevOps/Platform leaders: These folks typically feel the most acute pain from not having a developer portal. Likely, if you’re a platform or DevOps leader, you’ve probably come to hate the “who owns this service” messages in slack and been tracking services in a spreadsheet, creating a lot of processes, and need a better way to scale service ownership and health across your organization without weighing down your team. 

Site reliability engineering leaders: SREs often come to us after a big incident or downtime. They need to improve the performance of their software and the first step is often getting a handle on what’s out there and who owns it. They’re often most interested in the service maturity features and how they can improve health over time—and report those improvements up to senior leadership.

Product engineering leaders: VPs and Directors of Engineering often have marching orders from senior folks to improve output while reducing costs (a tale as old as time). They want to know that this investment will not only help them ship features and products faster, but also reduce downtime and costs associated. They’ll want to understand time to value of this investment and average adoption rate. An interesting metric that you should start tracking is the number of concurrent projects (per engineer). You can find out more on this metric in this podcast with CTO of DX, Laura Tacho.

Product engineers: It’s important to bring these folks into the mix when defining your IDP criteria since they’ll be early adopters and help champion the solution within their teams. If you spend a lot of time deciding on a solution but don’t consult the developers it’s meant to serve, you’ll risk missing key needs and ultimately risk poor adoption internally.

‍

Considerations during the procurement process

There are a host of things to consider while choosing the right developer portal for your organization. We’ve created this criteria matrix that you can modify to fit your needs.

A few key things to think about early are:

Is this an urgent need? If you need to start seeing the value of a developer portal (faster speed to market, less downtime), a DIY option (like Spotify’s Backstage framework) probably isn’t the best choice. It often takes teams 9+ months to build their IDP and then even longer to gain adoption. A SaaS developer portal is better if you need to make progress against goals within 12 months and the bonus is they’ll have customer success playbooks to help you drive internal adoption faster.

Do you have a lot of time to spend on maintaining the portal? Some SaaS offerings have a distinct infrastructure-as-code approach to their IDP causing admins and even service owners to dedicate a lot of time to keeping information accurate and up to date. If you need to start with YAML due to your current architecture, but eventually want to get to a more automated place, find a vendor who can do both simultaneously. This isn’t a “given” for all SaaS developer portals.

What’s your appetite for strict standards vs total team autonomy? If you’re like most organizations, you’re probably somewhere in the middle of that spectrum, so you’ll need ways to implement standards at both the team level and the global level. Make sure you can add checks and controls for each if this is important.

Having a clear understanding of what your organizational needs are and who should be involved in vetting solutions will set you up for success in making the right decision. 

‍

Takeaway

Making the case for an internal developer portal is one thing, but deciding on which solution to go with can be even more complex. Set yourself up for success with a little planning and a clear understanding of what you need before you start testing vendors.

Get started on your journey with a free trial of OpsLevel or book a time to get questions answered from our team of IDP experts.