A complete guide to building and managing internal APIs

Are you using Internal APIs? In 2002, the legend goes, Jeff Bezos issued “The API Mandate.” It read something like this:

It was a call to use internal APIs exclusively to share information between groups. Why did Bezos make this call? What are internal APIs and how can they help you? How should you build and manage them in your company?

Let’s find out.

Internal APIs

What’s an Internal API?

An internal API is a programmatic interface for sharing information between teams inside a company. They’re often used to share sensitive resources within an organization’s IT infrastructure.

Internal APIs are only accessible inside the organization because they carry sensitive data that the company can’t share with external parties, or simply because the infrastructure required to share it doesn’t exist.

Why Use Internal APIs?

There are several good reasons to use internal APIs:

• An API makes it possible for clients to get the data they need when they need it, as many times as required.
• Internal APIs standardize the connections between internal applications and their data sources.
• With an API, teams share data with each other asynchronously. This scales better than ad hoc relationships and data sharing.

Let’s look at a quick use-case for internal APIs. Imagine an IT group that maintains access control lists for internal computer systems and applications.

In order to keep their lists updated, they need a list of changes from human resources that includes:

• new hires
• firings and resignations
• transfers
• promotions

Without an API, the two departments need to coordinate sharing the information.

Here are some common options:

• Copy the IT department on human resource emails. This puts the onus on IT to interpret the message and risks leaking confidential employee information.
• Send a periodic list of changes. This introduces lag and the potential for errors and omissions.
• Give IT access to the HR database. This risks security problems and coupling between HR’s DB schema and the IT application.
• Send IT a periodic database dump. This also introduces lag.

With an API, the IT application can poll for changes when it needs them, and the only coupling between the two departments is the API. Both teams can make changes on “their side” without creating problems for the other.

Internal vs External APIs

The obvious difference between internal and external APIs is that external APIs are available to clients outside the organization. But this single attribute has far-reaching and serious implications.

Security

Every API should require authentication and implement access control. But, many teams build their internal APIs quickly, with little or no security at all. This obviously isn’t ideal, and won’t work with our example above, since access to HR data requires controls.

But launching with no security is never an option with an external API, especially if you’re building one that monetizes the data. You need to know who is connecting, and the API needs to control access to specific data sets based on what they’re paying for.

Of course, as soon as you bill customers, you need to manage payments. Even if you outsource the billing to a third-party system, it requires security that an internal API doesn’t need.

Scalability and Reliability

All APIs fail. Sometimes it’s because of a systems outage, other times it may result from unexpected demand. They may even fail because of an attack, so you need to consider security and reliability as closely coupled concerns.

When your external API fails, customers lose access to your service. This means angry clients, lost revenue, and, depending on the scale of the outage, bad public relations. So, your external API needs to scale to withstand demand and be resilient against hardware failure and attacks.

While it’s also possible for an internal API to experience unexpected demand and outages, communications between internal departments are easier and less fraught. When there’s an outage because of a failure or a heavy load, you can work with your teammates to solve the issue.

Support

Recovering from outages and demand spikes are part of support, but they’re not the only considerations. External APIs often require developer support in the form of a support team and documentation. Many organizations expect an internal API customer to fend for themselves.

Internal API Best Practices

Now that we’ve contrasted internal and external APIs, let’s look at some ways you can make sure your internal API runs smoothly and serves your internal customers well.

Internal API Security

Even though you may get away without robust security controls for your internal API, there are several reasons you should secure it with authentication, access controls, and encryption. You should consider securing your internal APIs as if they are external.

We briefly touched on one reason above: internal APIs often carry sensitive information. In the example above, we described an internal API that could carry Personal Identifiable Information (PII) about employees. Internal APIs can also contain similar information about clients, or other sensitive information such as billing data, sales records, or business secrets.

Transmitting that information between departments in the same company isn’t risk free. If a malicious employee or contractor has access to the API, they might retrieve the data and misuse it. So, being able to identify who accessed the API is important. So is implementing role-based access control (RBAC) so that you can limit who can retrieve sensitive data.

Internal API Documentation

Just like you might skimp on security, it’s easy to postpone or just ignore internal API documentation. But if you do, you’re making it harder for people to use your system and end up creating extra work for everyone.

Poor documentation makes it difficult for developers to use your API. When they hit a roadblock integrating your data into their systems, they’re left scratching their heads. Are they using your code incorrectly? Have they found a bug?

So, they either give up, or they ask for support. Neither reaction is what you want.

In almost every case, creating and maintaining API documentation is easier than the alternative.

Catalog Internal APIs

An API is only useful if people use it. If they do, you want to know so you can make sure it stays up and running. If they don’t, you’ll want to either improve it, let more people know that it’s there, or focus your efforts on the things that your customers need.

API catalogs help with these efforts. They help people discover APIs and also help you track which ones your clients use most. Catalogs also prevent duplicate work, since they act as an enterprise hub for API activity.

You can catalog internal APIs to:

• Avoid duplicated effort  
• Publicize and grant access to developers  
• Track API usage

Internal APIs Improve Efficiency

In this post, we’ve seen what internal APIs are, how they differ from external APIs, and how those differences affect how you manage them. Then we discussed a few best practices for maintaining internal API connections.

Internal APIs are a powerful tool for streamlining communication between teams. If you’re looking for a tool that will help you manage them, get in touch today.

This post was written by Eric Goebelbecker. Eric has worked in the financial markets in New York City for 25 years, developing infrastructure for market data and financial information exchange (FIX) protocol networks. He loves to talk about what makes teams effective (or not so effective!).