What is API design?

API design refers to the process of developing application programming interfaces (APIs) that expose data and application functionality for use by developers and users. APIs are important to modern organizations, adding new capabilities to everything from their operations and products to their partnership strategies. It’s no longer a stretch to say that most organizations don’t ask whether to engage in API programs, but how to do so. 

An effective API program has to build on an organization’s overarching corporate strategy and contribute to its objectives. You’ll know you have the makings of a great strategy when you can answer the following 3 questions in a clear way:

1. Why do we want to implement APIs?
2. What concrete outcomes do we want to achieve with these APIs?
3. How do we plan to execute the API program to achieve that?

The why

People often misinterpret this question in different ways. Firstly, rather than focus on the value of the API, it’s helpful to think of the value of the effect of the API. Remember, it’s the organization’s core business that’s valuable, not necessarily the API. An API is valuable when it becomes a channel that provides new types of access to the existing value an organization delivers.

Another common misconception is believing that for an API to be valuable users must be prepared to pay for it. This is true only if the API itself is the product. In most models, this is not the case. APIs are usually driving some other metric—sales, affiliate referrals, brand awareness, etc. The value of the API to users is the result of an API call (service request and response), rather than the call itself.

The most common business drivers for establishing an API program, according to a survey of 152 organizations conducted by the Cutter Consortium and Wipro, are to develop new partnerships, to increase revenue, to exploit new business models, to improve time to market, and to develop new distribution channels. The top technology drivers are to improve application integration, improve mobile integration, and support the connection to more devices. The benefits to the organization need to be strong enough to make the decision to invest in the APIs an obvious choice for the organization.

The what

The second question should be “What concrete outcomes do we want to achieve with these APIs?” In other words, “What do the APIs actually do and what impact do they have on the wider business strategy?”

Both the concepts of the internal view and the external view of an organization can help to define the what of the API. The internal view refers to specific, valuable assets an organization possesses. The more valuable and unique the services and resources offered the more suitable they are for an API program.

An organization that has unique data could take advantage of this resource by allowing access to the data via API. Unique content, data, and services can make access to the API extremely valuable.

When deciding what an API should do for a business, both internal and external views
need to be examined. The decision about the what is then usually a combination of the 2 views.

In concrete terms, while the why is unlikely to change often, the what may vary significantly based on external factors—such as markets, technical considerations, or economic conditions. Internal directions about the value of an asset may change, which could also affect what should be achieved with an API.

The how

The final question, “How do we design our API program to achieve what we want?” is all about implementation and execution.

Teams must ask themselves:

• What technology is used to build the APIs?
• How are the APIs designed?
• How are the APIs maintained?
• How are the APIs promoted inside the organization or marketed to the outside world?
• What resources are available?
• Who should be on the team?
• How do we track success against the business goals that have been set?

An API team is most closely related to a “product” team—whether your customers are internal or external, you are in charge of building, deploying, operating, and optimizing the infrastructure others depend on.

Just like product teams, API teams can also be very diverse, but typically they should include a product-centric person who acts as the keeper of strategy and goals, design-focused team members who ensure best practice in API design, engineers who put the API technology in place, and operations team members who will run the API.

Over time you may also have additional people involved, including support and community team members, API evangelists, security representatives, and others.

John Musser highlighted 5 “keys” to a great API in his 2012 talk at the O’Reilly Open Source convention:

1. Provide a valuable service
2. Have a plan and a business model
3. Make it simple, flexible, and easily adopted
4. It should be managed and measured
5. Provide great developer support

The first key, provide a valuable service, is especially important when thinking about the why of your API program The value proposition is the main driver for success of the API. If an API has the wrong value proposition (or none at all) it will be very difficult or impossible to find users.

Almost any company with an existing product, digital or physical, can generate value
through an API, if that API links to existing offerings and enhances them. As long as the API is structured in a way that covers meaningful use cases for developers, it will deliver value.

Finding and describing the value of your API is an iterative process. The first step is describing jobs your users are trying to get done. For example:

• Automatically sending urgent communications to team members in an emergency
• Backing up critical files to ensure they are never lost
• Collecting sample data to detect certain events

The next step is identifying particular challenges that affect users before, during, or after trying to get a job done:

• Ensuring the reliability of sending with multiple tries, detecting failures, worrying about many messages being sent rather than just one, and integrating with different message delivery systems depending on the location of the user
• Ensuring the safe delivery of files, but also minimizing the amount of transfer bandwidth
• Dealing with massive amounts of data and attempting to correlate it in real time

The third step is to summarize the potential gains a user could achieve:

• Sending other types of notifications, which create opportunity rather than warn of threat
• Getting rid of other storage equipment if reliability meets your needs
• Automatically triggering actions based on the events

When examining these pain points, think broadly and list things like support, documentation, or developer portals—everything that a customer could use. Next, outline how you intend to eliminate or reduce some of the things that may be annoying to API users before, during, or after trying to complete a job—or issues that prevent them from doing so. Then describe how you intend to create gains of any sort for your API users.

Through engaging in this process, our 3 examples above might result in:

• A multichannel messaging API with a single call to deliver messages and the ability to retry automatically until arrival is guaranteed (e.g., Twilio, PagerDuty).
• A storage synchronization API with optimized calls to efficiently check if new versions should be synchronized (e.g., Bitcasa, Box).
• An API aggregating several data sources into a configurable stream, which could be filtered, sampled, and easily manipulated (e.g., GNIP, DataSift).

Finally, a useful clarification exercise is to compose several statements that make the fit between the API and the user profile clear. If you find it hard to identify such fit statements, then the API model needs to be reconsidered. Maybe there are API features which need to be added, revised, refined, or eliminated. It could also be that your API does offer great value, but you are trying to address the wrong type of users.

When you condense and abstract your fit statements into one overarching statement, it becomes your value proposition for your APIs. In the case of the messaging API above this might be something like:

Our messaging API provides enterprise developers a reliable, guaranteed, no-latency text messaging functionality for highly-critical business applications. The API is also supported by software development kits (SDKs) covering the most popular programming languages for quick integration.

In some cases you might think “This seems like too much work. We’re just creating an internal API.” However, focussing on value is key, even in internal use cases. A poorly determined value proposition will lead to difficulty pitching the value of the API to other teams. A well-defined value proposition can help ease adoption and make the API program a key contributor to the business.

To help define your own API program’s value, consider these 5 questions:

1. Who is the user? This question should be answered in terms of their relationship to you (are they existing customers, partners, external developers), their role (are they data scientists, mobile developers, operations people) and their requirements or preferences.
2. What user pains are we solving and/or what gains are we creating for the user? This question should be answered in relationship to the customer’s business, challenges and gains defined by the value proposition, and whether or not a critical need is being fulfilled (is it a pain point, is it a revenue opportunity), and what metric is being improved for the user (speed, revenue, cost saving, being able to do something new).
3. Which use cases are supported with your API? Identify, with the help of the value proposition, the solutions to your user’s challenges or opportunities created by the API  that are most effective your organization and the user. Plan your API to address these use cases.
4. How can the value for the user be expanded over time? Plan your value proposition with future changes in mind. What are important upcoming milestones relating to internal or external changes?
5. What value is being created for your organization internally? Consider internal benefits and how the API can be of value within the business.

Being able to articulate the value of an API is an important milestone in designing your API-based program. However, APIs also generate cost and this consideration should be balanced by value. While the value may not be measured in monetary terms, it must be real, for example:

• What is the existing core business of the organization?
• How an API can be used to accelerate or augment this business?

In some cases APIs can lead to entirely new business opportunities outside of the existing business model of an organization. Even in these cases, APIs generally use existing assets or expertise to create opportunities in new ways.

In summary, there are 3 reasons why determining the right business model is important for designing effective API programs:

1. Determining the right business model brings the value of the API to the organization into focus, which drives the decision regarding long-term commitments to the API program. Without that commitment there are rarely resources in place to complete the tasks required for establishing and running an effective API program.
2. Determining the right business model helps to define the functionality of the product, which is needed to satisfy third parties and generate business.
3. Determining the right business model ensures attention is paid to roles and responsibilities within an organization, and to who retains which parts of the value generated by the API. This also implies defining what users of the API gain from use of the API and how that balances against what the API provider gains.

Good API design has some core principles, which may differ in implementation. Here’s an analogy: every car has a steering wheel, brake pedals, and an accelerator. You might find that the hazard lights, the trunk release, or radio are slightly different from model to model, but it’s rare that an experienced driver can’t figure out how to drive a rental car.

This level of “ready-to-drive” design is what great API teams strive for—APIs which require little or no explanation for the experienced practitioner to begin using them.

Simplicity

Simplicity of API design depends on the context. A particular design may be simple for one use case but very complex for another, so the granularity of API methods must be balanced. It can be useful to think about simplicity on several levels, including:

• Data format. Support of XML, JSON, proprietary formats, or a combination.
• Method structure. Methods can be very generic, returning a broad set of data, or very specific to allow for targeted requests. Methods are also usually called in a certain sequence to achieve certain use cases.
• Data model. The underlying data model can be very similar or very different to what is actually exposed via the API. This has an impact on usability, as well as maintainability.
• Authentication. Different authentication mechanisms have different strengths and weaknesses. The most suitable one depends on the context.
• Usage policies. Rights and quotas for developers should be easy to understand and work with.

Making an API simple may conflict with making it flexible. An API created with only simplicity in mind runs the risk of becoming overly tailored, serving only very specific use cases, and may not be flexible enough for other use cases.

To establish flexibility, first find out what the potential space of operations is based on, including the underlying systems and data models, and defining what subset of these operations is feasible and valuable. In order to find the right balance between simplicity and flexibility:

• Try to expose atomic operations. By combining atomic operations, the full space can be covered.
• Identify the most common and valuable use cases. Design a second layer of meta operations that combine several atomic operations to serve these use cases.

Arguably, the concept of hypermedia as the engine of application state (HATEOAS) can further improve flexibility because it allows runtime changes in the API and in client operations. HATEOAS does increase flexibility by making versioning and documentation easier, however, in API design, many questions must be considered.

In order to think through your API design, consider the following 5 questions:

1. Have we designed the API to support our use cases? The next step after identifying the main use cases is to design the API so that it supports these use cases. Flexibility is important so as not to exclude any use cases that may be less frequent, but should still be supported to allow for innovation.
2. Are we being RESTful for the sake of it? RESTful APIs are quite fashionable, but you shouldn't follow this trend just for the sake of fashion. There are use cases which are very well suited for it, but there are others that favor other architectural styles, such as GraphQL.
3. Did we expose our data model without thinking about use cases? An API should be supported by a layer that abstracts from your actual data model. As a general rule, don’t have an API that goes directly to your database—although there may be cases which require that.
4. Which geographic regions are most important and have we planned our datacenters accordingly? API design must also cover nonfunctional elements, such as latency and availability. Make sure to choose datacenters that are geographically close to where you have most of your users.
5. Are we synchronizing the API design with our other products? If the API is not the sole product of your business, make sure that the API design is coordinated with the design of the other products. It may be that you decide to completely decouple API design from other products. Even if this is the case, plans to decouple API design from other products needs to be made clear and communicated both internally and externally.

A key metric to improve API design for easy adoption is the “Time to first hello world” (TTFHW). In other words, how long does it take a developer to reach a minimum viable product with your API? This is a great way to put yourself in the shoes of a developer who wants to test your API to see what it takes to get something working.

When you define the start and end of the TTFHW metric, we recommend covering as many aspects of the developer engagement process as possible. Then optimize it to be as quick and convenient as possible.

Being able to go through the process quickly also builds developer confidence that the API is well organized, and things are likely to work as expected. Delaying the “success moment” too long risks losing developers.

In addition to TTFHW, we recommend another metric: "Time to first profitable app" (TTFPA). This is trickier, because “profitable” is a matter of definition, depending on your API and business strategy. Considering this is helpful because it forces you to think about aspects related to API operations as part of the API program.

The 2 underlying principles of developer experience are:

1. Design a product or service that provides a clear value to developers and addresses a clear challenge or opportunity. This can be monetary value or some other value, such as a way to increase reach, brand awareness, customer base, indirect sales, reputation for the developer, or simply the joy of using great technology that works.
2. The product needs to be easily accessible. This can include having a lightweight registration mechanism (or none at all), access to testing features, great documentation, and a lot of free and tidy source code.

We suggest that most API programs should have a developer program—regardless of whether you expose your APIs publicly, to partners only, or internally only. The provisions may be more or less elaborate depending on the audience.

Developer portal

The developer portal is the key element of a developer program; this is the core entry point for developers to sign up, access, and use your APIs. Getting access to your API should be simple and easy for developers. They should be able to get started quickly.

TTFHW is the best metric to measure this. You should also consider streamlining the sign-up process—the simpler and quicker, the better. A recommended best practice is that developers should be able to invoke your APIs to examine their behavior (request and response) without any sign-up at all. Also, supplementary content—such as getting started guides, API reference documentation, or source code—are great to lessen the learning curve.

Acceleration via ecosystem partners

As an API provider you are operating in an ecosystem of partners and vendors. These partners often have their own content distribution and communication networks and means. We recommend identifying alliances, which can be effective in helping to increase the adoption of your API. Often such alliances can be found when APIs are complementary and provide value to developers when combined.

Questions to consider to assess your developer experience:

1. How do we explain the value of the API in the first 5 minutes? Develop an “elevator pitch” about the value proposition of your API that best speaks to developers.
2. What is our TTFHW and TTFPA and how do we reduce it? This is a powerful way to improve the developer friendliness of your API by thinking about the end-to-end TTFHW. We recommend keeping the TTFHW and TTFPA metrics in mind when considering any elements that are added to the developer experience (like portals), and every aspect  of the API that changes.
3. What is the onboarding process for developers, and is it as easy as possible? This needs to be in-line with the use cases of your API. The level of security naturally needs to be higher for more sensitive APIs or data access, which probably needs more formal agreements. For everything else it should very simple and straightforward to allow for early developer success (TTFHW).
4. Are we allowing enough flexibility to make the API attractive for developers? It’s great if you’ve found the right value proposition, and developers sign up for your API. Keep in mind that helping them to be successful will retain and grow their numbers.
5. How do we support developers if they face problems? We believe in the self-service approach, which will help you to scale. Many developer questions can be covered by good documentation, FAQs, or forums. But self-service has its limits, and for more in-depth questions or other complications, like invoice problems, there should be some type of support mechanism in place.
6. Can our documentation support innovation? What support is there for developers who deviate from the normal use cases or wish to do something new? Great ideas can come from anywhere.