In our journey to successfully design and build both internally and externally consumed APIs for many clients over the years, we have learned a lot of valuable lessons. We would like to share the five biggest learnings for you to benefit from – without having to find out the hard way ☺️.
Know your target audience
It’s not by accident that we call the applications that run our APIs ‘services’ as they do exactly that; they ‘serve’ our consumers (or at least they should). A common pitfall for API designers is to design APIs to their personal tastes, without focusing enough on what your consumers want or need. Too many times we’ve seen APIs being built with the best intentions at heart but still completely missing the mark for consumers, ultimately leading to limited adoption and rework. Spend time to get to know your consumers, it will pay dividends if you do.
Something to consider here as well is that you can potentially have different target audiences you need to facilitate. For one of our consumers we’d have a target audience of businesses that wanted to connect their companies to the platform we exposed. They were generally not developers themselves, so the people doing the actual implementation formed a different group altogether. If this is the case, make sure you focus on both areas. Spend time to optimise the user experience by providing the right features at the right time. Spend time to optimise the developer experience by providing adequate documentation, standards and ease of integration. Focus on both audiences; both the developers and the businesses you want to attract to your API have specific needs that require attention.
Not all consumers are created equal; especially when exposing a public API you will find that it’s all about compromise. You will hardly ever be in a position to meet every consumer demand, so it’s important to always try and find the best possible compromise that will work as well as possible for everyone.
Standardize!
Look at things from the perspective of your consumers. Sometimes you need to implement a certain feature and need to use a piece of software that is maintained by another person, team or maybe even an entity outside of your organisation. For instance, say you need to validate if an address exists against an external API containing the addresses available in a certain country or region. If that piece of software does not follow a predictable structure or enforces standardised ways to do certain things, the journey to figure out how to get the job done can be infuriatingly frustrating. Don’t do this to your consumers.
Define standards for API endpoint structure. Define standards on how your models are structured and exposed, standards for error and validation messages, proper security standards and so on. Prefer open standards (such as OAuth, REST, HTTP, GraphQL) over custom solutions to ensure interoperability across a variety of consumers. Enforce and follow those standards as strictly as is possible. Ideally, if a consumer implements one endpoint, they should be able to also implement the others with great ease as they have seen it all before (and are able to re-use logic on their end as well).
A common practice is to communicate these standards on a developer portal. Explain your choices and why they were made and follow them to the letter yourself. Many companies have done this before you have and have in some cases published their vision on this (for reference, look at the bol.com retailer API or the Zalando REST style guide). In the end, it is up to you how much detail you put into the standards, but when it comes to these kinds of documents: less is more.
Versioning and release planning
As we all know, ‘change is the only constant’. Embrace this, but plan for it as well. Make sure you properly take care of versioning, by deciding ahead of time what changes are allowed to be made within the scope of a version (such as adding non-breaking changes that will not affect existing consumers). Also decide how to apply versioning, and at which granularity. For example, having a version per (group of) endpoint(s) can become unnecessarily complex for consumers. For a recent project we opted to have a global, sweeping version number as this makes it easier for consumers to know whether or not to upgrade and saves them time having to deduce this for each and every endpoint.
Another thing that became abundantly clear: not every organisation can keep the same pace of development as you can. Some organisations are not even IT centric, they integrate with your API to fulfil their business requirements and have to incidentally hire IT capacity to help them do this. Have a predictable release schedule, a strict cadence of releases, communicate clearly and ahead of time when certain versions will not be supported anymore. This allows organisations to plan and budget for needed IT capacity and gives them room to, say, ‘skip a version’ if nothing of value has been added, while still making sure they understand the urgency of eventually migrating because supporting each and every version produced is not a feasible option.
For organisations that feel that this approach limits them too much, consider adding a beta version. In general, we considered a released version to be immutable, only allowed to change if not doing so would cause great damage (generally not the case though). If you want to roll out early or experiment with a part of the API, provide a beta version that conveys this to your consumers. It allows your consumers to opt-in early if they are able and gives you room to make modifications during the beta phase if needed, while others can wait until the features become generally available. Key part is (again) communicating your intent so that your consumers are aware of the consequences when they adopt a certain version.
Measure, measure, measure!
When exposing an API, especially publicly, you do not always get the insights you need to make choices on how to evolve your product. Consumers, or people in general, are mostly outspoken on the things that they find troublesome, but will far less often communicate their positive experiences without being prompted. To make sure you know what’s going on at all times, make sure you add ways to measure the (lack of) success of your API. Measure how many calls are being made. Measure if there are inefficiencies in handling certain calls (think of long request processing times – maybe some things can be optimised by retrieving data in parallel). Identify which calls are more common than others. Identify adoption of versions that have been released. Identify functionality that has low or no usage so you can decide to deprecate and ultimately remove it (or, conversely, promote it more).
Besides measurements providing you with a baseline to make functional choices, there’s also value in measuring on the technical side. For a recent project, we kept a close eye on the metrics and grew very capable in predicting issues based on usage patterns changing in the metrics. In a lot of cases we were able to prevent a lot of calls to customer services by pro-actively spotting issues, notifying consumers and in some cases even fixing them before consumers really noticed. As customer service calls can potentially be costly, optimising here can be very beneficial.
Another aspect of being able to provide good service to your consumers is providing functional logging and using things like distributed tracing. More often than not, consumers experience issues with the provided API which is either to them misunderstanding intent or errors on the API side. Having some form of tracing allows you to identify what happened, to replay the activity in a representative environment and fix bugs. In our case we were able to close a lot of support cases quickly because from the logs alone it was already relatively easy to spot mistakes and help consumers fix or optimise their implementations.
Embrace your community
All of the points that were raised before have one thing in common: communication. Exposing APIs is a lot about being clear and open about your intentions, what you do and also do not offer, who to contact whenever there are issues, et cetera. Having a direct line of communication can be vital to the success of your API and its consumers. Make sure you invest in the right tools (build your own developer portal, use documentation platforms, use Github pages, etc.) to support this. In recent times it has become very common to have a developer portal to expose API specifications, to expose technical documentation and to document standards. Additionally, document the functional side of things as well, especially since you might not always be dealing with a target audience of only developers. There are many great tools available allowing you to build an API portal to contain all this information.
Don’t be afraid to ask for feedback. Contact your consumers if you know them directly or organise surveys to verify whether or not the services you are providing are still bringing enough value. Identify the blind spots and try to address them. Also do this for the developer experience side of things, as they may have a vastly different experience when it comes to the services you expose. Open yourself up for this, take it seriously and use it to improve your proposition.
If possible, also involve the services of a (technical) writer, as we cannot overstate the importance of proper documentation. Being developers ourselves at Sourcelabs, we have experienced first-hand how difficult it can be to deal with tools or APIs that are poorly documented. Make documentation part of your development cycle. Periodically review it for completeness and correctness.
That’s all we need to consider?
Sadly no, not by a longshot. There are many things that come into play when building (great) APIs, but we think this list of five items describes the most important items to think about when embarking on this journey. We’ve discovered, it’s not just about technology – although it does play an important part. It’s easy to get stuck into using (new) tools and frameworks and completely lose sight of who you are doing this for.
Did you find this an interesting perspective on APIs? Are there things you are interested in or struggling with and would like us to cover them in more detail? Would a follow-up article be of interest? Keep an eye on our website for more content like this in the future.