Two Forgotten Aspects to an API Design First Approach
How APIs are Being Used to Disrupt the Shipping and Logistics Sector
In an API Design First approach, the contract (or API specification) is documented before coding begins. This helps ensure the API focuses on a great developer experience for the end API user throughout the creation process, and helps create testing parameters that continually align with the core focus of the API, even if development processes veered off track once coding to solve technical challenges started.
But besides organisational issues like advocating internally for an API strategy and building API team leadership, there are two initial steps that come before what is traditionally thought of as “API First”: defining the data model and understanding the business model.
Looking at how the shipping and logistics sector is currently being disrupted — both from within, and from new market players, using APIs — provides a great insight into these two tasks that are essential to robust API design.
For the past year at least, both Dennis Dortland, Innovation Consultant at Portbase, the logistics information hub for Rotterdam and Amsterdam Ports, and Shan Lian, Product Marketing Manager at Shippo, a multi-carrier shipping API, have been focused on re-introducing these two often-forgotten aspects of API first design into their business’ API development processes.
Both are disrupting the shipping and logistics sector in their own way: Portbase by trying to reorient ports to be smart information hubs, and Shippo by abstracting unstructured shipping concepts to match real-life operations.
Defining The Data Model
Portbase is the information hub for all 4,000-plus users of the Amsterdam and Rotterdam ports: carriers, carrier agencies, port authorities, customs, and others.
Dutch ports bring in a large proportion of all European goods, and Dortland’s job is to help manage those imports in a rapidly changing environment where Industrial Internet of Things technologies are seeing a lot more data points come in, where regulations on CO2 emissions require new tracking, and where sectors like retail are bringing in new players to the market. Portbase’s mission is to introduce APIs to help make Dutch ports the smartest in the European Union. After having already reoriented their 45-something port services to achieve a completely paperless port, Portbase now has to reorient those services again so that they are API-driven.
“The tech is not an issue, the tech is so mature, there are so many good devs and so many good modules and libraries. The business transformation is the main work,” explains Dortland.
Dortland says the various carriers, carrier agencies and customs departments are all familiar with APIs, so they do not have to educate third party developer communities about the API advantage. Instead, the key initial work is to develop a data model that meets all the various needs of the different stakeholders, and that can identify where open data should be shared amongst all players and where individual companies need to share data with the port, but be confident that it is kept private as it has a competitive advantage for the supplier.
“If you look at all the cargo streams, there is a lot of optimisation able to be done there, but you need to know the customers’ deadlines, so you need to have a lot of data about customer demands. What is the capacity of barges and rail? What delays are expected? How is weather going to create risks in the network? What if your customer has a deadline, but also wants to be eco-friendly in their choice of transport options? All these small factors influence the decision, so we also need to share information on what options they have. It consists of many small packets of data that need to be shared across all parties,” says Dortland.
Dortland says some of the initial data modelling work is around trying to figure out what data is clearly able to be labeled open data, and which parts of the information system will be proprietary, encouraging stakeholders to share their data with Portbase but also confident that it is only used to optimise logistics and is not accessible to other providers. The increased competition within logistics, as well as fears that the tech giants will swoop in and take market share, is bringing the value of having access to the best data into sharp focus.
At Shippo, Lian has a similar take: it is not the tech that is the hold up here. “We need to abstract shipping concepts for developers so that they can understand how shipping operations work in real life,” says Lian. “There is a big gap between technology and what happens on the ground, for example, how pickups work in a warehouse, how do they split different orders in the backend. Are there are different parcels per shipment and how do you portray that in your database? For example, you might have a multi-piece shipment, like a table, and sometimes it fits into three boxes: tabletop, legs and screws. Even though they have the same tracking number, you need to account for how they fit into multiple parcels. We’ve had to abstract concepts like that in our databases and in the API for developers using Shippo to build their own shipping backends..”
Lian says growing their developer community has meant having clear documentation that explains all the different data fields in a shipping order, what their purposes are, and helping them translate that into their own systems. “The biggest gap between developers and operations is that you can build an integration and think its great until your postal collection arrives and says this is filled out incorrectly. For example, for multiple pickups, a carrier might say ‘Actually you need to give me a form that will scan one barcode for all 20 items.’ There are lots of human edge cases that are hard to control, so the physical aspects are very important,” Lian says.
In order to continually build their shipping API that abstracts all the individual idiosyncrasies of each shipping carrier, Lian says that Shippo ensures that they always have a physical client or partner who can help them test in a real-world warehouse before rolling out the carrier integration. They also officially certified each integration by the carriers themselves to again, ensure that no edge cases are missing.
Defining The Business Model
Making an API available is often done after there is a clear business case to introduce it, whether that be to increase sales or widen partnerships (revenue motivations), or for internal efficiencies (cost reduction motivations).
But introducing APIs also means opening up to facilitating new programmable business models, and often these need to be thought through with current partners and customers in tandem with engineers writing the API code.
At Portbase, Dortland sees the introduction of APIs as having huge implications for the business models of their stakeholder network. Changing the current paperless services to API-enabled services means that more granular data packets are defined in their data model, which in turn opens up new business models for their stakeholders in how they use that data, whether it is proprietary and they can sell it for competitive advantage, or whether it is open data that they can ingest or buy from another stakeholder to gain a competitive lead. Dortland says a big part of the conversations he has with stakeholders is to help them identify what data items should be opened, and which might inadvertently make competitors smarter than a business wants. “So trust plays a really big part of it. Our 4,000 customers trust us and share their data with us. Trust is the biggest factor involved in building our API platform.”
In order to identify the business model impacts and to maintain the trust they have fostered, Dortland says Portbase is currently working on small projects, each with two companies in the logistics chain that need to share more data. “For example, when a container is coming in, at the gate of terminal will it be picked up by truck, barge or rail? No matter the modality, the terminal and the collecting company need to know when the container will be released, are invoices paid, is it cleared by customs. The port wants to know when the container will be picked up. This is the type of situation where we want to provide more realtime information, so we are working with APIs with pub/sub subscription models,” Dortland says.
Again, Lian says Shippo are seeing their API as influencing business models of partners in a similar way. Bigger marketplace customers working with a range of vendors may want each vendor to sign up for the Shippo API rather than take on the seller’s shipping payment risks themselves. In other cases, warehouse inventory management systems are finding that they are winning new e-commerce store customers because they are able to offer shipping as part of their platform through the Shippo API..
The Shippo API is also creating more granular, predictable cost calculations for users which in turn impacts on a business’ operational budgets and decision-making. “A lot of carrier APIs don’t give you the cost of the shipment upfront,” explains Lian. “Often times, shipment rates are only calculated when the package is scanned and then you’d receive one consolidated bill at the end of the month. At Shippo, we’ve built a shipment ratings engine so you can upload your shipping rate table so that you can directly see the cost of the shipment when purchasing the label.” This level of accurate insight into cashflow can help a business reconsider its own business models and how much risk or new business it can take on, or how to better manage its monthly cashflow resources in order to maximise business opportunity.
APIs as Business and Technical Concerns
These experiences show again how APIs straddle both business and technical development. What the examples from shipping and logistics show is that creating an API is not a hand-off process: it is no longer an issue of defining the business case and documenting the business requirements and then having the technical team build the API.
It is a much more iterative process between technical and business, where API providers must continue defining the data model and testing potential business model implications as the API is being built and into its implementation.
