CGAP API Dashboard: Methodology

How Data is Collected and Analyzed

This page describes how each individual metric in the CGAP API Dashboard was calculated, what it measures, its benefits and limitations and how we can continue to improve the accuracy of each metric.

The CGAP API Dashboard relies on developer engagement metrics from a number of private and public sources. Calculating levels of engagement for API providers is a relatively new activity, and there are different opinions about how best to calculate data.

Our metrics system is constantly being reviewed and updated to try and reflect the actual level of engagement and business success of each API provider. We invite contributions from API providers who are willing to share their own data or metrics approaches in order to improve our measurement systems.

The metrics in the CGAP API Dashboard have been researched by API experts, focus-tested with API stakeholders and business experts, refined, and tested with target end users. We welcome comments and feedback from all CGAP API Dashboard users.


Main Dashboard Page metrics

APIs by Region

What this measures This map shows the countries where open APIs with the potential to improve financial inclusion are being made available, and the number of APIs available in those countries
Data source API provider’s developer portal/website
How it is calculated Each API provider’s developer portal is reviewed and details of where the API functions is tallied. Where no information is given, it is assumed that the country location of the API provider is the country where the API can be used. In some cases, where it has not been specifically mentioned, there are blog posts that describe users of the API being from a particular country, in which case those countries are tallied up in the totals.

Each API is listed with what countries it operates in. This is tallied together from the API pages and presented in the map on the home page.
Advantages This map gives an instant visual of which regions in the world are seeing more companies opening APIs.
Limitations Some of the APIs may be available in more regions that are not described in the API documentation.
How this metric can be improved API providers could clearly state in their documentation where the API is operative. API providers who review their data in the CGAP API Dashboard can contact us to let us know where our data can be improved.

Total APIs in Sector

What this measures An aggregate total of all open APIs made available by Digital Finance Services in Africa, South Asia, East Asia and Latin America, including APIs from other providers that may be relevant to financial inclusion.
Data source API provider’s website/developer portal
How it is calculated Regular internet searches are conducted to identify APIs that can be catalogued. The CGAP API team reviews some APIs that are not directly related to financial inclusion and makes an assessment of what should be included.

APIs are regularly reviewed from email alerts, expert suggestions, and internet research. APIs are entered in the database and the total number of APIs is summed from all entries in the CGAP API Dashboard and presented.

Advantages As a new industry opening APIs, there are few examples of Digital Financial Services providers opening APIs so it is useful to see what APIs are being made available amongst related industries and where DFS providers are starting to take strategic moves.
Limitations Because of the small number of DFS providers opening APIs, the majority of APIs currently listed in the CGAP API Dashboard are being made available by adjacent players such as payment aggregators, ecommerce, VoIP and even industries like delivery services. They do not reflect ONLY the Digital Financial Services providers.
How this metric can be improved A toggle capability will be introduced when there are more DFS provider APIs so that readers can view just the DFS APIs. For now, readers can search by tag to see the DFS only APIs.

A checklist can be prepared to identify whether the APIs should be listed as “relevant to financial inclusion”. Through the CGAP API newsletter and CGAP Advisory Committee, we can ask for more examples of open APIs being made available from DFS providers specifically. We can ask the developers in forums and slack groups of existing APIs what other APIs they are using. We continue to set up alerts and review potential open APIs being made available for financial inclusion in emerging markets.

We can write a blog post describing why some decisions were made to include certain APIs and how the could potentially be used to support financial inclusion.

Companies Offering

What this measures Number of Digital Financial Services and related industries who are opening APIs
Data source API provider’s website/developer portal
How it is calculated Based on the APIs listed in our catalog, we count the number of unique companies making these APIs available.
Advantages This shows an overall reflection of the level of activity in the market opening APIs, as showing only APIs may suggest a lot of activity but companies may release more than one API, so just looking at the number of APIs can create a distorted picture of the level of activity.
Limitations This score reflects all businesses opening APIs that may be relevant to financial inclusion, and the number of DFS providers opening APIs will be much lower.
How this metric can be improved A toggle capability will be introduced when there are more DFS provider APIs so that readers can view just the DFS APIs. For now, readers can search by tag to see the DFS only APIs.

Estimated size of developer community

What this measures Total addressable market of unique developers interested in using DFS-relevant APIs.
Data source Similar Web data is used to calculate monthly unique visits to the developer portal subdomain of websites for each company providing APIs.
How it is calculated The current month is calculated as: The total sum of each API provider (company)’s most recent full month’s number of unique visits to their developer portal.

The last month is calculated as: The total sum of each API provider (company)’s unique visits to their developer portal for the month previous to the most recent full month.

So for example, if looking at the CGAP API Dashboard on the 15th of January, the current month of data is from December, and the last month data is from November.
Advantages Helps build a business case by showing the potential number of external developers available to build products with open APIs.

In cases where Similar Web does not have agreement with providers to gain access to website analytics directly, the service uses an algorithm to calculate the actual number of unique visits to each developer portal. This is drawn from statistics where there is a specific subdomain for the developer portal, or where the company’s whole website is only focused on their APIs. If the API provider is a larger company with more services than just the APIs, but does not have a developer portal with a subdomain, this data may be missing or may be drawn from a subdomain for the technical documentation pages, which is usually much less frequented than a developer portal. The Similar Web algorithm may also be making assumptions about visitor numbers that do not accurately reflect the true number of unique developers that visit the website. In some cases, the data is also provided at a six week lag, so the current month data may read zero for some time which may show as a reduction in total developer community size compared with last month, which may not be accurate.

The total addressable market of developers (i.e. estimated developers consuming APIs) makes the assumption that every unique visitor who visits a company’s developer portal site is potentially interested in consuming the APIs but may not end up becoming developers due to other reasons. This assumes a website bounce rate of 0 and that all visitors are developers who would use the API if they had the right resources available to them. As a result, this may be overstating the actual size of the available developer community.

The CGAP API Dashboard does not show size of developer markets for each geographic region (although users can calculate this from the data available in the API Dashboard)
How this metric can be improved The SimilarWeb algorithm is constantly being improved so this production rate will become increasingly accurate over time. API providers are also invited to share their data directly on unique visits to their developer portal if they wish.

This metric should be read in conjunction with data showing active developers and commercial developers. (The CGAP API dashboard defines active developers as those who are engaged with the API provider’s user forums in some way, either their online forum or the API provider’s Slack account if it has one. Commercial developers are those who have produced an app built with an API provider’s APIs, have created a public integration, or who have said publicly they use the API in their businesses, as per below metrics).

Developer Mindshare

What this measures The most popular developer portals for DFS-related APIs.
Data source Similar Web data collected for the “estimated developers consuming APIs” metric aggregated for all companies.
How it is calculated For each company, the number of monthly developer portal unique visitors is divided by the total estimated number of developers consuming APIs. These scores are then ranked, with the top five API providers listed by their percentage of mindshare, and the rest grouped in “Other”.
Advantages This metric is indicative of which developer portals are most popular amongst the total addressable market and suggests which API providers are delivering the best engagement experience.
Limitations This metric assumes that out of the available developers, developers only use one company’s APIs. This is unlikely to be the case. In reality, developers probably use more than one company’s APIs in their products.

It is also based on the assumption that the developer portals with the most unique monthly visitors have the best developer experience.

A true measure of developer mindshare could be to rank API providers by the number of production apps that are built based on their APIs, however, this data is less transparent than developer portal visits and is insufficiently documented at present.

All limitations from the “estimated size of the developer community” are also an influence on this metric.
How this metric can be improved An additional metric that showed how many other APIs each developer was typically using could help introduce a weighting to mindshare to remove the assumption that each developer is unique to that API provider.

Improvements to the data collection of the number of commercial developers for each API and API company could help better reflect the real mind share of each API provider based on actual usage of their API instead of being based solely on developer portal visits.

Estimated number of apps built on DFS APIs

What this measures Number of apps built using DFS-related APIs
Data source API provider’s developer portal/website
How it is calculated For each API company, the list of apps mentioned on their developer portal and in their blogs and social media accounts is tallied.

This metric is the sum aggregate of each individual company’s list of apps.
Advantages Shows the capacity for an API provider’s APIs to be able to create commercial, production-level apps, including apps that improve financial inclusion.

Indicates that a third party market exists for releasing APIs that could generate new business for an API provider.

Suggests that third party developers could create apps to improve financial inclusion by using available APIs.
Limitations Measures total number of apps built using the DFS (or related) provider’s APIs, some of which may not have a financial inclusion impact.

Only tallies apps that are listed on the API provider’s website. The API provider may not announce some apps, or may not even be aware of apps that are available.
How this metric can be improved API providers could provide data directly on the number of apps created using their APIs via our feedback page. API providers could publicly announce and promote all the apps available built with their APIs, allowing us to count more accurately. We could reach out via each API provider’s community forum or slack channel to ask for details of any apps built.

If data on the types of apps was in sufficient depth, we could start calculating which apps being built have the potential to improve financial inclusion.

Estimated number of integrations built using DFS APIs

What this measures Number of publicly available integrations that makes an API provider’s open APIs available through another software service or product
Data source API provider’s developer portal/website
How it is calculated For each API company, the list of integrations mentioned on their developer portal and in their blogs and social media accounts is tallied.

This metric is the sum aggregate of each individual company’s list of apps.
Advantages Shows the capacity for an API provider to enter new markets and reach new segments through connecting to other services via API integrations.

Could signify that API providers are seeking to speed up accessibility to the data and functionalities being made available via their APIs.
Limitations Measures total number of integrations built using the DFS (or related) provider’s APIs, some of which may not have a financial inclusion impact.

Only tallies integrations that are listed on the API provider’s website. The API provider may not announce some integrations, or may not even be aware of integrations that are available.
How this metric can be improved API providers could provide data directly on the number of integrations created using their APIs via our feedback page. API providers could publicly announce and promote all the integrations available built with their APIs, allowing us to count more accurately. We could reach out via each API provider’s community forum or slack channel to ask for details of any integrations available.

If data on the types of apps was in sufficient depth, we could start calculating which apps being built have the potential to improve financial inclusion.

Estimated number of customers using DFS APIs

What this measures Number of known entities using an API provider’s available APIs
Data source API provider’s developer portal/website
How it is calculated Sum total of each company’s tally of logos of customers, individual customer case studies, testimonials, and stated number of customers mentioned on each API provider’s developer portal
Advantages In many cases, an API provider’s APIs do not lead to creating new apps or integrations, but enable new business processes or workflows to be created. This metric ensures those uses are also being tallied as a part of understanding the commercial use of an open API.
Limitations This data is based on publicly mentioned customers on an API provider’s website. There may be more customers that are not listed, or some customers that are listed may have stopped using the APIs.

This metric measures total number of known customers, not those who are using the API to improve financial inclusion.
How this metric can be improved API providers could provide data directly on the number of customers who use their APIs in production use cases via our feedback page. API providers could publicly announce and promote all production use cases created with their APIs, allowing us to count more accurately. We could reach out via each API provider’s community forum or slack channel to ask for details of any production use cases available.

If data on the types of production use cases was in sufficient depth, we could start calculating which ones have the potential to improve financial inclusion.

Top Communication Channels

What this measures Which developer resources should be considered “Minimum Viable Product” for any DFS related provider opening up APIs
Data source API provider’s website/developer portal
How it is calculated For each API provider, a full list of their developer communication channels is tallied on their company page. For the top five ranked API provider’s by mindshare, the communication channels they use are listed, showing which communication channels are the most important for the most popular developer portals.
Advantages This ranking of communication channels suggests to new open API providers what a minimum viable set of developer resources should be when attempting to engage developer communities, and helps API provider’s prioritize budget and resources.
Limitations This metric is based on unique visitors data rather than ranking of API providers with the most production apps, integrations and use cases.
How this metric can be improved This data could better reflect which combinations of communication resources were most common amongst the top ranking API providers. If data on commercial apps being built was more transparent, this data could be based on ranking of API provider’s with the biggest number of commercial developers rather than on unique visitor numbers to the API provider’s developer portal.

Apps built with DFS APIs

What this measures Types of apps being most regularly built with DFS-related APIs
Data source API provider’s developer portal/website
How it is calculated For each API company, the list of apps mentioned on their developer portal and in their blogs and social media accounts is listed on a spreadsheet describing the use case of the app. These use cases are then summarized into common categories.

This metric is the sum aggregate of each category for all apps built with the APIs.
Advantages Shows in broad categories the types of use cases being created using DFS-related APIs. Allows for some analysis of which apps being built may have an impact on improving financial inclusion.
Limitations Some apps may not have adequate use case descriptions to allow for categorizing. Each app can only be tallied in one category when it may have more than one use case.

The limitations of the estimated number of apps being built with DFS-related APIs are relevant to this metric.
How this metric can be improved More details on the apps being built with DFS-related APIs and their impact on financial inclusion will help better categorize apps.

Blog posts can be written to describe in greater detail the sort of apps that are being created with each company’s APIs and blog posts can describe in greater detail each set of category topics and what third party developers are building.

Business Models for DFS APIs

What this measures The potential revenue and future business models being enabled by open APIs
Data source API provider’s developer portal/website
How it is calculated For each API, it is listed whether the API is available free of charge, whether the developer pays for usage (for example, via a transaction fee for each API call made or via a subscription plan), whether the developer gets paid (for example, via affiliate commissions or a revenue-sharing model) or whether there is an indirect business model (for example, developers build for the API provider’s customer base or share some customer data from their integration).

This metric is the sum aggregate of each category of business model for all APIs.
Advantages Shows the level of current thinking amongst DFS-related providers regarding the potential revenue and business models from their open APIs.
Limitations Many open APIs do not publicly state the business model for their APIs at present. There are nuances within each category, for example, within “developer gets paid”, developers could be sharing in a revenue-split model for selling apps to a DFS provider’s existing customers. The DFDS provider strengthens their current customer relationships by providing a greater range of services while also sharing in revenue from new products sold to customers. This category setting does not fully show these revenue benefits being realized via the business model.
How this metric can be improved API providers could be more transparent about the current business model being used for each of their open APIs, or could share details via our feedback form.

Blog posts can be written to describe in greater detail the sort of business models that are being tested with open APIs.

APIs by Stage

What this measures The level of maturity of the open API market amongst DFS-related APIs
Data source API provider’s developer portal/website
How it is calculated For each API, it is tallied whether the API is available only to invited applicants/early adopters, whether any developer can sign up for test access, or whether it is available for production use cases.

All APIs are then aggregated by these categories and percentages for each category are calculated.
Advantages Shows the current level of maturity of the open API market, as DFS related providers may start with limited access and as developers begin testing their APIs, they may move to more general availability.
Limitations Several API providers do not explain whether their APIs are available at the production level. Others appear to be available for production use cases, but the lack of a pricing page or page outlining terms and conditions of usage make it difficult to clarify.
How this metric can be improved API providers could be more transparent about the current availability of their APIs by clearly stating in their API documentation or by providing details via our feedback form.

Blog posts can be written to describe in greater detail the maturity evolution of the DFS-related open API market. Additional data points showing the length of time that APIs stay in limited availability before moving to open beta and time between open beta and general availability would better reflect the API maturity model expected timelines.