Real-Time API Management with Apigee and Fanout
If you have a traditional RESTful API then you may already be using an API management system such as Apigee, 3Scale, Kong or Amazon API Gateway. Using an API management platform has several advantages such as:
- Managing API authentication
- Analytics and logging
- Caching and traffic manipulation (e.g. transforming JSON to XML or adding a CORS header)
- Developer portals and interactive documentation
- Ability to monetize an API
- An additional layer of security (such as API request throttling)
In a previous post I detailed how with Fanout Cloud (or the open-source Pushpin) it is possible to create HTTP streaming and WebSocket feeds such as those implemented on the Canadian Bitcoin Index.
However, whilst we have a real-time API it could also benefit from some of the features of an API management system such as detailed analytics and authentication (and if you are already using an API management system, simply integrating the real-time APIs into the existing setup). Whilst the process is very similar for any API management platform that are implemented as a reverse proxy, for the remainder of the post I will demonstrate how to integrate the Apigee Edge API platform to provide authentication and detailed analytics.
As Fanout and Apigee both operate as reverse proxies we can simply place Apigee between Fanout and our existing backend services. In such a scenario client requests are first routed to Fanout which proxies the request to Apigee which records our traffic and can perform additional tasks on the incoming request e.g. ensuring that a provided API key is allowed to access the resource. Apigee then proxies the request to our backend service which responds with the required headers to instruct Fanout to open a WebSocket connection with the client. As such the opening of all WebSocket connections pass through Apigee and we have greater visibility of our real-time API traffic.
Assuming you already have a Fanout (or Pushpin) endpoint the process of integrating Apigee can be completed in two simple steps.
1 — Create an API proxy at Apigee that maps to your existing backend
2 — Update Fanout to point to Apigee instead of the existing backend endpoint
For the purposes of this post we will add an authentication layer to the existing CBIX WebSocket feed which will be available at wss://realtime.cbix.ca/authenticated/trades
Apigee Edge Implementation
As a first step we need to add a new API proxy to Apigee with our required authentication scheme.
This new API proxy is accessible at https://cbix-prod.apigee.net/authenticated which maps to the apigee.cbix.ca domain on our backend server. In the Security section we will choose API Key as the authorization mechanism. Now to access this API we will need to provide an API key in the request (by default this will be a URL query parameter of apikey).
In order to test the API we will need to create a sample API product (you may create this when creating the API proxy), add a developer and sample app with access to the created API Product. Doing so will give us a valid API key to use in future requests (which will be the Consumer Key of the created application).
Finally, we can test the Apigee part of the implementation with a request to https://cbix-prod.apigee.net/authenticated/trades?apikey=your-api-key which should return a response from the backend service to instruct Fanout to open a WebSocket connection.
HTTP/1.1 200 OK
Sec-WebSocket-Extensions: grip
Content-Type: application/websocket-eventsOPEN\r\n
Fanout Implementation
Next we need to update Fanout so that requests to realtime.cbix.ca are proxied to Apigee rather than directly to our backend.
If you wish to use SSL then you will need to add a certificate to Fanout that matches the domain name (realtime.cbix.cain this case). You will also need to enable the Rewrite Host Header option, which causes the host header to be rewritten to use the hostname of the origin server, so that Apigee will correctly respond to any requests made through Fanout.
Now, attempting to access the websocket API via wscat using the following command wscat -c wss://realtime.cbix.ca/authenticated/trades will return an unauthenticated error response as expected, as we have not provided any authentication information.
error: Error: unexpected server response (401)Providing our API key and making a request to wscat -c wss://realtime.cbix.ca/authenticated/trades?apikey=your-api-key correctly opens and maintains the websocket connection.
connected (press CTRL+C to quit)And after a few seconds we receive our first message through the websocket connection.
{
"exchange":"QuadrigaCX",
"exchange_link":"http://www.quadrigacx.com",
"volume":0.06552763,
"price":1502.93,
"value":98.48,
"date":"2017-02-21T19:13:14+0000",
"transaction_id":394131
}Summary
We now have a working authenticated real-time WebSocket feed and all connection requests are routed through Apigee. As such, we can make use of a number of other features of the Apigee Edge system for example adding a message logging policy, provisioning developer accounts or utilizing Apigee’s analytics platform to return detailed reports regarding your real-time API traffic.
