Kong App ID Plugin

Problem

Different application consuming the API’s but the backend has no idea who is consuming it and from where the request coming is from. 

Solution

This problem can be solved if an application sends its ID with an API request.

Implementation

To implement this unique App ID can be issued to each application that they should send with an API request.

App ID Validation

The validation will be needed to verify if the application sending the correct App ID which is assigned to them. This validation can be done in Kong API gateway by writing the custom plugin. Before forwarding the request to upstream services plugin will validate App ID by getting the consumer ID from JWT plugin and then APP ID mapped to the consumer ID from the database. If the App ID sent with a request is same as a mapped for a consumer in DB then request will be forwarded to upstream service otherwise it will throw 403 access forbidden error with the error message `Invalid app ID`.

Performance

The plugin may need to frequently access datastore on every request to get the APP ID associated with the consumer for validation. This would happen on every request, and it would be very inefficient:

• Querying the datastore adds latency on every request, making the request processing slower.
• The datastore would also be affected by an increase of load, potentially crashing or slowing down, which in turn would affect every Kong node.

Solution

To avoid querying the datastore every time, we can cache custom entities in-memory on the node, so that frequent entity lookups don't trigger a datastore query every time, but happen in-memory, which is much faster and reliable than querying it from the datastore (especially under heavy load).

A. Lazy caching: Cache custom entities in-memory on the node on the first request, so that frequent entity lookups don’t trigger a datastore query every time (only the first time), but happen in-memory. 

• Pros
  • Size of cache will be less as it will cache the records that are in use.
• Cons
  • Every first request will hit the datastore.
  • For invalid app id request will hit the datastore every time. 

 B. Full table cache on kong service start: Cache the all the records exist in the table on every node immediately after kong service start, so entity lookups don’t trigger a datastore query every time.   

• Pros
  • No lookup on the data-store except initial so it will help for reducing latency in request response time. 
•  Cons   
  • If the database is size is large then the cache and invalidation process will take time. 
  • Every Node will query datastore after the start of service that will put a load on data-store for some time. 
  • It will cache the all records even if record not in use and that will unnecessarily increase cache size on every node. 

Cache Invalidation

Every time a particular record is being updated, deleted in the datastore cache invalidation can be done, we can explicitly remove the cached record from the cache to avoid having an inconsistent state between the datastore and the cache itself. Removing it from the in-memory cache will trigger the system to query the datastore again, and re-cache that record. Caching will be done against consumer id so whenever particular consumer record updated only that record will be invalidated and re-cached. 

Implementation 

Database table structure 

Enabling the plugin on an API

Configure this plugin on an API by making the following request:

$ curl -X POST http://localhost:8001/apis/{api}/plugins \
--data "name=appid"

Associating App ID to Consumer

$ curl -X POST http://localhost:8001/consumers/{consumer}/appids \ 
 --data "appid={appid}"

{
 "appid": "{appid}",
 "Consumer_id": "e1a622e8-c351-432a-83f5-a4e0e300449c",
 "Created_at": 1541056142000,
 "Id": "9cad374f-006c-4ec2-aa3e-cdca58873c47"
}

Delete App ID Associated to Consumer

$ curl -X DELETE http://localhost:8001/consumers/{consumer}/appids/{appid}

Retrieve the App ID for mapped to Consumer using the following request:

$ curl -X GET http://localhost:8001/consumers/{consumer}/appids

{
   "data":[
      {
         "created_at":1541419735000,
         "consumer_id":"e1a622e8-c351-432a-83f5-a4e0e300449c",
         "appid":"Portal",
         "id":"1b8e6843-7ecd-4b52-8349-a8159f22f8eb"
      },
      {
         "created_at":1541498560000,
         "consumer_id":"e1a622e8-c351-432a-83f5-a4e0e300449c",
         "appid":"NTP",
         "id":"1c96fd3a-0be4-4302-aeeb-eeb7d409ed14"
      },
      {
         "created_at":1541998382000,
         "consumer_id":"e1a622e8-c351-432a-83f5-a4e0e300449c",
         "appid":"Sunbird",
         "id":"888d3266-5450-4d0c-856e-502875d41c16"
      }
   ],
   "total":3
}

Retrieve all the App ID for all Consumers using the following request:

Note: Kong upgrade required

$ curl -X GET http://localhost:8001/appids

{
    "total": 3,
    "data": [
        {
            "appid": "diksha",
            "created_at": 1511391159000,
            "id": "724d1be7-c39e-443d-bf36-41db17452c75",
            "consumer_id": "89a41fef-3b40-4bb0-b5af-33da57a7ffcf"
        },
        {
            "appid": "sunbird",
            "created_at": 1511391162000,
            "id": "0905f68e-fee3-4ecb-965c-fcf6912bf29e",
            "consumer_id": "c0d92ba9-8306-482a-b60d-0cfdd2f0e880"
        },
        {
            "appid": "ekstep_portal",
            "created_at": 1509814006000,
            "id": "ff883d4b-aee7-45a8-a17b-8c074ba173bd",
            "consumer_id": "c0d92ba9-8306-482a-b60d-0cfdd2f0e880"
        }
    ]
}

You can filter the list using the following query parameters:

Retrieve consumers associated with the App Id

Note: Kong upgrade required

$ curl -X GET http://localhost:8001/appids/{id}/consumer

{
  "created_at":1507936639000,
  "username":"foo",
  "id":"c0d92ba9-8306-482a-b60d-0cfdd2f0e880"
}