Page Comparison

Introduction

This wiki explain the current design and implementation of tracking and monitoring collections. The challenges we have at scale and the proposed design to handle them.

Background & Problem Statement

The sunbird platform supports collection tracking and monitoring. It uses the below APIs to capture the content tracking data, generates progress and score metrics and provide the summary.

Content State Update API - To capture content progress and submit assessment.
Content State Read API - To read the individual content consumption and assessment attempts status.
Enrolment List API - To access all the enrolment metrics of a given user.

The content state update API capture the content progress and assessment data. It generate events for score and overall progress computation by activity-aggregator and assessment-aggregator jobs.

We have a single API (Content State Update) to capture all the tracking information. So, it has a complex logic to identify the given input is for content progress or assessment submission and etc,.

Key Design Problems:

Single API to capture all the tracking data.
Read after write of consumption data and basic summary.

Design:

Extended Enrolment Consumption:

Every instance adapting the sunbrid will have to select one of the option from different consumption modes, this would allow the application to mange the user in avoiding the consumption of content more than once based of specific predefined rules
Mode for any instance will be one-one mapping
Tracking and monitoring of the user consumption can be done for any new context like program, event etc

Image Removed

Image Added

Following are the different modes provided to new instance:

Scenario

Write Request

Read Request

1

Carry Forward Consumption

The content consumed is marked as complete irrespective of context

Code Block

language	js

{
    "userid": "<<userid>>"
    "collectionid" : "<<courseid>>",
    "contentid" :"<<contenid>>"
}

Note: Progress will be captured directly under the context

Code Block

language	json

{
  "userId": "<<userid>>",
  "collectionId": "<<courseid>>",
  "contentId": "<<contentid>>"
}

2

Copy Forward Consumption

The content consumed is marked as complete along with new entry in the database for the context

Code Block

language	js

{
    "userid": "<<userid>>",
    "collectionid": "<<courseid>>",
    "contextid": "<<programid>>",
    "contentid": "<<contentid>>"
}

Code Block

language	json

{
  "userId": "<<userid>>",
  "collectionId": "<<courseid>>",
  "contentId": "<<contentid>>"
}

3

Strict Mode Consumption

The content will be consumed as new one every time

Code Block

language	js

{
    "userid": "<<userid>>",
    "collectionid": "<<courseid>>",
    "contextid": "<<programid>>",
    "contentid": "<<contenid>>"
}

Viewing Service:

Viewing Service collects the “content view updates” and generate events to process and provide summary to the users.

When a user starts viewing a content, a view entry created. There are three stages when a user view the content. They are start, progress and end. Considering these three stages we have 3 API endpoints to capture this information for each stage.

An event will be generated when a content view ends. The summary computation jobs will read these event to process and compute the overall summary of the collection.

The computed summary will be available from API interface to download and view.

Summary Computation Jobs - Flink:

The Flink jobs are used to read and compute the summary of a collection consumption progress when the user view ends. It also computes the score for the current view and best score using all the previous views.

Info
The event is just a trigger to initiate the computation of the collection progress. The job fetches the raw data from DB to compute the overall progress. When an assessment type content (Ex: QuestionSet) view ends, it expects the ASSESS events data to assessment submit API for score metrics computation.

Once the view ends, the progress and score will be updated asynchronously by the flink jobs.

Content Consumption Flow:

Assessment Consumption Flow:

Viewer-Service - Content Consumption Scenarios:

The user can consume a content by searching it in our platform (organically) or via a collection when the user enrolled to a course.

With Viewer-Service, we will support tracking individual content consumption also. Below details explain how the data will be stored for a content consumption in different scenarios.

The below table has various scenarios considering the current and future use cases. Here we defined the database read/write logic to support these use case and fetch the save or fetch the required data from user_content_consumption table.

Table - user_content_counsumption

PRIMARY KEY (userid, collectionid, contextid, contentid) [userid, courseid, batchid, contentid]

Key words used in below table:

Carry forward content consumption - Considering the content consumed in any context to compute the progress or completion percentage (any collection, batch or individual content consumption).


{
    "userid": "<<userid>>",
    "contentid": "<<contentid>>"
}
{
    "userid": "<<userid>>",
    "collectionid": "<<courseid>>",
    "contextid": "<<batchid>>",
    "contentid": "<<contentid>>"
}
{
    "userid": "<<userid>>",
    "collectionid": "<<courseid>>",
    "contextid": "<<courseid>>",
    "contentid": "<<contentid>>"
}
{
    "userid": "<<userid>>",
    "collectionid": "<<courseid>>",
    "contextid": "<<batchid>>",
    "contentid": "<<contentid>>"
}
{
    "userid": "<<userid>>",
    "collectionid": "<<courseid>>",
    "contextid": "<<courseid>>",
    "contentid": "<<contentid>>"
}

Content View Lifecycle:

When the user view the content in context of a collection and batch, for the first time its start, progress update and end triggers are processed. Revisit (2nd - nth view) of the content will be ignored to process and update the DB.

Note
Shall we enable force ‘view end’ to handle the collection progress update sync issues?

Info
View Start API should insert the row only if the row not exists. View Update and End API should update the row only if the row exists.

Handling collection and batch dependencies:

For view start, end and update, courseId and batchId are non-mandatory. This would enable to track the progress for any content which is not part of a course.
This is handled in two ways:

If, collectionId and batchId are part of the request, then, individual content progress and overall collection progress is captured and computed.

In case of only userId and contentId, the progress is captured only for that content

Handling Collection Data types in DB:

With normal collection types, the map values gets distributed to multiple sstables with append, which might lead to read latency issues
To the handle the scenario, will consider the frozen collection types, which will helpful in avoiding tombstone and multiple sstable reads

Current vs New (Viewer-Service) APIs:

We need to continue supporting the current APIs before deprecate and delete. So, it requires to work with both the APIs with backward compatibility.

Enhance Current APIs to read summary from aggregate table.

Enhance the below APIs to read the progress and score metrics from user_activity_agg table.

Enrolment List API.
Content State Read API.

One time Data migration:

The content status and score metrics data should be updated to user_activity_agg table from user_enrolemnts and assessment_agg table for all the existing enrolment records.

API Spec

Content View Start

Expand

title	POST - /v1/view/start

Request:

Code Block

language	json

{
    "request": {
        "userId": "{{userId}}",
        "collectionId" : "{{collectionId}}",
        "contextId": "{{batchId}}",
        "contentId": "{{contentId}}"
    }
}

Response:

Code Block

language	json

{
    "id": "api.view.start",
    "ver": "v1",
    "ts": "2021-06-23 05:37:40:575+0000",
    "params": {
        "resmsgid": null,
        "msgid": "5e763bc2-b072-440d-916e-da787881b1b9",
        "err": null,
        "status": "success",
        "errmsg": null
    },
    "responseCode": "OK",
    "result": {
        "{{contentId}}": "Progress started"
    }
}

Content View Update

Expand

title	POST - /v1/view/update

Request:

Code Block

language	json

{
    "request": {
        "userId": "{{userId}}",
        "collectionId" : "{{collectionId}}",
        "contextId": "{{batchId}}",
        "contentId": "{{contentId}}",
        "progressDetails": {}, // Progress details specific for each mimetype
        "timespent": 10 // Timespent in seconds
    }
}

Response:

Code Block

language	json

200 OK:
{
    "id": "api.view.update",
    "ver": "v1",
    "ts": "2021-06-23 05:37:40:575+0000",
    "params": {
        "resmsgid": null,
        "msgid": "5e763bc2-b072-440d-916e-da787881b1b9",
        "err": null,
        "status": "success",
        "errmsg": null
    },
    "responseCode": "OK",
    "result": {
        "{{contentId}}": "SUCCESS"
    }
}

4XX or 5XX Error:
{
    "id": "api.view.update",
    "ver": "v1",
    "ts": "2021-06-23 05:37:40:575+0000",
    "params": {
        "resmsgid": null,
        "msgid": "5e763bc2-b072-440d-916e-da787881b1b9",
        "err": ERR_Error_Code,
        "status": "failed",
        "errmsg": ERR_error_msg
    },
    "responseCode": "BAD_REQUEST"/"SERVER_ERROR",
    "result": {
    }
}

Content Submit Assess

Expand

title	POST - /v1/assessment/submit

Request:

Code Block

language	json

{
    "request": {
        "userId": "{{userId}}",
        "collectionId" : "{{collectionId}}",
        "contextId": "{{batchId}}",
        "contentId": "{{contentId}}",
        "assessments": [{
            {{assess_event}} //Mandatory for self-assess contents
        }]
    }
}

Response:

Code Block

{
    "id": "api.view.assess",
    "ver": "v1",
    "ts": "2021-06-23 05:37:40:575+0000",
    "params": {
        "resmsgid": null,
        "msgid": "5e763bc2-b072-440d-916e-da787881b1b9",
        "err": null,
        "status": "success",
        "errmsg": null
    },
    "responseCode": "OK",
    "result": {
        "{{contentId}}": "SUCCESS"
    }
}

Content View End

Expand

title	POST - /v1/view/end

Request:

Code Block

language	json

{
    "request": {
        "userId": "{{userId}}",
        "collectionId" : "{{collectionId}}",
        "contextId": "{{batchId}}",
        "contentId": "{{contentId}}"
    }
}

Response:

Code Block

language	json

{
    "id": "api.view.end",
    "ver": "v1",
    "ts": "2021-06-23 05:37:40:575+0000",
    "params": {
        "resmsgid": null,
        "msgid": "5e763bc2-b072-440d-916e-da787881b1b9",
        "err": null,
        "status": "success",
        "errmsg": null
    },
    "responseCode": "OK",
    "result": {
        "{{contentId}}": "Progress ended"
    }
}

Content View Read

Expand

title	POST - /v1/view/read

Request:

Code Block

language	json

{
    "request": {
        "userId": "{{userId}}",
        "contentId": ["do_123", "do_1234"],
        "collectionId" : "{{collectionId}}", //optional
        "contextId": "{{batchId}}"   // optional
  
    }
}

Response:

Code Block

language	json

{
    "id": "api.view.read",
    "ver": "v1",
    "ts": "2021-06-23 05:37:40:575+0000",
    "params": {
        "resmsgid": null,
        "msgid": "5e763bc2-b072-440d-916e-da787881b1b9",
        "err": null,
        "status": "success",
        "errmsg": null
    },
    "responseCode": "OK",
    "result": {
    	"userId": "{{userId}}",
    	"collectionId": "{{collectionId}}",
    	"contextId": "{{batchId}}",
        "contents": [{
          "identifier": "{contentId}",
          "progress": 45,
    	  "score": {{best_score}},
    	  "max_score": {{max_score}}
        }]
    }
}

Content Assesment Read

Expand

title	POST - /v1/assessment/read

Request:

Code Block

language	json

{
    "request": {
        "userId": "{{userId}}",
        "contentId": ["do_123", "do_1234"],
        "collectionId" : "{{collectionId}}", //optional
        "contextId": "{{batchId}}"   // optional
   }
}

Response:

Code Block

language	json

{
    "id": "api.assessment.read",
    "ver": "v1",
    "ts": "2021-06-23 05:37:40:575+0000",
    "params": {
        "resmsgid": null,
        "msgid": "5e763bc2-b072-440d-916e-da787881b1b9",
        "err": null,
        "status": "success",
        "errmsg": null
    },
    "responseCode": "OK",
    "result": {
    	"userId": "{{userId}}",
    	"collectionId": "{{collectionId}}",
    	"contextId": "{{batchId}}",
        "contents": [{
    	  "score": {{best_score}},
    	  "max_score": {{max_score}}
        }]
    }
}

Viewer Summary - All enrolments

Expand

title	GET - /v1/summary/list/:userId

Response:

Code Block

language	json

{
  "id": "api.summary.list",
  "ver": "v1",
  "ts": "2021-06-23 05:59:54:984+0000",
  "params": {
    "resmsgid": null,
    "msgid": "95e4942d-cbe8-477d-aebd-ad8e6de4bfc8",
    "err": null,
    "status": "success",
    "errmsg": null
  },
  "responseCode": "OK",
  "result": {
    "summary": [
      {
        "userId": "{{userId}}",
        "collectionId": "{{collectionId}}",
        "batchId": "{{batchId}}",
        "enrolledDate": 1624275377301,
        "active": true,
        "contentStatus": {
          "{{contentId}}": {{status}}
        },
        "assessmentStatus": {
          "assessmentId": {
            "score": {{best_score}},
            "max_score": {{max_score}}
          }
        },
        "collection": {
          "identifier": "{{collectionId}}",
          "name": "{{collectionName}}",
          "logo": "{{logo Url}}",
          "leafNodesCount": {{leafNodeCount}},
          "description": "{{description}}"
        },
        "issuedCertificates": [{
          "name": "{{certName}}",
          "id": "certificateId",
          "token": "{{certToken}}",
          "lastIssuedOn": "{{lastIssuedOn}}"
        }],
        "completedOn": {{completion_date}},
        "progress": {{progress}},
        "status": {{status}}
      }
    ]
  }
}

Viewer Summary - Specific enrolment

Expand

title	POST - /v1/summary/read

Request:

Code Block
{ "request": { "userId": "{{userId}}", "collectionId" : "{{collectionId}}", "contextId": "{{batchId}}" } }

Response:

Code Block

language	json

{
  "id": "api.summary.read",
  "ver": "v1",
  "ts": "2021-06-23 05:59:54:984+0000",
  "params": {
    "resmsgid": null,
    "msgid": "95e4942d-cbe8-477d-aebd-ad8e6de4bfc8",
    "err": null,
    "status": "success",
    "errmsg": null
  },
  "responseCode": "OK",
  "result": {
        "userId": "{{userId}}",
        "collectionId": "{{collectionId}}",
        "contextId": "{{batchId}}",
        "enrolledDate": 1624275377301,
        "active": true,
        "contentStatus": {
          "{{contentId}}": {{status}}
        },
        "assessmentStatus": {
          "assessmentId": {
            "score": {{best_score}},
            "max_score": {{max_score}}
          }
        },
        "collection": {
          "identifier": "{{collectionId}}",
          "name": "{{collectionName}}",
          "logo": "{{logo Url}}",
          "leafNodesCount": {{leafNodeCount}},
          "description": "{{description}}"
        },
        "issuedCertificates": [{
          "name": "{{certName}}",
          "id": "certificateId",
          "token": "{{certToken}}",
          "lastIssuedOn": "{{lastIssuedOn}}"
        }],
        "completedOn": {{completion_date}},
        "progress": {{progress}},
        "status": {{status}}
  }
}

Viewer Summary Delete

Expand

title	DELETE - /v1/summary/delete/:userId?all - To Delete all enrolments

Response:

Code Block

language	json

Response: 
{
    "id": "api.summary.delete",
    "ver": "v1",
    "ts": "2021-06-23 05:37:40:575+0000",
    "params": {
        "resmsgid": null,
        "msgid": "5e763bc2-b072-440d-916e-da787881b1b9",
        "err": null,
        "status": "success",
        "errmsg": null
    },
    "responseCode": "OK",
    "result": {}
}

Expand

title	DELETE - /v1/summary/delete/:userId - To Delete specific enrolments

Request:

Code Block
{ "request": { "userId": "{{userId}}", "collectionId" : "{{collectionId}}", "batchId": "{{batchId}}" } }

Response:

Code Block

language	json

Response: 
{
    "id": "api.summary.delete",
    "ver": "v1",
    "ts": "2021-06-23 05:37:40:575+0000",
    "params": {
        "resmsgid": null,
        "msgid": "5e763bc2-b072-440d-916e-da787881b1b9",
        "err": null,
        "status": "success",
        "errmsg": null
    },
    "responseCode": "OK",
    "result": {}
}

Viewer Summary Download

Expand

title	GET - /v1/summary/download/:userId?format=csv

Response:

Code Block

{
    "id": "api.summary.download",
    "ver": "v1",
    "ts": "2021-06-23 05:37:40:575+0000",
    "params": {
        "resmsgid": null,
        "msgid": "5e763bc2-b072-440d-916e-da787881b1b9",
        "err": null,
        "status": "success",
        "errmsg": null
    },
    "responseCode": "OK",
    "result": {
      "url": "{{userId}}_viewer_summary.csv"
    }
}

Expand

title	GET - /v1/summary/download/:userId

Response:

Code Block

{
    "id": "api.summary.download",
    "ver": "v1",
    "ts": "2021-06-23 05:37:40:575+0000",
    "params": {
        "resmsgid": null,
        "msgid": "5e763bc2-b072-440d-916e-da787881b1b9",
        "err": null,
        "status": "success",
        "errmsg": null
    },
    "responseCode": "OK",
    "result": {
      "url": "{{userId}}_viewer_summary.json"
    }
}

Conclusion:

<TODO>

Version	Old Version 31	New Version 32
Changes made by	Revathi Kotla	Revathi Kotla
Saved on	Aug 23, 2021	Aug 23, 2021

Page Comparison

Versions Compared

Key

Introduction

Background & Problem Statement

Key Design Problems:

Design:

Extended Enrolment Consumption:

Viewing Service:

Summary Computation Jobs - Flink:

Content Consumption Flow:

Assessment Consumption Flow:

Viewer-Service - Content Consumption Scenarios:

Content View Lifecycle:

Handling collection and batch dependencies:

Handling Collection Data types in DB:

Current vs New (Viewer-Service) APIs:

Enhance Current APIs to read summary from aggregate table.

One time Data migration:

API Spec

Conclusion: