Introduction

This wiki explain explains the current design and implementation of tracking progress and score monitoring collections. The of content and collections, the challenges we have at scale and the proposed design to handle them.

...

The sunbird platform supports monitoring of collection tracking progress and monitoringscore computation. It uses the below APIs to capture the content tracking progress data, generates progress and score metrics and provide the summary.

...

We have a single API (Content State Update) to capture all the tracking progress information. So, it has a complex logic to identify the given input is for content progress or assessment submission and etc,.

...

1. Single API to capture all the tracking progress and score data.

2. Read after write of consumption data and basic summary.

3. Data is written and fetched from multiple tables leading to consistency issues between API and Reporting Jobs

4. The low level tables (user_content_consumption & user_assessments) grow at an exponential rate when we start to track monitor everything

5. Archiving old data is not possible as the API’s read data from low level tables

...

To be able to handle the above design problems, we have analyzed how similar products (like netflix) track monitor everything what a user does (or views) and that too at scale. Based on the analysis we have broken down the APIs into more granular APIs with a single DB update so that each API can be scaled independently. In addition we have designed the APIs as a general purpose service (similar to asset service/graph engine) on which various use-cases can be mapped.

...

With Viewer-Service, we will support tracking individual content consumption alsoprogress monitoring at individual content level as well (i.e.: all types of content can be monitored for progress). Below details explain how the data will be stored for a content consumption in different scenarios.

...

While the viewer-service as an infra component provides flexibility to configure any tracking progress monitoring use-case, there is a need for preconfigured “modes” enabled at an instance level to be able to make better use of the service and also to make any solution or usecase simpler to implement and understand.

...

Any content or collection consumption is strictly tracked monitored at the context it was consumed. This is the default mode setup currently in SunbirdEd

...

...

1. There might be cases where a program is launched with a course which the user is already part of or completed. The user needs to redo the entire course again (consumption outside of Context will not be considered towards the specific Context)

2. User experience will be complex if the user sees two courses (one within a program and another outside of the program). The user might complete the course assuming it is part of program and would probably realise that he hasn’t taken it in the context of the program

3. There would be multiple steps required by user to reach a content within the context of a program. Will make the discovery and browsing experience complex. (may not always be possible to ensure that users ‘discover’ relevant content in the context they are supposed to)

...

Any content or collection consumption is tracked monitored at its level only and is carry forwarded into any context. This is the expected mode of SunbirdCB. In this mode there are two sub-modes as well:

Content Mode

In this mode we track monitor everything at a content level only. The collectionid and contextid would be irrelevant.

...

Collection Mode

In this mode we track monitor everything at a collection level only. The contextid would be irrelevant.

...

In this mode, any content or collection that is trackedmonitored, is copied over into context based on the business rules configured.

...

An user content progress is copied into a course or a program if the user has completed the content in the last 3 months or within the program/course duration

Copy

...

Content Mode

...

API responses for various other actions done by the user:

Pros

1. Opens up many complex use-cases

2. UX can be made simpler. A content or collection can be shown only once

Cons

1. The complexity on the server side explodes exponentially. There needs to be multiple asynchronous jobs like:

   1. Copy the progress after evaluating the business rules on actions like “enroll”

   2. Copy the progress continuously whenever a content/collection is taken outside the context but within the acceptable business rules

2. Explainability of the data becomes very complex. Debugging any issue would be a nightmare. Support would be bombarded with explainability requests

3. UX would become complex.

   1. User would be confused why a progress is complete or not complete

   2. If a collection is part of multiple programs, usability will be very challenging if the collection doesn’t satisfy the business rules

4. UX should be designed specifically for it

5. There are other complicated behavior like if the collection is complete and added to learner passbook, would the program also add another entry or how would the learner passbook experience be?

Copy Collection Mode

Entry in DB

API responses for various other actions done by the user:

Pros

1. Opens up many complex use-cases

2. UX can be made simpler. A content or collection can be shown only once

...

1. The complexity on the server side explodes exponentially. There needs to be multiple asynchronous jobs like:

   1. Copy the progress after evaluating the business rules on actions like “enroll”

   2. Copy the progress continuously whenever a content/collection is taken outside the context but within the acceptable business rules

2. Explainability of the data becomes very complex. Debugging any issue would be a nightmare. Support would be bombarded with explainability requests

3. UX would become complex.

   1. User would be confused why a progress is complete or not complete

   2. If a collection is part of multiple programs, usability will be very challenging if the collection doesn’t satisfy the business rules

4. UX should be designed specifically for it

5. There are other use-cases complicated behavior like if the collection is complete and added to learner passbook, would the program also add another entry or how would the learner passbook experience be?

...

In this mode, any collection that is trackedmonitored, is moved over into context based on the business rules configured. Move mode ensures that there is only one copy for the active collection (open batch or active batch). It doesn’t support moving of content as content would have taken in multiple contexts and moving the content will lose relevance in respective contexts

...

• Every new instance adapting the sunbird platform will have to select one of the option from 3 context modes, this would allow the application to mange manage the user in avoiding the consumption of content more than once based of specific predefined rules

• Mode for any instance will be one to one mapping

• With the extended design, tracking progress and score monitoring of the user consumption can be done for any new context like program, event etc

...

{
    "userid": "<<userid>>"
    "collectionid" : "<<courseid>>",
    "contentid" :"<<contenid>>"
}

{
  "userIduserid": "<<userid>>",
  "collectionIdcollectionid": "<<courseid>>",
  "contentIdcontentid": "<<contentid>>"
}

{
    "userid": "<<userid>>",
    "collectionid": "<<courseid>>",
    "contextid": "<<programid>>",
    "contentid": "<<contentid>>"
}

{
  "userIduserid": "<<userid>>",
  "collectionIdcollectionid": "<<courseid>>",
  "contentIdcontentid": "<<contentid>>"
}

{
    "userid": "<<userid>>",
    "collectionid": "<<courseid>>",
    "contextid": "<<programid>>",
    "contentid": "<<contenid>>"
}

...

For view start, end and update, courseId collectionId and batchId contextId are non-mandatory. This would enable to track monitor the progress for any content which is not part of a coursecollection.
This is handled in two ways:

• If, collectionId and batchId contextId are part of the request, then, individual content progress and overall collection progress is captured and computed.

...

{
  "id": "api.summary.list",
  "ver": "v1",
  "ts": "2021-06-23 05:59:54:984+0000",
  "params": {
    "resmsgid": null,
    "msgid": "95e4942d-cbe8-477d-aebd-ad8e6de4bfc8",
    "err": null,
    "status": "success",
    "errmsg": null
  },
  "responseCode": "OK",
  "result": {
    "summary": [
      {
        "userId": "{{userId}}",
        "collectionId": "{{collectionId}}",
        "batchIdcontextId": "{{batchId}}",
        "enrolledDate": 1624275377301,
        "active": true,
        "contentStatus": {
          "{{contentId}}": {{status}}
        },
        "assessmentStatus": {
          "assessmentId": {
            "score": {{best_score}},
            "max_score": {{max_score}}
          }
        },
        "collection": {
          "identifier": "{{collectionId}}",
          "name": "{{collectionName}}",
          "logo": "{{logo Url}}",
          "leafNodesCount": {{leafNodeCount}},
          "description": "{{description}}"
        },
        "issuedCertificates": [{
          "name": "{{certName}}",
          "id": "certificateId",
          "token": "{{certToken}}",
          "lastIssuedOn": "{{lastIssuedOn}}"
        }],
        "completedOn": {{completion_date}},
        "progress": {{progress}},
        "status": {{status}}
      }
    ]
  }
}

...

{
    "request": {
        "userId": "{{userId}}",
        "collectionId" : "{{collectionId}}",
        "batchIdcontextId": "{{batchId}}"
    }
}

Response: 
{
    "id": "api.summary.delete",
    "ver": "v1",
    "ts": "2021-06-23 05:37:40:575+0000",
    "params": {
        "resmsgid": null,
        "msgid": "5e763bc2-b072-440d-916e-da787881b1b9",
        "err": null,
        "status": "success",
        "errmsg": null
    },
    "responseCode": "OK",
    "result": {}
}

...