...

To be able to handle the above design problems, we have analyzed how similar products (like netflix) track everything what a user does (or views) and that too at scale. Based on the analysis we have broken down the APIs into more granular APIs with a single DB update so that each API can be scaled independently. In addition we have designed the APIs as a general purpose service (similar to asset service/graph engine) on which various use-cases can be mapped.

Following are few of the Cassandra scale issues for various approaches:

...

Once the view ends, the progress and score will be updated asynchronously by the flink jobs.

Extended Enrolment Consumption:

• Every new instance adapting the sunbird platform will have to select one of the option from 3 context modes, this would allow the application to mange the user in avoiding the consumption of content more than once based of specific predefined rules

• Mode for any instance will be one to one mapping

• With the extended design , tracking and monitoring of the user consumption can be done for any new context like program, event etc

...

Following are the different modes provided to new instance:

...

Scenario

...

Write Request

...

Read Request

...

Carry Forward Consumption

• The content consumed is marked as complete irrespective of context

...

{
    "userid": "<<userid>>"
    "collectionid" : "<<courseid>>",
    "contentid" :"<<contenid>>"
}

Note: Progress will be captured directly under the context

{
  "userId": "<<userid>>",
  "collectionId": "<<courseid>>",
  "contentId": "<<contentid>>"
}

...

Copy Forward Consumption

• The content consumed is marked as complete along with new entry in the database for the context

{
    "userid": "<<userid>>",
    "collectionid": "<<courseid>>",
    "contextid": "<<programid>>",
    "contentid": "<<contentid>>"
}

{
  "userId": "<<userid>>",
  "collectionId": "<<courseid>>",
  "contentId": "<<contentid>>"
}

...

Strict Mode Consumption

• The content will be consumed as new one every time

{
    "userid": "<<userid>>",
    "collectionid": "<<courseid>>",
    "contextid": "<<programid>>",
    "contentid": "<<contenid>>"
}

Viewer-Service - Content Consumption Scenarios:

The user can consume a content by searching it in our platform (organically) or via a collection when the user enrolled to a course.

With Viewer-Service, we will support tracking individual content consumption also. Below details explain how the data will be stored for a content consumption in different scenarios.

...

The below table has various scenarios considering the current and future use cases. Here we defined the database read/write logic to support these use case and fetch the save or fetch the required data from user_content_consumption table.

Key words used in below table:

• Carry forward content consumption - Considering the content consumed in any context to compute the progress or completion percentage (any collection, batch or individual content consumption).

...

Scenario

...

Write Request

...

Read Request(Condition)

...

Read Query

...

User consuming individual content. [New]

...

{
    "userid": "<<userid>>",
    "collectionid": "<<contentid>>",
    "contextid": "<<contentid>>",
    "contentid": "<<contentid>>"
}

{
    "userid": "<<userid>>",
    "contentid": "<<contentid>>"
}

...

WHERE userid='<<userid>>' and collectionid ='<<contentid>>' and contentid = <contentid> and contextid = <contentid>

...

User consuming a content with in a collection. [Existing]

...

Viewer Service - Database Design

The user can consume a content by searching it in our platform (organically) or via a collection when the user enrolled to a course.

With Viewer-Service, we will support tracking individual content consumption also. Below details explain how the data will be stored for a content consumption in different scenarios.

...

The below table has various scenarios considering the current and future use cases. Here we defined the database read/write logic to support these use case and fetch the save or fetch the required data from user_content_consumption table.

user_content_consumption (
    userid text,
    collectionid text, // currently labelled as courseid
    contextid text, // currently labelled as batchid
    contentid text,
    last_access_time timestamp,
    last_completed_time timestamp,
    last_updated_time timestamp,
    progressdetails json,
    status int,
    PRIMARY KEY (userid, collectionid, contextid, contentid)
)

assessment_aggregator (
    user_id text,
    collection_id text, // currently labelled as courseid
    context_id text, // currently labelled as contextid
    content_id text,
    attempt_id text,
    created_on timestamp,
    grand_total text,
    last_attempted_on timestamp,
    questions list<frozen<question>>,
    total_max_score double,
    total_score double,
    updated_on timestamp,
    PRIMARY KEY ((user_id, collection_id), context_id, content_id, attempt_id)
)

user_activity_agg (
    activity_type text,
    activity_id text,
    user_id text,
    context_id text,
    agg map<text, int>,
    content_status frozen<map<text,int>>,
    agg_last_updated map<text, timestamp>,
    PRIMARY KEY ((activity_type, activity_id, user_id), context_id)
)

{
  userid: "<userid>",
  contentid: "<contentid>"
}

INSERT into ucc(userid, collectionid, contextid, contentid, status) 
values('<userid>','<contentid>','<contentid>','<contentid>','<status>')

{
  userid: "<userid>",
  contentid: "<contentid>"
}

from ucc where userid='<contentid>' and collectionid='<contentid>' 
and contextid='<contentid>' and contentid='<contentid>'

{
  userid: "<userid>",
  collectionid: "<collectionid>",
  contentid: "<contentid>"
}

INSERT into ucc(userid, collectionid, contextid, contentid, status) 
values('<userid>','<collectionid>','<collectionid>','<contentid>','<status>')

{
  userid: "<userid>",
  collection: "<collectionid>"
}

from ucc where userid='<contentid>' and collectionid='<collectionid>' 
and contextid='<collectionid>'

{
  userid: "<userid>",
  collectionid: "<collectionid>",
  contextid: "<contextid>", // batchId, programId, programBatchId
  contentid: "<contentid>"
}

INSERT into ucc(userid, collectionid, contextid, contentid, status) 
values('<userid>','<collectionid>','<contextid>','<contentid>','<status>')

{
  userid: "<userid>",
  collectionid: "<collectionid>",
  contextid: "<contextid>" // batchId, programId, programBatchId
}

from ucc where userid='<contentid>' and collectionid='<contentid>' 
and contextid='<contextid>'

Mapping Usecases

Extended Enrolment Consumption:

• Every new instance adapting the sunbird platform will have to select one of the option from 3 context modes, this would allow the application to mange the user in avoiding the consumption of content more than once based of specific predefined rules

• Mode for any instance will be one to one mapping

• With the extended design , tracking and monitoring of the user consumption can be done for any new context like program, event etc

...

Following are the different modes provided to new instance:

{
    "userid": "<<userid>>",
    "collectionid" : "<<courseid>>",
    "contextidcontentid": "<<batchid>>",
    "contentid": "<<contentid>><<contenid>>"
}

{
 
  "useriduserId": "<<userid>>",
    "collectionid": "<<courseid>>",
    "contextid""collectionId": "<<courseid>>",
    "contentidcontentId": "<<contentid>>"
}

<<contentid>>"
}

{
    "userid": "<<userid>>",
    "collectionid": "<<courseid>>",
    "contextid": "<<programid>>",
    "contentid": "<<contentid>>"
}

{
  "userId": "<<courseid>><<userid>>",
 
  "contextidcollectionId": "<<batchid>><<courseid>>",
 
  "contentidcontentId": "<<contentid>>"
}

{
    "userid": "<<userid>>",
    "collectionid": "<<courseid>>",
    "contextid": "<<courseid>><<programid>>",
    "contentid": "<<contentid>><<contenid>>"
}

Content View Lifecycle:

When the user view the content in context of a collection and batch, for the first time its start, progress update and end triggers are processed. Revisit (2nd - nth view) of the content will be ignored to process and update the DB.

...