Page Comparison

...

1. The mobile team has seen challenges with manifest JSON beyond 2MB. So the combined manifest for the content + question sets will need to be lesser than 2MB. One way to achieve this is to put a limit on the number of questions that can be added to an interactive content.

Option 2 - Chained publishing

...

with new coordinator job

Instead of a single publish call for question sets and content, the publishing will happen sequentially using chaining a chain of publish events pushed to the publish topic.

A chain of published event will processed by a new job which will act as a coordinator to call further jobs depending on object type in published chain array.

• Behaviour of new job -

1. New job will read and validate the published chain input event from topic. If the event is valid, it will read publish chain array and internally generate events and publish on respective topic of questionset-publish or content-publish depending on type of event.

2. The publishing of the first question set will happen first, followed by the subsequent question set, and finally the content.

...

1. When the publish API for

...

1. interactive video is called, all the question sets and the video is moved to Processing state. This way, in case of a failure, videos stuck in Processing state can be looked up, and the failure analysed.

2. In the event published to questionset-publish or content-publish job, publishChain field will be passed along with metadata to notify that this event belongs to publish chain.

3. Changes in existing events of the questionset-publish and content-publish job -

“publishChain” field in metadata which will contain entire publish chain event (string format)

{
  "eid": "BE_JOB_REQUEST",
  "ets": 1619527882745,
  "mid": "LP.1619527882745.32dc378a-430f-49f6-83b5-bd73b767ad36",
  "actor": {
    "id": "content-publish",
    "type": "System"
  },
  "context": {
    "channel": "Sunbird",
    "pdata": {
      "id": "org.sunbird.platform",
      "ver": "3.0"
    },
    "env": "dev"
  },
  "object": {
    "id": "do_11341490059629363211",
    "ver": "1637561126014"
  },
  "edata": {
    "publish_type": "public",
    "metadata": {
      "identifier": "do_11341490059629363211",
      "mimeType": "video/mp4",
      "objectType": "Content",
      "lastPublishedBy": "",
      "pkgVersion": 2,
      "publishChain": "{\n  \"eid\": \"BE_JOB_REQUEST\",\n  \"ets\": 1634812181709,\n  \"mid\": \"LP.1634812181709.88b311c2-a733-4846-b2a2-31d9dceda436\",\n  \"actor\": {\n    \"id\": \"interactivevideo-publish\",\n    \"type\": \"System\"\n  },\n  \"context\": {\n    \"pdata\": {\n      \"ver\": \"1.0\",\n      \"id\": \"org.sunbird.platform\"\n    },\n    \"channel\": \"0133866213417451520\",\n    \"env\": \"dev\"\n  },\n  \"object\": {\n    \"ver\": \"1634810156009\",\n    \"id\": \"do_213392364484722688135\"\n  },\n  \"edata\": {\n    \"publish_type\": \"public\",\n    \"metadata\": {},\n    \"publishchain\": [\n      {\n        \"identifier\": \"do identifier of questionset 1\",\n        \"mimeType\": \"application\/vnd.sunbird.questionset\",\n        \"lastPublishedBy\": \"2ca3f787-379f-42d4-9cc3-6bc03a2d4cca\",\n        \"pkgVersion\": null,\n        \"objectType\": \"QuestionSet\",\n        \"state\": \"Processing\",\n        \"publishErr\": \"\",\n        \"order\": 1\n      },\n      {\n        \"identifier\": \"do identifier of questionset 2\",\n        \"mimeType\": \"application\/vnd.sunbird.questionset\",\n        \"lastPublishedBy\": \"2ca3f787-379f-42d4-9cc3-6bc03a2d4cca\",\n        \"pkgVersion\": null,\n        \"objectType\": \"QuestionSet\",\n        \"state\": \"Processing\",\n        \"publishErr\": \"\",\n        \"order\": 2\n      },\n      {\n        \"identifier\": \"do_213392364484722688135\",\n        \"mimeType\": \"video\/mp4\",\n        \"lastPublishedBy\": \"2ca3f787-379f-42d4-9cc3-6bc03a2d4cca\",\n        \"pkgVersion\": null,\n        \"objectType\": \"Content\",\n        \"state\": \"Processing\",\n        \"publishErr\": \"\",\n        \"order\": 3\n      }\n    ],\n    \"action\": \"publishchain\",\n    \"iteration\": 1,\n    \"contentType\": \"Resource\"\n  }\n}\n\n"
    },
    "action": "publish",
    "iteration": 1
  }
}

• Changes to existing publish jobs -

1. Event structures of questionset-publish and content-publish will have new field “publishChain“ which identifies if the event is from chained publish job.

2. Change the state of question sets and video.

3. Send event back to topic of coordinator job with updated state of the jobs so that coordinator job can pick up the next object in publish chain.

• Publish event structure -

The publish event structure will be enhanced to add the details of the chain i.e. the sequence and the IDs of the content. As the chain progresses, each publish event will have the current state of the chain i.e. which are the preceding successful publishes, and which are the ones up next. Proposed changes to the publish event are below

{
  "eid": "BE_JOB_REQUEST",
  "ets": 1634812181709,
  "mid": "LP.1634812181709.88b311c2-a733-4846-b2a2-31d9dceda436",
  "actor": {
    "id": "Publish Samza Jobinteractivevideo-publish",
    "type": "System"
  },
  "context": {
    "pdata": {
      "ver": "1.0",
      "id": "org.sunbird.platform"
    },
    "channel": "0133866213417451520",
    "env": "dev"
  },
  "object": {
    "ver": "1634810156009",
    "id": "do_213392364484722688135"
  },
  "edata": {
    "publish_type": "public",
    "metadata": {},
    "publishchain": [
      {
        "identifier": "do identifier of questionset 1",
        "mimeType": "application/vnd.sunbird.questionset",
        "lastPublishedBy": "2ca3f787-379f-42d4-9cc3-6bc03a2d4cca",
        "pkgVersion": null,
        "objectType": "QuestionSet",
        "state": "Processing",
        "publishErr": "",
        "order": 1
      },
      {
        "identifier": "do identifier of questionset 2",
        "mimeType": "application/vnd.sunbird.questionset",
        "lastPublishedBy": "2ca3f787-379f-42d4-9cc3-6bc03a2d4cca",
        "pkgVersion": null,
        "objectType": "QuestionSet",
        "state": "Processing",
        "publishErr": "",
        "order": 2
      },
      {
        "identifier": "do_213392364484722688135",
        "mimeType": "video/mp4",
        "lastPublishedBy": "2ca3f787-379f-42d4-9cc3-6bc03a2d4cca",
        "pkgVersion": null,
        "objectType": "Content",
        "state": "Processing",
        "publishErr": "",
        "order": 3
      }
    ],
    "action": "publishchain",
    "iteration": 1,
    "contentType": "Resource"
  }
}

...

There is no change to the ECAR generation, each publish event will work as today, to generate an ECAR for each question set and the content.

Option 3 - Chained publishing without new coordinator job

Changes to Publish Jobs

Both publish jobs (content & question set) will be enhanced to handle both types of events

• publish

• publishchain

If the event is publish, whatever today happens will continue to happen
If the event is publishchain the job will do it's work and produce another publishchain event with the chain updated

The publish chain event will be same as mentioned in option 2.

Handling Publishing Failure

...