Page Comparison

...

As a State Admin, I would want to have access to scores of self-assessment/s of all users within a batch of a course, So that I can get some insights on the quality of the contents within a course.

Solution I:

As part of Every question units having evaluation logic and plugin knows the question attempted or not. This data we are not sending as part of Telemetry.

After evaluation, there is a callback to the question-set /assessment, we are generating the "ASSESS" telemetry event for all questions during the evaluation of the question. So each ASSESS event contains question params and resValues.

So looking into telemetry Assess event resValues empty or contains data in an array based on this data will consider Question attempted or skipped.

...

Expand

title	Skipped question

"edata": {
"item": {
"id": "do_11286000975564800011904",
"maxscore": 1,
"type": "ftb",
"exlength": 0,
"params": [
{
"1": "{\"text\":\"1\"}"
},
{
"eval": "order"
}
],
"uri": "",
"title": "FTB: QuestionService fix____\n",
"mmc": [],
"mc": [],
"desc": ""
},
"index": 1,
"pass": "No",
"score": 0,
"resvalues": [],
"duration": 26
}

...

title	Skipped question

...

plugin. so while returning data from question unit plugins will send attempted: true/false and questionId:"do_id" as part of the result. This data will carry to question-set and dispatch event to org.ekstep.questionset:summary.

So it will help to know the score report of the assessment.

Example:

Expand

title	MCQ state data

Code Block

language	js

{
"attempted":true,
"eval": true,
"state": {
"val": 0,
"options": [
{
"text": "<p>True</p>\n",
"image": "",
"audio": "",
"audioName": "",
"hint": "",
"isCorrect": true,
"$$hashKey": "object:1456"
},
{
"text": "<p>False</p>\n",
"image": "",
"audio": "",
"audioName": "",
"hint": "",
"isCorrect": false,
"$$hashKey": "object:1457"
}
]
},
"score": 1,
"params": [
{
"1": "{\"text\":\"True\\n\"}"

...

},

...


"2": "{\"text\":\"False\\n\"}"

...

},

...


"answer": "{\"correct\":[\"1\"]}"

...

],

...

values": [
{
"

...

1": "{\"text\":

...

\"True\\n\"}"
}
],

...

type":

...

 "mcq",
"questionId":"do_11287200486896435214"

...

Solution II:

As part of the question-set/assessment, we are generating the "ASSESS" telemetry event for all questions during the evaluation of the question. So each ASSESS event contains question params and resValues.

So looking into telemetry Assess event resValues empty or contains data in an array based on this data will consider Question attempted or skipped.

FTB

MCQ

MTF

Sequence

Reorder

ASSESS Event resvalues

Expand

title	Skipped question

"edata": {
"item": {
"id": "do_1128600090344325121190211286000975564800011904",
"maxscore": 1,
"type": "mtfftb",
"exlength": 0,
"params": [
{
"lhs": "[{\"1\": \ "{\\\"text\\\":\\\"A\\\\n\\\"1\"}\"
},
{\
"2\eval": \ "{\\\"text\\\":\\\"B\\\\n\\\"}\"},{\"3\":\"{\\\"text\\\":\\\"C\\\\n\\\"}\"}]"
},
{
"rhs": "[{\"1\":\"{\\\"text\\\":\\\"b\\\\n\\\"}\"},{\"2\":\"{\\\"text\\\":\\\"c\\\\n\\\"}\"},{\"3\":\"{\\\"text\\\":\\\"a\\\\n\\\"}\"}]"
},
{
"answer": "{\"lhs\":[\"1\",\"2\",\"3\"],\"rhs\":[\"3\",\"1\",\"2\"]}"
}
],
"uri": "",
"title": "MTF:QuestionService fix\n"order"
}
],
"uri": "",
"title": "FTB: QuestionService fix____\n",
"mmc": [],
"mc": [],
"desc": ""
},
"index": 1,
"pass": "No",
"score": 0,
"resvalues": [],
"duration": 26
}

Expand

title	Skipped question

"edata": {
"item": {
"id": "do_11286000940202393611903",
"maxscore": 1,
"type": "mcq",
"exlength": 0,
"params": [
{
"1": "{\"text\":\"True\\n\"}"
},
{
"2": "{\"text\":\"False\\n\"}"
},
{
"answer": "{\"correct\":[\"1\"]}"
}
],
"uri": "",
"title": "MCQ: QuestionService fix\n",
"mmc": [],
"mc": [],
"desc": ""
},
"index": 32,
"pass": "No",
"score": 0,
"resvalues": [
{}
],
"lhs": "[{\"duration": 6
}

Expand

title	Skipped question

"edata": {
"item": {
"id": "do_11286000903443251211902",
"maxscore": 1,
"type": "mtf",
"exlength": 0,
"params": [
{
"lhs": "[{\"1\":\"{\\\"text\\\":\\\"A\\\\n\\\"}\"},{\"2\":\"{\\\"text\\\":\\\"B\\\\n\\\"}\"},{\"3\":\"{\\\"text\\\":\\\"C\\\\n\\\"}\"}]"
},
{
"rhs": "[{\"1\":\"{\\\"text\\\":\\\"b\\\\n\\\"}\"},{\"2\":\"{\\\"text\\\":\\\"c\\\\n\\\"}\"},{\"3\":\"{\\\"text\\\":\\\"a\\\\n\\\"}\"}]"
},
],{
"durationanswer": 3
}

Expand

title	Skipped question

"edata": {
\"itemlhs\": {
"id": "do_11286001060099686411906",
"maxscore": 1,
"type": "sequence",
"exlength": 0,
"params": [
{
"1": "{\"text\":\"Two\"}"
},
{
"2": "{\"text\":\"One\"}"
},
{
"answer": "{\"seq\":[\"2\",\"1\"]}"
}
],
"uri": "",
"title": "Question service fix \n",
"mmc": [],
"mc": [],
"desc": ""
},
"index": 4,
"pass": "No",
"score": 0,
"resvalues": [
{
"1": "{\"text\":\"Two\"}[\"1\",\"2\",\"3\"],\"rhs\":[\"3\",\"1\",\"2\"]}"
}
],
"uri": "",
"title": "MTF:QuestionService fix\n",
"mmc": [],
"mc": [],
"desc": ""
},
"index": 3,
"pass": "No",
"score": 0,
"resvalues": [
{
"lhs": "[{\"1\":\"{\\\"text\\\":\\\"A\\\\n\\\"}\"},{\"2\":\"{\\\"text\\\":\\\"B\\\\n\\\"}\"},{\"3\":\"{\\\"text\\\":\\\"C\\\\n\\\"}\"}]"
},
{
"2rhs": "[{\"1\":\"{\\\"text\\\":\\\"Oneb\\\\n\\\"}\"
}
],
{\"duration2\": 3
}

Expand

title	Skipped question

"edata": {
"item": {
"id": "do_11286001026143846411905",
"maxscore": 1,
"type": "reorder",
"exlength": 0,
"params": [
{
"1": "{\"text\":\"4\"}"
},
{
"2": "{\"text\":\"8\"}"
},
{
"3\"{\\\"text\\\":\\\"c\\\\n\\\"}\"},{\"3\":\"{\\\"text\\\":\\\"a\\\\n\\\"}\"}]"
}
],
"duration": 3
}

Expand

title	Skipped question

"edata": {
"item": {
"id": "do_11286001060099686411906",
"maxscore": 1,
"type": "sequence",
"exlength": 0,
"params": [
{
"1": "{\"text\":\"6Two\"}"
},
{
"42": "{\"text\":\"5One\"}"
},
{
"5answer": "{\"textseq\":[\"2\",\"1\"]}"
}
],
{
"6uri": "{\"text\",
"title": "Question service fix \n"7\,
"}"
}mmc": [],
{
"7mc": "{\"text\":\"1\"}[],
"desc": ""
},
{
"answerindex": 4,
"{\pass"seq\": [\ "3\No",\
"7\score": 0,
"resvalues": [
{
"1": "{\"5text\",:\"4Two\",\}"
},
{
"2\": ",{\"6text\",:\"1One\"]}"
}
],
"uriduration": 3
}

Expand

title	Skipped question

"edata",: {
"titleitem": "QuestionService fixArrange the given words in proper order to form a sentence.\n\n \n",
"mmc": [],
"mc": [],
"desc": ""
},
"index": 5,
"pass": "No",
"score": 0,
"resvalues": [],
"duration": 3
}

Expand

title	Attempted question

"edata": {
"item": {
"id": "do_11286000975564800011904",
"maxscore": 1,
"type": "ftb",
"exlength": 0,
"params": [
{
"1{
"id": "do_11286001026143846411905",
"maxscore": 1,
"type": "reorder",
"exlength": 0,
"params": [
{
"1": "{\"text\":\"4\"}"
},
{
"2": "{\"text\":\"8\"}"
},
{
"3": "{\"text\":\"6\"}"
},
{
"4": "{\"text\":\"15\"}"
},
{
"eval5": "order{\"text\":\"2\"}"
},
],{
"uri6": "",
"title{\"text\": "FTB: QuestionService fix____\n",
"mmc": [],
"mc": [],
"desc": ""\"7\"}"
},
{
"7": "{\"text\":\"1\"}"
},
{
"indexanswer": 1,
"pass"{\"seq\":[\"Yes3\",
\"score7\": 1,
"resvalues": [
{
"1": "{\"text\":,\"5\",\"4\",\"2\",\"6\",\"1\"]}"
}
],
"durationuri": 588
}

Expand

title	Attempted question

"edata": {,
"itemtitle": {
"id": "do_11286000940202393611903",
"maxscore": 1,
"type": "mcq",
"exlength": 0,
"params": [
{
"1": "{\"text\":\"True\\n\"}"
},
{
"2": "{\"text\":\"False\\n\"}"
},
{
"answer"QuestionService fixArrange the given words in proper order to form a sentence.\n\n \n",
"mmc": [],
"mc": [],
"desc": ""
},
"index": 5,
"pass": "No",
"score": 0,
"resvalues": [],
"duration": 3
}

Expand

title	Attempted question

"edata": {
"item": {
"id": "do_11286000975564800011904",
"maxscore": 1,
"type": "ftb",
"exlength": 0,
"params": [
{
"1": "{\"correcttext\":[\"1\"]}"
},
{
"eval": "order"
}
],
"uri": "",
"title": "MCQFTB: QuestionService fix____\n",
"mmc": [],
"mc": [],
"desc": ""
},
"index": 21,
"pass": "Yes",
"score": 1,
"resvalues": [
{
"1": "{\"text\":\"True1\\n\"}"
}
],
"duration": 293588
}

Expand

title	Attempted question

"edata": {
"item": {
"id": "do_1128600090344325121190211286000940202393611903",
"maxscore": 1,
"type": "mtfmcq",
"exlength": 0,
"params": [
{
"lhs": "[{\"1\": \ "{\\\"text\\\":\\\"ATrue\\\\n\\\"}\"
},
{\
"2\": \ "{\\\"text\\\":\\\"BFalse\\\\n\\\"}\"
},
{\
"3\answer": \ "{\\\"textcorrect\\\":\\[\"C\\\\n\\1\"]}\"
}
]"
},
{
"rhsuri": "[{\"1\":\"{\\\"text\\\":\\\"b\\\\n\\\"}\"},{\"2\":\"{\\\"text\\\":\\\"c\\\\n\\\"}\"},{\"3\":\"{\\\"text\\\":\\\"a\\\\n\\\"}\"}]"
},
{
"answer": "{\"lhs\":[\"1\",\"2\",\"3\"],\"rhs\":[\"3\",\"1\",\"2\"]}"
}
],
"uri": "",
"title": "MTF:QuestionService fix\n",
"mmc": [],
"mc": [],
"desc": ""
},
"index": 3,
"pass": "No",
"score": 0.33,
"resvalues": [
{
"lhs": "",
"title": "MCQ: QuestionService fix\n",
"mmc": [],
"mc": [],
"desc": ""
},
"index": 2,
"pass": "Yes",
"score": 1,
"resvalues": [
{
"1": "{\"text\":\"True\\n\"}"
}
],
"duration": 293
}

Expand

title	Attempted question

"edata": {
"item": {
"id": "do_11286000903443251211902",
"maxscore": 1,
"type": "mtf",
"exlength": 0,
"params": [
{
"lhs": "[{\"1\":\"{\\\"text\\\":\\\"A\\\\n\\\"}\"},{\"2\":\"{\\\"text\\\":\\\"B\\\\n\\\"}\"},{\"3\":\"{\\\"text\\\":\\\"C\\\\n\\\"}\"}]"
},
{
"rhs": "[{\"1\":\"{\\\"text\\\":\\\"Ab\\\\n\\\"}\"},{\"2\":\"{\\\"text\\\":\\\"Bc\\\\n\\\"}\"},{\"3\":\"{\\\"text\\\":\\\"Ca\\\\n\\\"}\"}]"
},
{
"rhsanswer": "[{\"1lhs\":[\"{\\1\",\"2\",\"text3\"],\"rhs\":[\"3\",\"c1\",\"2\\n\\\"]}\"
}
],{\
"2\uri": \"{\\\"text\\\":\\\"b\\\\n\\\"}\"},{\"3\":\"{\\\"text\\\":\\\"a\\\\n\\\"}\"}]"
}
],
"duration": 63
}

Expand

title	Attempted question

"edata": {
"item": {
"id": "do_11286001060099686411906",
"maxscore": 1,
"type": "sequence",
"exlength": 0,
"params"",
"title": "MTF:QuestionService fix\n",
"mmc": [],
"mc": [],
"desc": ""
},
"index": 3,
"pass": "No",
"score": 0.33,
"resvalues": [
{
"lhs": "[{\"1\":\"{\\\"text\\\":\"Two\\"A\\\\n\\\"}\"
},
{
\"2\":\"{\\\"text\\\":\\\"OneB\\\\n\\\"}\"
},
{
\"answer3\":\"{\\\"seqtext\\\":[\\\"2C\",\"1\"]}\\n\\\"}\"}]"
},
],{
"urirhs": "",
"title": "Question service fix \n",
"mmc": [],
"mc": [],
"desc": ""
},
"index": 4,
"pass": "Yes",
"score": 1,
"resvalues": [
{
"2": "{\"text\":\"One\"}"
},
{
"1": "{\"text\":\"Two\"}"[{\"1\":\"{\\\"text\\\":\\\"c\\\\n\\\"}\"},{\"2\":\"{\\\"text\\\":\\\"b\\\\n\\\"}\"},{\"3\":\"{\\\"text\\\":\\\"a\\\\n\\\"}\"}]"
}
],
"duration": 6263
}

Expand

title	Attempted question

"edata": {
"item": {
"id": "do_1128600102614384641190511286001060099686411906",
"maxscore": 1,
"type": "reordersequence",
"exlength": 0,
"params": [
{
"1": "{\"text\":\"4Two\"}"
},
{
"2": "{\"text\":\"8One\"}"
},
{
"3answer": "{\"textseq\":[\"62\",\"1\"]}"
}
],
{
"4uri": ""{\"text\,
"title": \ "5Question service fix \n"}"
},
{
"5mmc": "{\"text\":\"2\"}"[],
"mc": [],
"desc": ""
},
{"index": 4,
"6pass": "Yes"{\"text\,
"score": \"7\"}"
},1,
"resvalues": [
{
"72": "{\"text\":\"1One\"}"
},
{
"answer1": "{\"seqtext\":[\"3Two\",\"7\",\"5\",\"4\",\"2\",\"6\",\"1\"]}"
}
],
"uri": "",
"title": "QuestionService fixArrange the given words in proper order to form a sentence.\n\n \n",
"mmc": [],
"mc": [],
"desc": ""
},
"index": 5,
"pass": "No",
"score": 0,
"resvalues": [
}"
}
],
"duration": 62
}

Expand

title	Attempted question

"edata": {
"item": {
"id": "do_11286001026143846411905",
"maxscore": 1,
"type": "reorder",
"exlength": 0,
"params": [
{
"1": "{\"text\":\"4\"}"
},
{
"2": "{\"text\":\"8\"}"
},
{
"3": "{\"text\":\"6\"}"
},
],
"duration": 55
}

Solution II:

Based on Telemetry Response event. Currently, MCQ type of questions only generating a Response event.

If the response event generates so it is considered as an attempted question.

MCQFTBMTFSequenceReorderResponse event

Expand

title	Attempted question

{
"eid": "RESPONSE",
"ets": 1571385741820,
"ver": "3.0",
"mid": "RESPONSE:c35bb556e3920e6a5b287ac699d96448",
"actor": {
"id": "874ed8a5-782e-4f6c-8f36-e0288455901e",
"type": "User"
},
"context": {
"channel": "b00bc992ef25f1a9a8d63291e20efc8d",
"pdata": {
"id": "dev.sunbird.portal",
"ver": "2.4.0",
"pid": "sunbird-portal.contenteditor.contentplayer"
},
"env": "contentplayer",
"sid": "nm-dQSSXROxfJpbONAmdXSmY_2ttSA6C",
"did": "d1b3344d6b8d8b5441c3b47842b61abd",
"cdata": [
{
"id": "f799686c9c913589cf270153edf8dbf6",
"type": "ContentSession

{
"4": "{\"text\":\"5\"}"
},
{
"5": "{\"text\":\"2\"}"
},
{
"6": "{\"text\":\"7\"}"
},
{
"7": "{\"text\":\"1\"}"
},
{
"answer": "{\"seq\":[\"3\",\"7\",\"5\",\"4\",\"2\",\"6\",\"1\"]}"
}
],
"uri": "",
"title": "QuestionService fixArrange the given words in proper order to form a sentence.\n\n \n",
"mmc": [],
"mc": [],
"desc": ""
},
"index": 5,
"pass": "No",
"score": 0,
"resvalues": [
{
"2": "{\"text\":\"8\"}"
},
{
"3": "{\"text\":\"6\"}"
}
],
"

rollup

duration":

{}

55
}

,

"object"

Solution III:

...

Based on Telemetry Response event. Currently, MCQ type of questions only generating a Response event.
If the response event generates so it is considered as an attempted question.

MCQ FTB MTF Sequence Reorder
Response event
Expand
title Attempted question
{
"eid": "RESPONSE",
"ets": 1571385741820,
"ver": "13.0",
"rollupmid": {}
},
"tags": []"RESPONSE:c35bb556e3920e6a5b287ac699d96448",
"edata": {
"target"actor": {
"id": "org.ekstep.questionunit.mcq874ed8a5-782e-4f6c-8f36-e0288455901e",
"vertype": "1.3User"
},
"typecontext": "plugin"
},{
"typechannel": "CHOOSEb00bc992ef25f1a9a8d63291e20efc8d",
"valuespdata": [
{
"option0id": "<p>True</p>\n"
}
]
}
}

Solution III:

In questionSet when the user selects or filled the answer, we are preserving the answers for state management. So this data will store in evaluated result we call state data.

For example: MCQ: result:state:value => if attempted => selected option index

result:state:value=> if skipped => result:state:value: undefined

FTB: result:state:value => if attempted => array of answers (result:state:value['apple','mango'])

result:state:value=> if skipped => result:state:value:[]

...

dev.sunbird.portal",
"ver": "2.4.0",
"pid": "sunbird-portal.contenteditor.contentplayer"
},
"env": "contentplayer",
"sid": "nm-dQSSXROxfJpbONAmdXSmY_2ttSA6C",
"did": "d1b3344d6b8d8b5441c3b47842b61abd",
"cdata": [
{
"id": "f799686c9c913589cf270153edf8dbf6",
"type": "ContentSession"
}
],
"rollup": {}
},
"object": {
"id": "do_11287200486896435214",
"type": "Content",
"ver": "1.0",
"rollup": {}
},
"tags": [],
"edata": {
"target": {
"id": "org.ekstep.questionunit.mcq",
"ver": "1.3",
"type": "plugin"
},
"type": "CHOOSE",
"values": [
{
"option0": "<p>True</p>\n"
}
]
}
}

Solution IV:

In questionSet when the user selects or filled the answer, we are preserving the answers for state management. So this data will store in evaluated result we call state data.

For example: MCQ: result:state:value => if attempted => array of selected answers selected option index

result:state:value=> if skipped => result:state:value: undefined

FTB: result:state:value => if attempted => array of answers (result:state:value['apple','mango'])

result:state:value=> if skipped => result:state:value:[]

Evaluated result state object

...

Solution IV:

Every question units having evaluation logic and plugin knows the question attempted or not. This data we are not sending as part of Telemetry.

After evaluation, there is a callback to the question-set plugin. so while returning data from question unit plugins will send attempted: true/false and questionId:"do_id" as part of the result. This data will carry to question-set and dispatch event to org.ekstep.questionset:summary.

So it will help to know the score report of the assessment.

Example:

...

title	MCQ state data

...

language	js

...

reorder: result:state:value => if attempted => array of selected answers (result:state:value['apple','mango'])

result:state:value=> if skipped => result:state:value:[]

Evaluated result state object

	MCQ	FTB	Reorder	Sequence	MTF
Attempted
Skipped

Versions Compared

Old Version 2

New Version Current

Key

Solution I:

Solution II:

Solution II:

Solution III:

Solution III:

Solution IV:

Solution IV: