Definition
For OGHR, Platform needs to enable user to undertake exams. Platform should be capable of processing the user responses and evaluate these responses with relevant applicable scores. Outcome of this design should enable the platform users to take up examination with server side evaluation of results.
Background
In the present system, Question Set consist of following :
Assessment User : Active seeker of Assessment in the platform.
QuML Player : QuML Player has capability to play the questions in QuML format. It also has capability to locally validate the user input. Response Validation,Score Computing is completely handled in player. Once User Submits the overall response, client validated scores and response are updated using Content State Update API.
Flink Jobs : Flink Jobs aggregates the content state using Collection Activity Aggregator, Collection Assessment Aggregator, Cert Pre-processor, Cert Generator jobs.
Objective Types:
a) MCQ (Multiple Choice Questions)
b) MSQ (Multiple Select Questions)
c) MTF (Match The Following)
d) FTB (Fill in The Blanks)
Subjective Types
a) VSA - (Very Short Answer)
b) SA - (Short Answer)
c) LA - (Long Answer)
Building Blocks :
Building Blocks | API | Flink Jobs |
|---|---|---|
Sunbird InQuiry | Question Set Hierarchy API | |
Question List Read API | ||
QuestionSet Create API | ||
Question Create API | ||
Sunbird Lern | Content State Read | Collection Activity Aggregate |
Content State Update | Collection Assessment Aggregate | |
Enrollment List API | Cert Pre Processer | |
Cert Generator | ||
Sunbird RC | Cert Registry Download API |
Design Problems :
Allow the response processing of question to be done on server rather than client as happening today.
Scalable Response Processing Solution for Question Sets.
Score Calculation Based on Content State Update need to be shifted to server side evaluation.
Solution to the question needs to be masqueraded or excluded from the Question Read API/ Question List API.
Response Processing can happen in two ways:
Entire Question Set Response Processing
Question by Question Response Processing
Current Workflow
Solution Proposed
Proposed System:
Technical Design Details:
Step 1 : “serverEvaluable“ attribute at question and questionSet Object
A new attribute in QuestionSet to be introduced to have evaluated response on server. (“serverEvaluable“).
//Question Set Object
"questionSet": {
"serverEvaluable": true //#true for Server Side Valuation Default:#false for client side validation
}
//Question Object
"question" : {
"serverEvaluable": true
}
Question Create API:
It is possible to permanently mark the question to be “serverEvaluable“ in nature. Any serverEvaluable question would automatically qualify for serverEvaluable question Set. These Questions can be part of Evaluable QuestionSet only.
KeySet (Futuristic):
Any Responses on Question Set can be persisted separately to ensure the keyset can be used for administrative purposes.
There are two modes of accessing questionSet as per design:
a) non-evaluable mode : Player invokes questionSetHierarchy GET API.
b) evaluable mode : Player would invoke questionSetHierarchy POST API (New API). This change would be used to manage requirements like Question Randomization, Question Token generation etc.
Step 2: Introduce new POST method for QuestionSet Hierarchy API.Enable QuestionSetToken in Question Set Hierarchy API (New API). Mark Question and QuestionSet Objects as “serverEvaluable“
QuestionSet Hierarchy API (POST) : A new API method to be introduced for QuestionSet Hierarchy of Exam Question Sets. This API will have payload as follows
"request": {
"contentID": "",
"collectionID": "",
"userID": "",
"attemptID": ""
}
Every QuestionSet Hierarchy will attach “QuestionSetToken“ based on encrypted value of user-id, content-id, collection-id,attempt-id+selected_questionid_list recieved as part of hierarchy payload. This token will further be used to validate the request during response processing on server.
"questionSet": {
"timeLimits": "{\"maxTime\":\"3600\"}",
"questionSetToken": "", //#Question Set token to be generated at hierarchy read API with combination of "Question Set ID + userID"
"serverEvaluable": true //#true for Server Side Valuation Default:#false for client side validation
}
QuestionSetToken : This key is almost equivalent of jwt token created as follows:
“questionSetToken“ = > {
"contentID": "",
"collectionID": "",
"userID": "",
"attemptID": "",
“questionList“: [“<do-id-1>“,”<do-id-2>”,…..”<do-id-n>”]
}
Step 2: Trim Off correctResponse from Question List API
Question Read API : Any Question Associated with serverEvaluable behaviour to trim off response declaration from Question Set.
"serverEvaluable": true,
"responseDeclaration": {
"response1": {
"maxScore": 1,
"cardinality": "single",
"type": "integer",
-- To be Trimmed off ----
"correctResponse": {
"value": "0",
"outcomes": {
"SCORE": 1
}
},
-- To be Trimmed off --
}
},
There are multiple attributes which persists correct answer in QuML
a) responseDeclaration: (Shown above)
b) answer
c) editorState
Step 3: Introduce QuestionReponseValidate API to process user responses to QuestionSet.
QuestionResponseValidateAPI (Async API Behaviour):
QuestionSetToken generated in Hierarchy is sent as part of this request. This token will ensure the user’s attempt against questionSetID on a collection.
API accepts the request payload similar to content state update API.
{
"request": {
"userId": "843a9940-720f-43ed-a415-26bbfd3da9ef",
"questionSetToken": "",
"assessments":[
{
"assessmentTs": 1681284869464,
"batchId": "0132677340746629120",
"collectionId": "do_213267731619962880127",
"userId": "843a9940-720f-43ed-a415-26bbfd3da9ef",
"attemptId": "5486724f41afb4997118e6d97695684f",
"contentId": "do_2129959063404544001107"
},
"events":[],
responses:[{
"identifier":"<question-id>",
"questionType": "",
"userResponse":[""]
}]
}],
"contents": [
{
"contentId": "do_2132671468826214401203",
"batchId": "0132677340746629120",
"status": 2,
"courseId": "do_213267731619962880127",
"lastAccessTime": "2023-04-12 12:56:45:687+0530"
},
]
}
}
The Above payload mimics ContentStateUpdate API to mimic responses as above.
Question Set Response Processing flow
QuestionList Validation
In order to check for validity of Question inside a QuestionSet. Design proposes the value to be encrypted based on QuestionID List in hierarchy and persist this information against QuestionSetToken.
console.time("dbsave");
var crypto = require('crypto');
var encryptList = [];
var string = "sdgxsoksgdaodjqwdhuwdh";
var count = 0;
var salt = "salty";
for(var i=0;i<1000000;i++) {
var encrypt = crypto.pbkdf2('secret'+i, salt, 10, 64, 'sha512', (err, derivedKey) => {
if (err) throw err;
});
encryptList.push(encrypt);
if(string === encrypt) {
count++;
}
}
console.log(count);
console.log(encryptList.length);
console.timeEnd("dbsave");
Response Times for Benchmarking
1000000 encrypt generation dbsave: 4011.281ms 100000 encrypt generation dbsave: 377.208ms 10000 encrypt generation dbsave: 57.326ms