Page Comparison

Definition

For OGHR, Platform needs to enable user to undertake exams. Question set evaluation is currently on the client side and this implies that the correct answer is available to the client. We need to introduce the capability to process the user responses, evaluate and compute scores on the server side so that the correct answer does not have to be sent to the client side.

...

Background

In the present system, Question Set consist of following :

...

QuML Player : QuML Player has capability to play the questions in QuML format. It also has capability to evaluate and compute score. Response Validation, Score Computing is completely handled in player as of now. Once User Submits the overall response, client validated scores and response are sent to the backend as ASSESS events using Content State Update API.

...

...

...

Solution of Problem :

• Allow the assessment evaluation to be done on server also for question sets that are marked accordinglychosen to be server evaluable.

• Solution Answers to the question needs to be excluded from the Question Read API/ Question List API for server side assessment evaluation .

• Scalable server side response processing for Question Sets.

• Content State Update needs to happen from the server side response processing API.

• Response Processing can happen in two ways:

• Entire Question Set Response Processing (Current solution scope)

Question by Question Response Processing

questions.

...

Current Workflow

...

Solution Proposed

Image RemovedImage Added

Technical Design Details:

Creation side Enhancements:

Introduce “serverEvaluable“ “eval“ : {“mode“ : “client/server“} attribute at question and questionSet Object Metadata level

A new attribute in QuestionSet to be introduced to mark it for server side evaluation. The editor need to allow setting this flag on a questionSet/questions. The creation APIs would be updated to support this attribute. Any question that is marked as serverEvaluable:true “eval” : { “mode“ : “server“ } can only be part of a questionSet that is also marked as serverEvaluable:true“eval” : { “mode“ : “server“ } . Also a question set that is marked serverEvaluable:true “eval” : { “mode“ : “server“ } should only contain questions with serverEvaluable:true“eval” : { “mode“ : “server“ }.

(“serverEvaluable“evaluate at server “).

//Question Set Object
"questionSet": {
  “eval” : { “mode“ : "serverEvaluable": true“server“ } //#true for Server Side Valuation Default:#false for client side validation
}
//Question Object
"question" : {
  "serverEvaluable": true“eval” : { “mode“ : “server“ }
}

Consumption side Enhancements

There are two modes of accessing questionSet post this proposed change:
a) clientevaluable-evaluable mode : default ( serverEvaluable In case eval attribute does not exist or is falsemode = client)

In this mode, theplayer uses the new GET questionSetHierarchy to fetch the hierarchy, existing question list API to fetch question body and the existing content state update API to submit the ASSESS events. There are no changes to this processing as part of this changeCalculation of assess score remains at client side.

b) serverevaluable-evaluable mode : serverEvaluable:trueserver

We are proposing that, in this mode, the player uses a the new API for questionSetHierarchyPOST questionSetHierarchy to fetch the hierarchy, the existing question list API and a new submitAssessment APIthe existing Content State Update API without passing “score” & “pass”. Content State Update will fetch “score“ & “pass“ using new Inquiry Assessment API which introduced as part of this feature.

1. QuestionSet Hierarchy API (new POST API)

The current questionsetHierarchy API is a get call and does not take in arguments. Introduce a new POST method for QuestionSet Hierarchy API that can take in request body. This API will have payload as follows

...

follows

{
    "request": {
        "questionset": {
            "contentID": "",
            "collectionID": "",
            "userID": "",
            "attemptID": ""
        }
    }
}

This API would handle shuffling of options , selection of a subset of questions and randomization randomisation (currently done by player) as indicated by the metadata in the questionSet. The API will also return a “QuestionSetToken“ which is a signed token which has contains user-id, content-id, collection-id,attempt-id+selected_questionid_list recieved received , eval mode as part of hierarchy payload.This token will further be used to validate the request during the passed to Content State Update & to new submitAssessment API from Content State Update.“QuestionSetToken“ will be validated by submitAssessment API call on server.

"questionSet": {
  "timeLimits": "{\"maxTime\":\"3600\"}",
  "questionSetToken": "", //#Question Set token to be generated at hierarchy read API with combination of "Question Set ID + userID"
  "serverEvaluable": true“eval” : { “mode“ : “server“ } //#true for Server Side Valuation Default:#false for client side validation
}

2. Updates to QuestionList API

Question Read API : Any Question Associated with serverEvaluable eval-mode-server behaviour to trim off response declaration from Question SetSet and other answer displayed reference parameters in the response.

"serverEvaluable": true,
"responseDeclaration": {
          "response1": {
            "maxScore": 1,
            "cardinality": "single",
            "type": "integer",
            -- To be Trimmed off ----
            "correctResponse": {
              "value": "0",
              "outcomes": {
                "SCORE": 1
              }
            },
            -- To be Trimmed off --
           
          }
        },

...

3. SubmitAssessment Assessment API to evaluate user responses & calculate score.

QuestionResponseValidateAPI (Sync API Behaviour):

...

API accepts the request payload similar to content state update API.

...

{
  "request": {
        "assessmentTsuserId": 1681284869464,
   "843a9940-720f-43ed-a415-26bbfd3da9ef",
    "batchIdquestionSetToken": "0132677340746629120",

       "collectionIdassessments":[
"do_213267731619962880127",         "userId": "843a9940-720f-43ed-a415-26bbfd3da9ef",{
        "attemptIdassessmentTs": "5486724f41afb4997118e6d97695684f"1681284869464,
        "contentIdbatchId": "do_2129959063404544001107"
        }0132677340746629120",
        "eventscollectionId":[] "do_213267731619962880127",
        responses"userId":[{
  "843a9940-720f-43ed-a415-26bbfd3da9ef",
        "identifierattemptId": "<question-id>5486724f41afb4997118e6d97695684f",
 
        "questionTypecontentId": "do_2129959063404544001107"",
        },
 "userResponse":[""]         }]"events":[],
      }],
    "contents": [
      {
        "contentId": "do_2132671468826214401203",
        "batchId": "0132677340746629120",
        "status": 2,
        "courseId": "do_213267731619962880127",
        "lastAccessTime": "2023-04-12 12:56:45:687+0530"
      },
      
    ]
  }
}

...

Question Set Response Processing flow

...

...

Future Reference :

Questions List Validation using

...

back-end persistence

Current design validity check of assigned Questions from QuestionSet collection proposes assigned list of questionIds to be added in the JWT during hierarchy call & validate same during New Inquiry Assessment API. Storing same in Redis & Cassandra is an option which currently not considered.

• Scalable server side response processing for Question Sets.

• Response Processing can happen in two ways:

  • Entire Question Set Response Processing (Current solution scope)

  • Question by Question Response Processing