inQuiry- [Design]- API Implementation of QuML Multi-language

Background:

inQuiry need to enable multi-language support for Questions & QuestionSets where data like question content (body), hints, instructions can be stored in multiple language. So that user can select the language of their choice (user may select one or multiple language) during creation & consumption.

Problem Statement:

Currently Question & QuestionSet V1 api’s doesn’t support multilingual data for body, hints, instructions, feedback etc.
As per QuML Spec below metadata should support multi-language:

objectType	metadata	Current Data Type (v1 api)	Expected Data Type (v2 api)
QuestionSet	instructions	object	object
QuestionSet	feedback	object	object
Question	body	string	object
Question	instructions	object	object
Question	answer	string	object
Question	hints	array of string	object
Question	feedback	object	object
Question	solutions	array of object	object
Question	interactions	object	object

Solution:

inQuiry will provide Question & QuestionSet V2 api’s which will provide CRUD operation for multilingual data.

api version	qumlVersion	schemaVersion	version	compatibilityLevel
v1	v1 but not stamped in the data as of now.	1.0	1	5
v2	1.0	2.0	2	6

version or schemaVersion metadata will be set as 2 by default from micro-service for v2 api’s.
Current compatibilityLevel is set to 5 for the data created using v1 api’s. For v2 api’s, it will be set to 6 so that users should be able to discover and play only on latest app having latest player.
async-questionset-publish flink job will handle different data validation based on schemaVersion / version.
schemaVersion will be used inside player, based on schemaVersion, player will switch the logic to process data.
player won't understand comaptibilityLevel, but will upgrade it to 6 for backward compatibility (existing apps).
data versioning will be implemented to store multiple version data at platform.
if quml specification will get updated in future,
- if it is a minor change, the change can be accommodated in v2 api’s.
- if spec will have major breaking change, we need to write next version of api’s.
For existing data created using v1 api’s, we have two approach as mentioned below:
- No static Data Migration for existing objects and data transformation in question read & list api at run time to support v2 api spec.
- One time static Data Migration

No Static Migration:

Pros:

Old version of questionset-editor & mobile app works fine with v1 api.
Once older questions edited using v2 editor and api's, it will be migrated to v2 data model. So end user (consumption) won’t be affected on immediate basis and at that point old player will not be able to discover the question (comaptibilityLevel will be upgraded to maintain backward compatibility of Sunbird-Ed mobile app).

Cons:

Platform need to understand all multi-language data (including editorState) and transform it to v2 api format for every read & list api call.
API Performance will be affected (specially question list api). To improve performance, cache will be implemented for list api to hold data post data transformation (currently no cache implementation present for list api).
Once the data is migrated to v2 data model, old player will not be able to discover the question (comaptibilityLevel will be upgraded).

Static Migration:

Pros:

Consumption api’s (question read & list api) doesn't need any run-time data transformation. So api performance won’t be affected.

Cons:

old mobile app won't be able to render existing questions for online consumption.
old editor also start breaking even for v1 api’s

API Specification:

Question Create API:

request_1:

{
  "request": {
    "question": {
      "name": "question_title",
      "code": "{{$randomUUID}}",
      "mimeType": "application/vnd.sunbird.question",
      "primaryCategory": "Multiple Choice Question",
      "body": "question body as string when question can be used in only one language.",
      "instructions": {
        "language_code": "instructions in HTML format."
      },
      "answer": "answer",
      "hints": [
        "hint-1",
        "hint-2"
      ],
      "feedback": {
        "feedback_id_1": "feedback-1",
        "feedback_id_2": "feedback-2"
      },
      "solutions": [
        {
          "id": "bc6e4294-1a19-4e75-fb43-871f0e54b0e8",
          "type": "html",
          "value": "solution in HTML format"
        },
        {
          "id": "bc6e4294-1a19-4e75-fb43-871f0e54b0e9",
          "type": "html",
          "value": "a question can have more than one solution. in such cases, another solution in HTML format"
        }
      ],
      "interactions": {
        "response1": {
          "type": "choice",
          "options": [
            {
              "label": "<p>Floral organs</p>",
              "value": 0
            },
            {
              "label": "<p>Veins and veinlets in a lamina</p>",
              "value": 1
            }
          ]
        },
        "validation": {
          "required": "Yes"
        }
      }
    }
  }
}

In the above request spec, a default language code (configured in the system at object level schema) will be injected and data will be transformed (as mentioned in request_2 ) to v2 format and then saved to database.
The data transformation will be done to support both formats mentioned in quml specification.

request_2:

{
  "request": {
    "question": {
      "name": "question_title",
      "code": "{{$randomUUID}}",
      "mimeType": "application/vnd.sunbird.question",
      "primaryCategory": "Multiple Choice Question",
      "body": {
        "language_code_1": "question body as string in specified language."
      },
      "instructions": {
        "language_code": "instructions in HTML format."
      },
      "answer": {
        "language_code": "answer in specified language"
      },
      "hints": {
        "language_code": [
          "hint-1",
          "hint-2"
        ]
      },
      "feedback": {
        "feedback_id_1": {
          "language_code": "feedback-1"
        },
        "feedback_id_2": {
          "language_code": "feedback-2"
        }
      },
      "solutions": {
        "language_code": [
          {
            "id": "bc6e4294-1a19-4e75-fb43-871f0e54b0e8",
            "type": "html",
            "value": "solution in HTML format"
          },
          {
            "id": "bc6e4294-1a19-4e75-fb43-871f0e54b0e9",
            "type": "html",
            "value": "a question can have more than one solution. in such cases, another solution in HTML format"
          }
        ]
      },
      "interactions": {
        "response1": {
          "type": "choice",
          "options": [
            {
              "label": {
                "language_code": "<p>Floral organs</p>"
              },
              "value": 0
            },
            {
              "label": {
                "language_code": "<p>Veins and veinlets in a lamina</p>"
              },
              "value": 1
            }
          ]
        },
        "validation": {
          "required": "Yes"
        }
      }
    }
  }
}

Response:

{
    "id": "api.question.create",
    "ver": "5.0",
    "ts": "2023-01-30T19:43:43ZZ",
    "params": {
        "resmsgid": "ca1ed6b6-994d-487c-8ecd-0ca164a84c94",
        "msgid": null,
        "err": null,
        "status": "successful",
        "errmsg": null
    },
    "responseCode": "OK",
    "result": {
        "identifier": "do_2137224832856555521270",
        "versionKey": "1675107822985"
    }
}

Question Read API:

Request:

curl --location --request GET 'https://dev.inquiry.sunbird.org/api/question/v2/read/do_213688205764337664161?mode=edit&fields=answer,instructions' \
--header 'Authorization: {{api_key_new}}'

Response:

question read api will always return will data in below format:

{
  "id": "api.question.read",
  "ver": "5.0",
  "ts": "2023-01-30T03:28:23ZZ",
  "params": {
    "resmsgid": "db23eb9e-207c-488c-a45a-7da36a2084ee",
    "msgid": null,
    "err": null,
    "status": "successful",
    "errmsg": null
  },
  "responseCode": "OK",
  "result": {
    "question": {
      "identifier": "do_213688205764337664161",
      "body": {
        "language_code_1": "question body as string in specified language."
      },
      "instructions": {
        "language_code": "instructions in HTML format."
      },
      "answer": {
        "language_code": "answer in specified language"
      },
      "hints": {
        "language_code": [
          "hint-1",
          "hint-2"
        ]
      },
      "feedback": {
        "feedback_id_1": {
          "language_code": "feedback-1"
        },
        "feedback_id_2": {
          "language_code": "feedback-2"
        }
      },
      "solutions": {
        "language_code": [
          {
            "id": "bc6e4294-1a19-4e75-fb43-871f0e54b0e8",
            "type": "html",
            "value": "solution in HTML format"
          },
          {
            "id": "bc6e4294-1a19-4e75-fb43-871f0e54b0e9",
            "type": "html",
            "value": "a question can have more than one solution. in such cases, another solution in HTML format"
          }
        ]
      },
      "interactions": {
        "response1": {
          "type": "choice",
          "options": [
            {
              "label": {
                "language_code": "<p>Floral organs</p>"
              },
              "value": 0
            },
            {
              "label": {
                "language_code": "<p>Veins and veinlets in a lamina</p>"
              },
              "value": 1
            }
          ]
        },
        "validation": {
          "required": "Yes"
        }
      }
    }
  }
}

the read api transforms data created using v1 api’s or spec-1 of v2 api’s. It injects default language code configured in the system and return the response to user.
same transformation will be applicable in Question List API

Question Update API:

Question Update API also support both spec as mentioned in create api.
For older data (created with v1 api’s), on v2 api update call, data will be migrated to v2 api format as well as version will be set to 2.

Request_1:

{
  "request": {
    "question": {
      "name": "updated_question_title",
      "versionKey": "1234",
      "body": "question body as string when question can be used in only one language.",
      "instructions": {
        "language_code": "instructions in HTML format."
      },
      "answer": "answer",
      "hints": [
        "hint-1",
        "hint-2"
      ],
      "feedback": {
        "feedback_id_1": "feedback-1",
        "feedback_id_2": "feedback-2"
      },
      "solutions": [
        {
          "id": "bc6e4294-1a19-4e75-fb43-871f0e54b0e8",
          "type": "html",
          "value": "solution in HTML format"
        },
        {
          "id": "bc6e4294-1a19-4e75-fb43-871f0e54b0e9",
          "type": "html",
          "value": "a question can have more than one solution. in such cases, another solution in HTML format"
        }
      ],
      "interactions": {
        "response1": {
          "type": "choice",
          "options": [
            {
              "label": "<p>Floral organs</p>",
              "value": 0
            },
            {
              "label": "<p>Veins and veinlets in a lamina</p>",
              "value": 1
            }
          ]
        },
        "validation": {
          "required": "Yes"
        }
      }
    }
  }
}

Request_2:

{
  "request": {
    "question": {
      "name": "updated_question_title",
      "versionKey": "1234",
      "body": {
        "language_code_1": "question body as string in specified language."
      },
      "instructions": {
        "language_code": "instructions in HTML format."
      },
      "answer": {
        "language_code": "answer in specified language"
      },
      "hints": {
        "language_code": [
          "hint-1",
          "hint-2"
        ]
      },
      "feedback": {
        "feedback_id_1": {
          "language_code": "feedback-1"
        },
        "feedback_id_2": {
          "language_code": "feedback-2"
        }
      },
      "solutions": {
        "language_code": [
          {
            "id": "bc6e4294-1a19-4e75-fb43-871f0e54b0e8",
            "type": "html",
            "value": "solution in HTML format"
          },
          {
            "id": "bc6e4294-1a19-4e75-fb43-871f0e54b0e9",
            "type": "html",
            "value": "a question can have more than one solution. in such cases, another solution in HTML format"
          }
        ]
      },
      "interactions": {
        "response1": {
          "type": "choice",
          "options": [
            {
              "label": {
                "language_code": "<p>Floral organs</p>"
              },
              "value": 0
            },
            {
              "label": {
                "language_code": "<p>Veins and veinlets in a lamina</p>"
              },
              "value": 1
            }
          ]
        },
        "validation": {
          "required": "Yes"
        }
      }
    }
  }
}

Response:

{
    "id": "api.question.update",
    "ver": "5.0",
    "ts": "2023-01-30T19:43:43ZZ",
    "params": {
        "resmsgid": "ca1ed6b6-994d-487c-8ecd-0ca164a84c94",
        "msgid": null,
        "err": null,
        "status": "successful",
        "errmsg": null
    },
    "responseCode": "OK",
    "result": {
        "identifier": "do_2137224832856555521270",
        "versionKey": "1675107822985"
    }
}

Question Review API & Question Retire API:

there is no change in spec from v1 to v2 api.
there is no change for logical behaviour of the api

Question Publish API:

there is no change in spec from v1 to v2 api.
For older data, the api will throw client error if the data version is 1.

Question Copy API:

there is no change in spec from v1 to v2 api.
if source object version is 1, copied data will be created using v2 format.

QuestionSet API’s:

will be updated soon..