Bulk Upload Questions

Teachers have a large pool of questions they would have already created for conducting tests, quizzes, and exams. We want to leverage all these assets available in abundance in the community of educators through ecosystem participation.

Sunbird enables creation of question sets through various workflows enabled in Sourcing solution. We plan to complement those workflows by enabling bulk upload of questions in following workflows (in order of priority)

1. Bulk upload questions within a question set, where the question set might be created in a target collection driven sourcing project or in a taxonomy driven sourcing project.

2. Bulk upload questions within a question set, where the question set is the target object in a sourcing project.

3. Bulk upload questions in a framework driven sourcing project.

To enable the above capabilities fundamentally requires

1. Ability to bulk upload questions

2. Link them to a question set

We will be enabling bulk upload of questions and linking them to a question set using a CSV (comma separated values). Users are likely to use tools such as Google Sheets, Microsoft Office Excel, and other spreadsheet editing tools. Detailing out the key milestones below

Milestone 1: Bulk Upload Questions

The goal is to upload questions with its associate media (images)

1. Support Multiple Choice Question (MCQ) with validations such as

   1. Minimum 2 options, maximum 8 options

   2. At least and only one correct option

2. Question and Options can have images. Google Drive path for image is provided by user which is extracted by system (bulk upload tool). This is already available as a component. Read more here.

3. Every question can have following details (metadata)

   1. Name

   2. Taxonomy Framework Categories: Board, Medium, Class, Subject, Topic, Learning Outcome. These will derived from the Question set if questions are being uploaded within a question set.

   3. Keywords

   4. Author

   5. Attributions

4. User provides CSV in the prescribed format filled with required details

5. Basic validations such as

   1. Text contains only unicode characters

   2. Any cell does not contain any images

   3. All mandatory columns are filled for a particular row in CSV

• Kartheek Palla Please share a final format of CSV for Bulk Uploading Questions similar to bulk upload content

Milestone 2: Link Questions to a Question Set which are being bulk uploaded

The goal is to link questions at relevant place in a question set hierarchy structure

1. Level 1 Question Set Unit

2. Org_FW_topics

3. Target_FW_Medium

4. Target_FW_gradeLevel

5. Target_FW_subject

6. Target_FW_topic

Reference material

1. Bulk Upload Content related /wiki/spaces/DO/pages/1581350917

   1. https://project-sunbird.atlassian.net/browse/DP-18

   2. https://project-sunbird.atlassian.net/browse/DP-947

   3. https://project-sunbird.atlassian.net/browse/DP-1480

   4. https://project-sunbird.atlassian.net/browse/DP-967

   5. https://project-sunbird.atlassian.net/browse/DP-1480

Implementation details

1. Create a generate QuML API for various interaction types such that it takes required parameters as input and generate a QuML output. For example, Multiple Choice Question fundamentally contains Question, Options, and Correct Answer. The API takes these 3 as inputs in HTML or JSON format and generates a QuML spec. This also allows QuML to evolve rapidly as just by updating this ‘Generate QuML’ API with latest QuML spec, we can upgrade various places where Questions & Question sets are getting created such as Question Set Editor, Bulk Upload Questions & Question sets.

   1. This logic exists today ingrained in the Question creation component. It needs to be extracted out and made available as API / something.

Bulk Upload Question Set

https://project-sunbird.atlassian.net/browse/SB-22801

Many times organisations have a pool of questions available in some format - as Word docs on Hard disk, as Scan PDFs of exam papers, as questions in Excel sheet, and so on.

By enabling bulk upload of these questions or question sets would allows us to leverage what is already available in abundance and create maximum value out of it.

→ Bulk uploading of question sets can be done by providing data in the prescribed format. System will create questions, question sets and even link them to a collection (if details are provided).

→ Bulk upload format should support

1. Uploading of various question categories such as MCQ, Reference, FTB (in future), MTF (in future), and so on..

2. Providing Question set configuration and metadata

→ After bulk uploading, creator can review the uploaded question sets and bulk submit them for review.

→ System should have a configurable upper limit on number of questions in a question set that can be uploaded.

→ User flow should be similar to Bulk upload of content with an additional step of bulk submit for the contributor. (Check Issue navigator - JIRA (atlassian.net))

• User can download prescribed format

• Upload the data for bulk upload job

• (new) Preview question sets created as draft, edit, and submit

• (new) Bulk submit all question sets

→ Reviewer flow remain as-is similar to other bulk upload / approve projects.

Refer to current bulk upload process, template and guidelines here https://docs.google.com/document/d/1PW5b-Mdie6--wsFhGzC-fxVcPbvax4cA-hJyajr8Xew/edit#

https://project-sunbird.atlassian.net/browse/SB-23374

Bulk upload question set format should support

1. creating multiple question types (V1 will support MCQ with layouts, Subjective reference questions)

2. creating multiple question sets at once with their details & configurations

3. linking of these question sets to (target) collections

Bulk upload question set workflow is detailed out here in this ticket

https://project-sunbird.atlassian.net/browse/SB-23375

Contributors can access bulk upload question set functionality similar to how they would access bulk upload content functionality. Broadly, bulk upload can happen in context of project with target collection or in framework category driven projects.

As a contributor

1. I can download the prescribed format for uploading question set from the portal. Details of prescribed format are here

2. The question sets are uploaded as draft

3. Contributor can preview and edit any of the question sets, and the questions in them

4. Contributor can bulk submit question sets. Bulk Submit is can be performed for any asset in draft state. It will submit all the draft content for review.

As a reviewer, the workflow remains as-is - I can review assets individually or bulk approve.

At the time of uploading, the upload page has options such as

1. Upload as draft or Submit for review (on / off)

2. Question set category: (list of primary categories in the project) [Should this be a column in the sheet?]

Decision tree

1. Upload question sets in the project

   1. Format for each question type

   2. Format for all question types

2. Upload questions in a question set

   1. Format for each question type

   2. Format for all question types

Open questions

1. Should user upload Question set in a sourcing project or Upload questions in a Question set?

2. Should we have separate CSV formats for each question (interaction) type (primary category) or one single format supporting all types?

   1. Expectation is that each question is checked for validation before uploading

3. Should Images be supported? What are the limitations?

4. Questions and Question sets should be in separate sheets - Yes / No?

5. Benefits of using individual formats is that we can do client side validations such as.. Validations during upload (real-time) for Content upload

   1. Name of the columns

   2. Mandatory column cannot be blank

   3. Maximum 300 content per sheet

6. Other validations that happen async on server

   1. URL Validation while downloading from source (Google Drive)

   2. Framework values matching

   3. Primary category

   4. File Size limit

   5. ..

Question Set APIs:

http://docs.sunbird.org/latest/apis/questionapi//#tag/QuestionSet-APIs

Question APIs:

http://docs.sunbird.org/latest/apis/questionapi//#tag/Question-APIs