...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
Bulk Upload Questions
https://project-sunbird.atlassian.net/browse/SB-22801
Teachers have a large pool of questions they would have already created for conducting tests, quizzes, and exams. We want to leverage all these assets available in abundance in the community of educators through ecosystem participation. We have seen following scenarios where the need of bulk upload of questions has come up:
An organisation already have questions in an existing system. Hence creating them manually through UI is a lot of effort
An organisation is creating new questions, but wants to import questions in multiple systems, not just Sunbird (e.g. DIKSHA). Hence they would like to keep the questions in a common spreadsheet format and bulk uploaded into multiple systems
An organization is creating new questions and also simultaneously getting the questions translated into multiple languages. Hence having the questions in a spreadsheet is easy to share them with multiple translators and get the translations done.
Sunbird enables creation of question sets through various workflows enabled in Sourcing solution. We plan to complement those workflows by enabling bulk upload of questions in following workflows (in order of priority)
Bulk upload questions within a question set such that questions are linked at the right place in the question set, where . Here the question set might be created in a target collection driven sourcing project or in a taxonomy driven sourcing project.
Bulk upload questions within a question set , where such that questions are linked at the right place in the question set. Here the question set is the target object in a sourcing project.
(Future) Bulk upload questions in a framework driven sourcing project.
To enable the 1 & 2 above, capabilities fundamentally requires
Ability to bulk upload questions
Link them to at the right place in a question set
We will be enabling bulk upload of questions and linking them to a question set using a considering CSV (comma separated values) input. Users are likely to use tools such as Google Sheets, Microsoft Office Excel, and other spreadsheet editing tools. Detailing out the key milestones belowMilestone 1: Bulk Upload Questions
The goal is to upload questions with its associate media (images)
Support Multiple Choice Question (MCQ) with validations such as
Minimum 2 options, maximum 8 4 options
At least and only one correct option
Question and Options can have images - provided as Google Drive public link. Google Drive path for image is provided by user which is extracted by system (bulk upload tool). This is already available as a component. Read more here.
Every question can have following details (metadata)
Name
Taxonomy Framework Categories: Board, Medium, Class, Subject, Topic, Learning Outcome. These will derived from the Question set if questions are being uploaded within a question set.
Keywords
Author
Attributions
.
Images will be placed at the beginning of the text with left align and small size (25%).
Question and Options can have either Images only or text only or both.
User can download sample bulk upload format. User provides CSV in the prescribed format filled with required details.
Mandatory columns (configurable) are coloured in red.
Basic validations such as
Text contains only unicode characters
Any cell does not contain any images
All mandatory columns are filled for a particular row in CSV
- Kartheek Palla Please share a final format of CSV for Bulk Uploading Questions similar to bulk upload content
Milestone 2: Link Questions to a Question Set which are being bulk uploaded
The goal is to link questions at relevant place in a question set hierarchy structure
Level 1 Question Set Unit
Org_FW_topics
Target_FW_Medium
Target_FW_gradeLevel
Target_FW_subject
Target_FW_topic
Reference material
...
Bulk Upload Content related /wiki/spaces/DO/pages/1581350917
Implementation details
Create a generate QuML API for various interaction types such that it takes required parameters as input and generate a QuML output. For example, Multiple Choice Question fundamentally contains Question, Options, and Correct Answer. The API takes these 3 as inputs in HTML or JSON format and generates a QuML spec. This also allows QuML to evolve rapidly as just by updating this ‘Generate QuML’ API with latest QuML spec, we can upgrade various places where Questions & Question sets are getting created such as Question Set Editor, Bulk Upload Questions & Question sets.
This logic exists today ingrained in the Question creation component. It needs to be extracted out and made available as API / something.
| Note |
|---|
Please do not refer to section below. Information below is outdated and will be deleted soon. |
Bulk Upload Question Set
https://project-sunbird.atlassian.net/browse/SB-22801
Many times organisations have a pool of questions available in some format - as Word docs on Hard disk, as Scan PDFs of exam papers, as questions in Excel sheet, and so on.
By enabling bulk upload of these questions or question sets would allows us to leverage what is already available in abundance and create maximum value out of it.
→ Bulk uploading of question sets can be done by providing data in the prescribed format. System will create questions, question sets and even link them to a collection (if details are provided).
→ Bulk upload format should support
Uploading of various question categories such as MCQ, Reference, FTB (in future), MTF (in future), and so on..
Providing Question set configuration and metadata
→ After bulk uploading, creator can review the uploaded question sets and bulk submit them for review.
→ System should have a configurable upper limit on number of questions in a question set that can be uploaded.
→ User flow should be similar to Bulk upload of content with an additional step of bulk submit for the contributor. (Check Issue navigator - JIRA (atlassian.net))
User can download prescribed format
Upload the data for bulk upload job
(new) Preview question sets created as draft, edit, and submit
(new) Bulk submit all question sets
→ Reviewer flow remain as-is similar to other bulk upload / approve projects.
Refer to current bulk upload process, template and guidelines here https://docs.google.com/document/d/1PW5b-Mdie6--wsFhGzC-fxVcPbvax4cA-hJyajr8Xew/edit#
https://project-sunbird.atlassian.net/browse/SB-23374
Bulk upload question set format should support
creating multiple question types (V1 will support MCQ with layouts, Subjective reference questions)
creating multiple question sets at once with their details & configurations
linking of these question sets to (target) collections
Bulk upload question set workflow is detailed out here in this ticket
https://project-sunbird.atlassian.net/browse/SB-23375
Contributors can access bulk upload question set functionality similar to how they would access bulk upload content functionality. Broadly, bulk upload can happen in context of project with target collection or in framework category driven projects.
As a contributor
I can download the prescribed format for uploading question set from the portal. Details of prescribed format are here
The question sets are uploaded as draft
Contributor can preview and edit any of the question sets, and the questions in them
Contributor can bulk submit question sets. Bulk Submit is can be performed for any asset in draft state. It will submit all the draft content for review.
As a reviewer, the workflow remains as-is - I can review assets individually or bulk approve.
...
Columns / Information to be provided by Contributor
...
Description
...
In which version
...
QUESTION DETAILS
...
Question Name / Title *
...
Human readable / relatable Name or Title or any identifying value for searching and retrieval later
...
V1
...
Question Primary Category *
...
MCQ or Subjective?
...
V1 - MCQ
...
Question Additional Category
...
Derived from target collection or question set
...
Framework categories -
Org & Target FW
(If K-12, then B,M,C,S, Topics, LO
If TPD, then Subjects, Topics)
...
Should be able to create questions for any of the frameworks. If uploaded within a target collection, then relevant metadata (B,M,C,S) is derived from it. Additional metadata (Topics, LO) can be provided.
...
V1 - B,M,C,S
...
Question Body *
...
Body or stem of the question
...
V1
...
Question Image
...
Image to be used for the question body
User provides Google Drive path. System will insert at the beginning of the text, small size, left aligned
...
Options
...
Should support min 2 & max 8 options
...
V1 - Only 4
...
Option Images
...
Should support Google Drive link for the images. Insert images always at the beginning of the question text in small size & left alignment
...
V1
...
Option Layout
...
Support Horizontal, Vertical, and Grid layout. Default = vertical.
...
V1
...
Answer
...
Should be a number between 1 - 8 for MCQ. Can be text in case of Subjective questions
...
V1
...
Solution Text
...
V2
...
Solution Image
...
V2
...
Solution Video
...
V2
...
Author
...
Provide if not same as user name of the creator ID on the platform
...
Attributions
...
Copyright
...
Default = Tenant name ?
...
License
...
Default / Always = CC BY … ?
...
QUESTION SET DETAILS 👇🏽
...
Question set ID
...
Any unique ID to group the questions
...
V1
...
Question set name
...
Question set category
...
Question set framework
...
Derived from target collection
...
Question set settings: Shuffle
...
On / Off
...
Question set settings: Display
...
Any number < 0 and < total number of questions in the set
...
Question set settings: Feedback
...
On / Off
...
Question set settings: Submit
...
On / Off
...
Question set settings: Attempts
...
0 < Attempts < 25
...
Author
...
Provide if not same as user name of the creator ID on the platform
...
Attributions
...
Copyright
...
Default = Tenant name ?
...
License
...
Default / Always = CC BY … ?
At the time of uploading, the upload page has options such as
Upload as draft or Submit for review (on / off)
Question set category: (list of primary categories in the project) [Should this be a column in the sheet?]
Decision tree
Upload question sets in the project
Format for each question type
Format for all question types
Upload questions in a question set
Format for each question type
Format for all question types
Open questions
...
Should user upload Question set in a sourcing project or Upload questions in a Question set?
...
Should we have separate CSV formats for each question (interaction) type (primary category) or one single format supporting all types?
Expectation is that each question is checked for validation before uploading
...
Should Images be supported? What are the limitations?
...
Questions and Question sets should be in separate sheets - Yes / No?
...
Benefits of using individual formats is that we can do client side validations such as.. Validations during upload (real-time) for Content upload
Name of the columns
Mandatory column cannot be blank
Maximum 300 content per sheet
Other validations that happen async on server
...
URL Validation while downloading from source (Google Drive)
...
Framework values matching
...
Primary category
...
File Size limit
...
Explanation of Bulk Upload Question format
https://docs.google.com/spreadsheets/d/1ndzapGGV6q8698x-NQzK_ufln4YX1HQ09jsFsC7kA60/edit?usp=sharing
Details to be provided by contributor using CSV format
Name is set to default as Multiple Choice Question for this version.
Mandatory: No
Default: Multiple Choice Question
Validations: - Upto 120 characters - No special characters
Tip: Please provide name for the question
Question Text will support text as per the Text validations listed below. Maximum character limit for Question Text is 1000 (configurable). Question Text and Question Image are merged to compose the Question Body. Question Body is required to create a question in the platform.
Mandatory: Conditional - any one of text or image should be provided.
Default: None
Validation: Length not more than 1000 characters.
Tip: Provide text in Unicode.
Question Image will support Google Drive path (publicly accessible) for images. Images will be placed at the beginning of the Question Text with left align and small size (25%) styling.
Mandatory: Conditional - any one of text or image should be provided.
Default: None
Validation: Only JPG format. Google Drive Link - Publicly accessible.
Tip: Only JPG format. Google Drive Link - Publicly accessible.
Option Layout will support three possible values: Horizontal, Vertical, Grid OR 1, 2, 3. Default value = Vertical.
Mandatory: Yes
Default: None
Validation: Only 1, 2, or 3
Tip: Layout number = Needs involved decision making, depending on question length and answer/option length.
1 = vertical (most flexible)
2 = horizontal (suitable for one-two words or images)
3 = grid (suitable for a few words or images)
OptionX will support text as per the Text validations listed below. Maximum character limit for Option Text is 500 (configurable). Option Text and Option Image are merged to compose the Option Body. Option Body is required to create a question in the platform.
(Same as Question Text)Mandatory: Conditional - any one of text or image should be provided.
Default: None
Validation: Length not more than 1000 characters. Minimum 2 options, maximum 4 options.
Tip: Provide text in Unicode.
OptionXImage will support Google Drive path (publicly accessible) for images. Images will be placed at the beginning of the Option Text with left align and small size (25%) styling.
(Same as Question Image)Mandatory: Conditional - any one of text or image should be provided.
Default: None
Validation: Only JPG format. Google Drive Link - Publicly accessible.
Tip: Only JPG format. Google Drive Link - Publicly accessible.
Answer No will be a number between 1 to 4.
Mandatory: Yes
Default: None
Validation: At least and only one correct option. Only 1, 2, 3, or 4. If user has provided only 2 options and provided “3” as correct answer, system should throw an error.
Tip: Enter Correct answer option value between 1 to 4
Level 1 Question Set Section will be Level 1 Section’s (unit) Name so that question can be linked to that folder.
Mandatory: No
Default: None
Validation: Should match with the names provided in the question set hierarchy. Not case-sensitive / Case insensitive.
Tip: Provide name of the folder where you want to upload a question
Keywords
Mandatory: No
Default: None
Validation: Comma separated values. No special characters.
Tip: Keywords
Audience
Mandatory: No
Default: Audience category of the Question Set (derived value).
Validation: Supported values in the platform
Tip: Audience of the question. Should be same as audience of the question set.
Author
Mandatory: No
Default: None
Validation: No special characters. Max 300 characters.
Tip: The person or organization who has authored the content
Copyright
Mandatory: No
Default: Name of the tenant.
Validation: No special characters. Max 300 characters.
Tip: Person or Organization who owns the copyright. Default name is DIKSHA tenant.
Attributions
Mandatory: No
Default: None
Validation: No special characters. Max 300 characters.
Tip: List of persons or organizations who have contributed to this content.
Details to be pre-filled / derived by system
Question Category will be Multiple Choice Question for this version.
Mandatory: Yes
Default: MCQ
Validation: If anything other than MCQ, reject the question
Tip: Always fill "MCQ"
Additional Category is derived from the question set
Mandatory: No
Default: Primary category of the Question Set (derived value).
Validation:
Tip: Purpose of the question. Should be same as content type of question set.
Target Question Set ID to be auto-derived when user is uploading questions within a question set. This will not be shown in the CSV upload format to the contributor.
Mandatory: Yes
Default: Question Set ID where Question Upload file was provided
Validation: Should exist in the platform
Tip: Identifier of the Question Set where questions are to be linked
Taxonomy Framework Categories: These will derived from the Question set when questions are being uploaded within a question set. Questions can be uploaded in a question created for any framework - all available categories should be derived and tagged to questions. Below are sample categories for K-12 framework:
Org_FW_Board
Org_FW_Medium
Org_FW_Class
Org_FW_Subject
Org_FW_Topic
Org_FW_LearningOutcome is not supported in bulk upload sheet. User can edit from UI.
Org_FW_Skill is not supported in bulk upload sheet. User can edit from UI.
License
Mandatory: No
Default: License of the tenant (derived).
Validation: No special characters. Max 300 characters.
Tip: One of the supported licenses in DIKSHA. If this is empty, default to the default license configured for the tenant.
Editing questions and details: After uploading questions, contributor can edit details as configured in the primary category of the question set. Math Formulae, Additional text formatting, and other rich text features can be used in the editor.
User flow - Contributor
Flow 1: Contributors will be able to upload questions within a question set
As a contributor I should be able to bulk upload questions for a question set in a sourcing project. Following is the flow to enable this:
When a contributor logs into contribution portal and opens a target collection page of a project to which she can contribute (i.e. her nomination is accepted), currently against each target collection there is a “Create New” action using which user can create a question set.
Given user has access to a sourcing project where she can contribute, When she creates a question set within the sourcing project, Then she can upload questions to the question set
User will see an option to “Bulk Upload Question” in the question set similar to “QR codes” in the collection/question set editor.
Clicking “Bulk Upload Question” option, Bulk Upload Question screen should open up. The screen should have following options
Option to select a Bulk Upload Question (metadata) file from local folder (of user’s system)
Assumption: the metadata file will have publicly accessible URLs to the question related filesThere is a link to sample metadata file: “Sample Bulk Upload Question metadata file”
User selects metadata file, user clicks “Upload”. System provides a message “Validating file”
The system should first validate metadata file against the selected files. Following are the validations:
All the columns are available
All the mandatory columns have values filled in
In case there are errors in the metadata file validation, display relevant error message on the Upload dialog
Some columns are not available:
“Metadata file validation failed. Following columns are not found in the file. Please check and upload again: <list the missing column names>”Some mandatory columns have values missing:
“Metadata file validation failed. Following rows have missing values. Please check and upload again: <list the row numbers (starting from 1) with missing values>“
In case of metadata file validation errors, “Upload” button is disabled unless user re-selects a metadata file again.
In case metadata file doesn’t have any validation errors, the dialog shows
“Bulk Upload is in progress.
Number of questions uploaded successfully: <no.>
Number of questions failed: <no.>
Number of questions pending: <no.>”After the bulk upload is complete. There is an option to download status report as a csv. The status report should include identifier of the question (as generated by the system i.e. API response) and status (Success, Failure, Error, Invalid, etc)
User can close it dialog box while a bulk upload is in progress.
In the Question Set page, whenever user clicks “Bulk Upload Questions”, in case a bulk upload is in progress, it shows the status dialog as described in point 8. (This will be in future since we do not have support for background / minimised upload in the first version)
User can edit any question after it is uploaded and saved as draft in the question set. Using this user can Learning Outcome or any other detail of the question.
Limits on number of question, size:
Number of question per question set for one bulk upload job: 300. Maximum 300 questions per CSV
Maximum size of each content is same as max size supported by system.
...
Reference material
Bulk Upload Question Guidelines (current manual script driven process) https://docs.google.com/document/d/1PW5b-Mdie6--wsFhGzC-fxVcPbvax4cA-hJyajr8Xew/edit#heading=h.we2s93hce9f4
Bulk Upload Content related /wiki/spaces/DO/pages/1581350917
Implementation details
Create a generate QuML API for various interaction types such that it takes required parameters as input and generate a QuML output. For example, Multiple Choice Question fundamentally contains Question, Options, and Correct Answer. The API takes these 3 as inputs in HTML or JSON format and generates a QuML spec. This also allows QuML to evolve rapidly as just by updating this ‘Generate QuML’ API with latest QuML spec, we can upgrade various places where Questions & Question sets are getting created such as Question Set Editor, Bulk Upload Questions & Question sets.
This logic exists today ingrained in the Question creation component. It needs to be extracted out and made available as API / something.
Kartheek Palla What will be the technology stack? Please list out specific tech stack
Angular (version 9) and Java (with Scala)
Images: Extracting files from Google Drive is already available as a component in context of ‘Bulk Upload Content’. Read more here.
Images also exist independently as Media Asset in the system - so first they need to be uploaded and then the system generated identifier need to be referred in the Question Body.
Question Set APIs:
http://docs.sunbird.org/latest/apis/questionapi//#tag/QuestionSet-APIs
Question APIs:
http://docs.sunbird.org/latest/apis/questionapi//#tag/Question-APIs