Content Visibility on Consumption and Souricing

Use case - Question paper

A state wants to source question paper for conducting SAT in the state. For the creation fo SAT question paper, the questions are sourced by teachers/ resource groups. Once the questions are added and a first level review happens. The reviewed and accepted questions are shared further with the State SCERT Assessment wing. The assessment wing, selects and curates a paper from the pool of reviewed questions. Once the question paper is curated:

Offline mode - The question paper’s Word/pdf doc is generated and distributed either via courier or email to districts to further print and conduct the examination in pen and paper format. However given the pandemic situation, the State has moved to Online mode.

Online mode - The question paper is shared to the state APP via Authorized and authenticated API call. The state app after receiving the question paper, serves it to students for consumption and stores relevant data points of student performance.

Key requirement: Before the exam - The questions / question paper must have limited visibility and be only accessible to authenticated and authorized users.
This applies to all the

1. Questions created for the SAT project - Draft, Submitted, Approved, Change Requested or Rejected

2. Question paper - All the approved questions by sourcing reviewer

After exam -

1. The state can choose to make the question paper visible to the students for practice - This will be available on the consumption portal (Dikhsa, or even external apps via API)

2. Only the questions approved by the sourcing reviewer to be part of the question paper are visible to the student

Overall Flow

For the first use case implementation the question paper will be created using the,

1. Editor version - 0.5 Editor

2. Target Collection - Question paper collection

3. Target Content - Exam Question.

User roles:

Admin - SCERT assessment wing admin who will create collections for all the question papers, create projects, and manage roles.

Question Contributor - Subject teachers or Resource person who will contribute a question

Question Reviewer - (Contribution reviewer) Subject matter expert who will review all the questions that are contributed and can accept or request for change

Question paper creator - (Sourcing reviewer) They select or reject the question that are submitted by question reviewer. All the selected questions are treated to be part of the question paper.

Question paper sourcing

1. Admin will create the question paper collection on Consumption portal - One for each question paper

2. Admin will create a project to source the questions for the collections.

   1. In one single project multiple collections can be created

   2. At the time of project creation the admin gets the option to edit the visibility of each of the question paper collection

   3. Default value for the question paper collection is Picked from the Question paper collection’s property

   4. The value set for the question paper collection is applicable to collection as well as target content (questions) created for the question paper.

3. Question paper is created on Diksha on a state channel, and only users mapped to that particular state (e.g. Haryana) are able to contribute to that question paper on Sourcing

Question paper creation flow

Contribution - Questions are added by Question contributors in the relevant question papers and projects.

Question reviewer - Question reviewer, reviews the questions for quality, correctness and tagging, if required requests for changes from the contributor or approve the question

Question paper creator - They get all the questions approved as part of the list, which they can further reject or accept form the list to curate the question paper.
- Once the question paper is created they set the question paper start and end date. This value is also passed on to question paper when it is published on consumption.
- The start and end date for a question paper can be updated/ modified. On modifying the date and saving on the sourcing portal, will also update the dates for the question paper published on the consumption

Admin - Admin will go to consumption (Dikhsa) and Submit the question paper collection for review. Once it is approved for publishing the question paper will be published on consumption. The visibility will remain as the same as was defined at the time of project creation.

Question paper consumption

In Online mode - The External registered application, will make an authorized and authenticated API call to Consumption to get the published ‘Question paper’ resource.

There are 3 steps to identifying a question paper:

1. Getting list of all the relevant question papers

   1. Search for All the question papers created and published by a the State (Haryana can search for resources created by Haryana)

   2. Search based on Board, medium, Grade, Subject

   3. Visibility status - Only search for questions papers that are published and have visibility of Private

   4. Question paper start and end date - All the question papers for which the current date/ date of request lies between the start and end date set for the question paper (by the question paper creator)

2. Identifying the question paper - After previous step the list of all the question papers are shared to the external app, then either

   1. Show all the question papers from the previous step to the user to choose, or

   2. Do some filtering at their end and show selected question papers to the users from the list

3. Getting the complete content of the question paper

   1. An API call is made to fetch the complete details of the questions in the question paper

      1. Question - Text, images (along with size), special symbols, etc. (along with question editor state)

      2. Answer key

      3. Tags and other properties - attached to the each question

After Exam

After the examination, the admin can choose to make the question paper public, so that students can access the past year question papers for practice.

Steps:

1. Admin requests the sourcing reviewer to edit the visibility property of one or more collections in the project

2. Sourcing reviewer modifies the visibility property of one or more question paper collection and saves. On save, the visibility property for the published question paper resource and questions inside it are also updated.

3. If made public - the question paper can now be visible under:

A state SCERT sources Question paper:

1.1 Question paper is created on Diksha on a state channel, and only users mapped to that particular state (e.g. Haryana) are able to contribute to that question paper on Sourcing

1.2 Contribution, Contribution Approval and Sourcing Approval happens on the Sourcing

1.3 When the sourcing reviewer has accepted all the questions that they want to include, they set the START and END Date for the question paper (This value of the question paper can be edited at a later time)

1.4 Admin Publishes the content on Diksha, however it is not visible publicly to all the Diksha users

1.5 The content is visible to only users/ API calls which have permission to view this content (Permission implies Users/Applications who have access to the content created by the state user/application is linked to and has access to the content with limited visibility)

1.6 Admin has the permission to modify the dates of the question paper

1.7 Admin has the provision to change the visibility of the question paper to Public (once the exam is done)

1.8 After it is made public all/any user can access the content directly on Diksha

Glossary:

Object Types - This is the master definition for Types of items that are supported on sunbird. These have properties for which each category can have a seperate value assigned to it.
e.g. - Question paper collection, Question Set, Questions, Content are all objects.

Category - This are specific instance of an object for which each of the property variable is assigned a value. Thos defines the type of object is being created. e.g. Question, Digital Textbook, Exam question, practice question set or any other.

Visibility types:

Content visibility:

Action that transfers data from Sourcing to Consumption - Sourcing Approval

Use case of Question paper

Questions that are Approved by sourcing reviewer → Gets added to a public question library

Questions that are Rejected by sourcing reviewer → Gets added to a private question library

Directly Publishing a question paper/ question object on the Diksha does not make sense for consumption.

Q1. Given this is the scenario, why do we even need the Publishing of any question paper on Consumption?

Q2. Do we just need to ensure that Question paper type content is not published ever to the consumption?

Ideal scenario: (taking the example of question sets)

Project creation:

• Create a new Question set - The default value for visibility is set. This can be modified by the Admin.

• Multiple Question sets, within a single Project - Visibility for each one can be set differently

  • This defines the question set behavior uniquely under a project.

Use cases:

1. Question paper for an upcoming exam has Visibility = Private

2. Question paper of a past exam has visibility = Public

3. Question paper with visibility = Private can be modified to be public after the exam is conducted

If an object has visibility as private will imply:

• The public link for all the content inside the object will stop working

• The search API will stop showing the content in the search result

Adhoc solution:

What was tried:

1. We updated default visibility of Question Paper & Exam Question categories to private

2. After this, the status of these would remain as Processing in Sourcing workflow because as soon as they were Approved, the system would publish them with visibility = private.

Issues with this approach:

1. The Sourcing workflow uses Search API to list content linked to a collection in order to show it to Contributors & Reviewers. These Search calls do not specify any visibility explicitly, so we would see only content with default visibility.

2. Thus we need to revert the Visibility back to default, and still solve for this content not being visible to anyone on the consumption portal.

Implemented Workaround:

1. As a workaround, based on Printable = true we updated the “Approve” implementation. Once the content is Approved it will be marked as approved (because only then Print preview will show it) but it won’t copy this content to consumption repo (so that it is not searchable).

Correct solution:

Phase 1: Inherit the Visibility from the Config for each object at the time of the creation

1. Visibility = private to be supported in API

2. Sourcing flows to be updated to support private visibility

   1. Publish logic (which is part of sourcing flow) should be updated too.

3. Remove the patchy implementation which is based on Printable attribute at the moment.

Open Questions:

1. What setup would be required to make the relevant changes in the API?

2. What is the backend stack?

3. Will the questions, when shared via API will from Consumption or Sourcing? What is the difference between telemetry from either of them?