Page Comparison

Problem Statement

1. Currently, the resource creators have to create the entire observation, survey and observation with rubric form via excel sheet templates which is a hectic and tiresome process. Furthermore, the implementation/support team has to rework on this and upload it at the backend after checking. This creates a lot of friction in the creation of the form.

2. The creation of observation is not self-serviceable and requires a lot of hand holding at each step of the creation.

3. The creator doesn’t get an idea of the look and feel of the questions and question sets on the app while creating the form.

High Level Approach

A self-serviceable creation portal should be built for the resource creators to create observation or survey forms.

Personas / Narrative

• Resource creators - To create the observation/survey via the question set editor

• Reviewer - To review the observation/survey and give feedback

• Sourcing Admin - To manage user role, create sourcing projects and access and track progress of sourcing activities.

Goals

1. The creator should be able to create questions and group them at the same time using the question set editor.

2. The creators should be able to create the observation form with ease in self service mode.

3. A review process for all the created forms is enabled on the portal.

4. The dependencies for creation should not come on the support or implementation team.

5. The creation of all the different resource types i.e. Observation and survey, should be at a single place.

6. Sourcing admin should be able to create a project to seek content for “Not for a target collection” and seek survey, observation and observation with rubric as content type in the sourcing project.

7. Sourcing Admin can manage user roles, access, and track progress of sourcing activities.

Solution Alignment

To enable the creation of the following two primary categories of content for the object type Question Set:

1. Observation - Observations are simple forms for collecting data for specific processes/functions.
   The observations are recorded for/against an entity. An entity can be an individual or an institution. Data is collected for/against that selected entity (Institution/Individual). An observation form can be submitted multiple times by a user as per their need and requirement by generating multiple instances of the same form. For eg:

   • Class observations for a class/teacher in a school by the CRPs for all schools in a cluster on a weekly basis.

   • An observation of the daily processes and functioning of the school by a BEO
     

2. Observation with rubric - For a comprehensive evaluation and/or accreditation purpose, such observations are conducted on the basis of a rubric driven quality framework. A framework consists of multiple domains and criteria. Levels with particular score ranges through the scoring for questions are also defined for the criteria. This provides a quantified measurement for both objective and subjective parameters to be assessed. Subjective parameters like effectiveness of teacher instructions or classroom quality and objective parameters like no. of toilets, student teacher ratio or sports equipment. For eg:

   • Comprehensive assessment of Schools by the HMs on the basis of a framework like Shala Siddhi or School Quality Assessment and Assurance, SQAA.
     

3. Survey - Surveys provide data as opinion / feedback of one who is filling the survey. These are not taken up or recorded against any specific entity. They can only be taken up once as the surveys are filled as feedback or information or ideas for an event/activity/process. Surveys can also be time bound and the creator can define the time period. After the defined time period, the survey expires and users will not be able to access the form. Though, data submitted by any user before expiring is always available.

Key Features

Plan of record

As a creator, I should be able to

1. Add Meta data for detail of Observation Creation (Like observation name, description, keywords, etc.)

   1. All the meta data fields for observation without rubric are listed in this story - SB-25782

   2. All the meta data fields for observation with rubric are listed in this story - SB-26401
      Detailed comparison the metadata fields provided below in Table

2. The following meta data configuration at the observation level will affect the question behaviour also:

   1. In observation without rubric - ‘Enable Scoring’ will add on the field of score in the questions which wouldn’t be present otherwise

   2. In observation with rubric - 'Add evidence collection methods' will add on the field of ECM to tag to the question which wouldn’t be present otherwise

3. Create section and provide details

4. Include various (text, date, MCQs both single and multi select and slider) question types

5. Configuring ability to add remarks or evidences for any question type.

6. Add Hints

7. Add sub questions

8. Allow multiple instances for questions in a section

9. Change question sequence

10. Save the draft

11. Submit for review
    As a reviewer, I should be able to

12. Review and publish the submitted content
    As an admin, I should be able to

13. Create a project to “seek content not for a target collection”

14. Seek survey, observation and observation with rubric as content type in the sourcing project.

15. Manage user roles, access, and track progress of sourcing activities.

Table comparing meta data for Survey, Observation, and Observation with Rubric

Future considerations

1. Creator will see consumption data points of the solutions

2. Copying and editing an existing observation 

3. Review flow for the observation/survey creation

Product Pipeline (Overall Summary)

Consider making an overall summary and status table for Survey, Obs, Obs with Rubric using this format.

Sourcing project creation flow
(The sourcing project to seek content not for a target collection is a critical flow for observation and survey creation)

Detailed flow of creating an Sourcing project

• Sourcing org admin logs into the system and starts creating a sourcing project

• The system provides option to select the project scope
  -Seek content for a certain framework 

• As the user selects the option “Seek content for a certain framework”, the following flow is triggered.

• The use has following options to select as the project scope.

• There will be a default framework selected as per system configuration - say “K-12 Curriculum” by default. 

• Selecting framework type is mandatory. Only one framework type can be selected per project.

• All the content related to the classroom will come under ’K-12 curriculum’ and all the content related to school and Administration will come under ‘Continuous professional development’. For example, a survey related to classroom, teaching and subject will come under the K-12 curriculum and the metadata fields will be board, medium, subjects.

• For any framework, an observation or survey, observation with rubric can be created.However, a certain few things can be considered
  - For students, observation might not be relevant as observations are done against entities and this entity structure is not applicable.
  -There is no direct impact of the selected framework on observation without rubric and survey
  -On the basis of the selected framework, the values for tree structure of domain and criteria would be auto-populated. As explained below, for observation with rubric the level concept category from framework will be taken in use for domain criteria structure.

• When a framework type is chosen, the framework available in the tenant of the selected framework type is taken as the framework to be used for the project. The assumption is that there is a maximum one framework of a given type that is configured in the tenant. In case there is no framework of the selected type configured at tenant level, the framework of that type configured at system level is taken. The assumption is that a maximum one framework of a given type is configured at the system level. In case there are no framework of the selected found either at tenant or system level, system shows an error message “Incorrect framework configuration. Please check with the system administrator”.

• Users can select one or more values from the framework category. None of the attributes are mandatory. Users also have the option to select all.

• Observations, observation with rubric and surveys will be the new primary categories in content type dropdown while creating the sourcing project.

• Users have the option to select one or more content types - this should show all primary categories of all the non-collection object types (content, question set).

• Apart from the project scope, the user enters other project details that are the same as the existing project creation flow, except selecting target collections.

• Users can save the project as draft.

• Users can reopen a saved project in draft state. The saved values are displayed. Users can modify any values and save.

• Users can publish the project once all required details are provided.

• The bulk upload option on the contribution page, would not be applicable for observation and survey since bulk upload is only for uploaded resources (like pdfs, MP4 etc.) and not for resources that are to be created on the portal.

Contribution flow for Survey and Observation

Key flows for Creation for Observation or Survey

New Question categories that would be added

1. Slider - Slider easily captures range based response, it shows a range specified by creator, the user can drag the point to the option he wants to choose. (Please refer the ‘Slider’)

2. Text - These would be subjective response type questions for user to enter the responses in free text. Number input, which prompts only number response by user, will be a validation in text type question

3. Date - Date type questions for user to input date in set format. Creator can also set configuration for auto capturing the date of the day.

4. MCQ - The MCQs will be for providing multiple choice based question

   1. Single select response questions (configured by default) let the users select just one option in an MCQ.

   2. Multi-select response question (creator can set this configuration) lets user select multiple response in an MCQ.
      

Link to stories

Contribution flow for observation with rubric

Observation with rubric contribution flow

Framework and Domain criteria values for Observation with rubric

An observation with Rubric content would be created for a particular framework which would also be tagged in the ‘sourcing project not for a target collection’. The observation would have a Domain-criteria structure that would be defined through the values in these frameworks for seeking content in the sourcing project.

The frameworks like CPD will have a particular flat list of concepts over different levels like L1 concepts, L2 concepts, etc. The values in these level concepts would have direct associations among each, like, multiple L2 concepts would be associated to a particular L1 concept, multiple L3 concept would be associated to a particular L2 concept.

The observation with rubric will directly derive the level concept values as the domain criteria values. The highest level concept will be taken as domain and the lowest level concept as criteria. For eg. If there are 3 level concept L1 concept, L2 concept and L3 concept, then, the L1 concept values will be taken as domain values in the Observation with rubric and the L3 concept values will be taken as criteria values.

Key flows for Creation

Review Flow