Introduction

This wiki explains the design approach for enhancing the Question and QuestionSet Objects to support creation and consumption of Observation with Rubrics.

Problem Statement

• Creator should be able to create an observation with rubrics against a framework which will have domain & criteria (criteria’s will be associated to domain).

• Creator should be able to tag observation and its questions with Evidence Collection Method (ECM).

• Creator should be able to decide on page numbers for questions under either ECM or Criteria

• Creator should be able to define Scoring, Max and Min Scoring level (at each criteria level).

• Gathering additional metadata at each framework category level should be driven be form configuration.

• Consumer should be able to consume the Observation in two ways

  • by browsing taxonomy structure i.e: domain & criteria

  • by browsing list of Evidence Collection Method (ECM)
    

Approach 1: Creating hierarchy folder structure as per the taxonomy structure

• Domain, Criteria & Section will be created as nested folders under observation root node.

• The hierarchy structure without ECM will be as below:
  

  {
    "Observation": {
      "name": "observation",
      "description": "test observation",
      "primaryCtegory": "Observation"
      "identifier": "do_1_observation"
      "children": [ 
        {
          "identifier": "do_1_domain1",
          "name": "<domain name entered by user>",
          "domain": "domain1",
          "parent": "do_1_observation",
          "children": [
            {
              "identifier": "do_1_criteria1",
              "name": "<criteria name entered by user>",
              "domain": "domain1",
              "criteria": "criteria1",
              "parent": "do_1_domain1",
              "children": [
                {
                  "identifier": "do_1_section1",
                  "name": "section1",
                  "domain": "domain1",
                  "criteria": "criteria1",
                  "parent": "do_1_criteria1",
                  "children": [
                    {
                      "identifier": "do_1_question1",
                      "name": "question1",
                      "parent": "do_1_section1",
                      "domain": "domain1",
                      "criteria": "criteria1"
                    },
                    {
                      "identifier": "do_1_question2",
                      "name": "question2",
                      "parent": "do_1_section1",
                      "domain": "domain1",
                      "criteria": "criteria1"
                    } 
                  ]
                }
              ]
            }
          ]
        }
      ],
      "view": [
        {
          "type": "criteria",
          "domain": "domain1",
          "criteria": "criteria1",
          "section": [
            "do_1_section1"
          ]
        }
      ]
    }
  }

Approach 2: FrameworkTemplate and View

Observation Type - 'Self' (without ECM)

{
  "Observation": {
    "name": "observation",
    "type": "self",
    "frameworkTemplate": {
      "domain": [
        {
          "label": "<name_given_by_creator>",
          "name": "domain1",
          "description": ""
        }
      ],
      "criteria": [
        {
          "name": "<name_given_by_creator>",
          "domain": "domain1",
          "description": "",
          "criteria": "criteria1"
        },
        {
          "name": "<name_given_by_creator>",
          "domain": "domain1",
          "description": "",
          "criteria": "criteria2"
        }
      ]
    },
    "sections": [
      {
        "id": "do_1_section",
        "children": [
          "do_1_question",
          "do_2_question"
        ]
      },
      {
        "id": "do_2_section",
        "children": [
          "do_2_question"
        ]
      }
    ],
    "view": [
      {
        "type": "criteria",
        "domain": "domain1",
        "criteria": "criteria1",
        "section": [
          "do_1_section",
          "do_2_section"
        ]
      }
    ]
    "children": [ // existing structure with new question properties
      {
        "id": "do_1_question",
        "domain": "domain1",
        "criteria": "criteria1"
      },
      {
        "id": "do_2_question",
        "domain": "domain1",
        "criteria": "criteria2"
      }
    ]
  }
}

Observation Type - 'External' (with ECM)

{
  "Observation": {
    "name": "observation",
    "type": "external",
    "ecm": ["ecm1", "ecm2"],
    "frameworkTemplate": {
      "domain": [
        {
          "name": "<name_given_by_creator>",
          "domain": "domain1",
          "description": ""
        }
      ],
      "criteria": [
        {
          "name": "<name_given_by_creator>",
          "domain": "domain1",
          "description": "",
          "criteria": "criteria1"
        },
        {
          "name": "<name_given_by_creator>",
          "domain": "domain1",
          "description": "",
          "criteria": "criteria2"
        }
      ]
    },
    "sections": [
      {
        "id": "do_1_section",
        "children": [
          "do_1_question",
          "do_2_question"
        ]
      },
      {
        "id": "do_2_section",
        "children": [
          "do_2_question"
        ]
      }
    ],
    "view": [
      {
        "ecm": "ecm1",
        "type": "ecm",
        "section": [
          "do_1_section",
          "do_2_section"
        ]
      }
    ],
    "children": [ // existing structure with new question properties
      {
        "id": "do_1_question",
        "domain": "domain1",
        "criteria": "criteria1",
        "ecm": "ecm1"
      },
      {
        "id": "do_2_question",
        "domain": "domain1",
        "criteria": "criteria2",
        "ecm": "ecm1"
      }
    ]
  }
}

Pros

1. Creating the views (ECM & non-ECM) for both editor and player becomes easier with the flattened structure.

2. It addresses all the possible scenarios of sequencing and presentation of question on the consumption side.

3. It allows to store the snapshot of the taxonomy with additional metadata.

4. This design allows to add any framework categories under frameworkTemplate based on the need and views based on any metadata (ex: ECM)

Cons

-

Open Questions:

1. QUESTION - The relation between the criteria and sections ?

Behaviour:

• Creator will see the framework hierarchical structure of domain & criteria for the first time, when creation of observation begins.

• Once the observation is created, the taxonomy structure will be copied to the observation and same will be shown to the user from next loading of editor.

• Label and some additional information (description, keywords, etc) can be updated by creator for the copied taxonomy structure.