Sourcing - Telemetry Standards

Owned by Kameswararao Bh
Last updated: Sept 07, 2021
8 min read

Overview

This document captures a common set of standards to be followed for generating telemetry events in sourcing solution. This is to ensure that:

  1. The telemetry event data is easy to understand for developers - both who develop code to generate them as well as who develop code to utilize them in reports

  2. Testing team can easily understand the expected events and test accordingly

  3. Any new telemetry data that needs to be generated does not need a detailed requirements, design and review efforts

Telemetry Events

All the telemetry events described in the below sections are mandatory for all pages and user actions. All the data fields specified are mandatory, unless otherwise specified explicitly here.

All the values for the fields have to be in lowercase, unless otherwise specified.

START, END

These events have to be captured for any multi-step activity performed by a user that has a clear start and an end.  

The purpose of these events is to capture the number of sessions and session times for each activity that is performed as a session.

There can be sessions that have a START event but no END event, in the following scenarios:

  • When user clicked browser back button

  • When user clicked browser close

  • Session timeout

In sourcing solution, the following are key user activities that need to have these events generated for all users.

Session

Successful Login should generate START and successful logout should generate END events.

EDATA

  • type (same for START and END events): session

  • mode (only for END event): Label  of the button clicked when closing the session, converted lowercase. For example “logout”

  • duration (only for END event): Start to end time

  • pageid (same for START and END events): The page id of the page from which the session is started. Note: Please refer to the pages section below that has a list of all pages

Content->CDATA

None

OBJECT

None

Edit Session 

START event when the user opens the content edit page for creation and END event when user saves content or submits content.

EDATA

  • type (same for START and END events): editor

  • mode (only for END event): Label  of the button clicked when closing the session, converted lowercase. For example “save” or “submit”.

  • duration (only for END event): Start to end time

  • pageid (same for START and END events): The page id of the page from which the session is started. Note: Please refer to the pages section below that has a list of all pages

Content->CDATA

  • sourcing_organization of the project (in case edit happens through a project)

  • project (in case edit happens through a project)

  • linked_collection (in case the content is linked to a collection)

OBJECT

Details of the content being edited.

  • id: content id, type: object type

Play Session 

START event when the user starts playing a content and END when the content play completes.

EDATA

  • type (same for START and END events): player

  • mode (only for END event): Label  of the button clicked when closing the session, converted lowercase. For example “play”, “stop”, “pause”.

  • duration (only for END event): Start to end time

  • pageid (same for START and END events): The page id of the page from which the session is started. Note: Please refer to the pages section below that has a list of all pages

Content->CDATA

  • sourcing_organization (in case edit happens through a project)

  • project (in case edit happens through a project)

  • linked_collection (if the content is linked to a collection)

OBJECT

Details of the content being edited.

  • id: content id, type: object type

Project Create Session 

When user opens create project page (START), when user saves project or publishes project (END)

EDATA

  • type (same for START and END events): workflow

  • mode (only for END event): Label  of the button clicked when closing the session, converted lowercase. For example “save” or “submit”.

  • duration (only for END event): Start to end time

  • pageid (same for START and END events): The page id of the page from which the session is started. Note: Please refer to the pages section below that has a list of all pages

Content->CDATA

  • sourcing_organization

OBJECT

Only for the END event. Details of the project created.

  • id: project id, type: project

IMPRESSION

This event has to be captured when any page is visited by a user.

The main purpose of this event is to identify how many times a given page is visited and what is the loading time of the page. Page visits data is useful for funnel analysis and loading time data is useful to identify system responsiveness/usability.

This event has to be triggered when a page completely loads. Please refer to the pages section below that has a list of all pages.

EDATA

  • type: Page type  of the page being visited

  • pageid: Page id of the page being visited

  • uri:  Relative URL of the page being visited

  • duration: Time taken to render the page in seconds

  • visits: This is only in case a page has a table or a list of items. This will be an array of VISIT objects that will capture objid and objtype of the objects being listed. Object types can be:

    • One of the object types of like collection, content, question set, question etc.

    • project (in case items listed are projects)

    • nomination (in case items listed are nominations)

    • user (in case items listed are users)

Context->CDATA

  • sourcing_organization (in case the page is within a project or the page is on sourcing portal)

  • project (in case the page is within a project)

  • nomination (in case the page is within a nomination)

  • linked_collection (in case the page is within a linked collection)

OBJECT

  • id: content id, type: content (if the page is related to a content)

INTERACT

This event has to be captured when a user performs a UI interaction on a page.

The main purpose of this event is to track a set of user activities on the system. This is primarily useful to identify what features are being used most frequently and what features are not being used much. These events can also be used to trace a set of actions by a user in a given time.

Event has to be generated for the following events:

Click of an action button, close icon on a dialog or link

This action needs to be triggered when an action button or “close icon” on a dialog or a link (url) is clicked. 

Example: “Select Content Type”, “Submit”, “Nominate” or “Upload” button is clicked. “Terms and Policies” is clicked.

EDATA

  • type: Click

  • subtype:
    launch - If the button click that launches a new page or dialog
    cancel - If the button click is to cancel or close the initiated action
    submit - If the button click is to apply, save or submit the initiated action

  • id: Button label changed to lowercase and in between spaces replaced by “_”. For example, the id for the button “Select Content Type” is “select_content_type”. Id for “Submit” is “submit”.
    In case the button is an icon, use the tooltip of the button as button label.

  • pageid:
    If the clicked button or link is in a page - page id of the page
    If the clicked button or link is inside a dialog - page id of the page from which the dialog is launched.

  • extra:

This is required whenever the clicked button or link is inside a dialog.
values: 

  1. Dialog details

    1. Key: dialog_id

    2. Value: Dialog title changed to lowercase and in between spaces replaced by “_”. For example if “Apply” button in “Filter Textbooks” dialog is clicked, the extra value is “filter_textbooks”

  2. In case the event subtype is “submit” and the dialog contains any dropdown or radio buttons, store the selected values in dropdown or radio button. Each selection is a separate value in the array

    1. Key: selection_field_id (id of the dropdown or radio button)

    2. Value: Value selected in that field

Content->CDATA

  • sourcing_organization (in case the page is within a project or the page is on sourcing portal)

  • project (in case the page is within a project)

  • nomination (in case the page is within a nomination)

  • linked_collection (in case the page is within a linked collection)

OBJECT

  • id: content id, type: content (if the page is related to a content)

Selection of a dropdown or radio button

This action needs to be triggered whenever a user selects a dropdown or a radio button. 

Note this is only on explicit action by the user, not when the system sets a value by default. 

For a dropdown, this is triggered only once when a user clicks on the dropdown field. It is not generated each time a value within dropdown is selected.

Example: A subject is selected from “Subject” dropdown in Filter Textbooks dialog.

EDATA

  • type: select

  • subtype:
    multi_select - If the dropdown is multiselect
    single_select - If the dropdown is single select
    radio_button - If selected value is from a radio button

  • id: Label the dropdown or radio button, changed to lowercase and in between spaces replaced by “_”. For example, the id for the dropdown “Subject” is “subject”.
    If the dropdown is part of a table and doesn’t have a separate label, take the table column in which the dropdown is present as the dropdown label.

  • pageid:
    If the dropdown or radio button is in a page - page id of the page
    If the dropdown or radio button is inside a dialog - page id of the page from which the dialog is launched.

  • extra:

This is required whenever the dropdown or radio button is inside a dialog.
values: Dialog title changed to lowercase and in between spaces replaced by “_”. For example if value from “Subject” dropdown in “Filter Textbooks” dialog is selected, the extra value  is “filter_textbooks”

Content->CDATA

  • sourcing_organization (in case the page is within a project or the page is on sourcing portal)

  • project (in case the page is within a project)

  • nomination (in case the page is within a nomination)

  • linked_collection (in case the page is within a linked collection)

OBJECT

  • id: content id, type: content (if the page is related to a content)

ERROR

This event has to be captured whenever any error occurs in the system.

The main purpose of this is to be able to trace all errors generated in the system by any user activity and we have proper information to debug and analyze the root cause of the issue.

Any error that is shown to the end user or an error that internally occurs when an API is called has to be captured using this event.

The error could be due to any of the reasons such as:

  1. System error

  2. Incorrect arguments passed to API 

  3. Business validations failed

  4. User has not provided required/correct data

EDATA

  • err: Error code generated by system

  • errtype: TBD (need to check the different error types defined in the system)

  • stacktrace:  Error stacktrace

  • pageid: Page id where the error has occurred

  • object: Object on which the error occurred.

Content->CDATA

  • sourcing_organization (in case the page is within a project or the page is on sourcing portal)

  • project (in case the page is within a project)

  • nomination (in case the page is within a nomination)

  • linked_collection (in case the page is within a linked collection)

OBJECT

  • id: content id, type: content (if the page is related to a content)

Page Definition

Following spreadsheet provides the page definitions for all pages in sourcing and contribution portals:

https://docs.google.com/spreadsheets/d/1PGdyC86PUSD6xZQ4WFMYUdTec7LSaHD3WbxWLPmNbWE/edit?usp=sharing

Sample Telemetry Events

Following spreadsheet provides a sample of the telemetry events generated (impression and interact events)

https://docs.google.com/spreadsheets/d/1q14NRwumF69-KorZUnXl5N1CIzaorh6XmM1r5nT__AM/edit?usp=sharing

Telemetry Specification

Following page has the details of Sunbird Telemetry Specifications:

http://docs.sunbird.org/latest/developer-docs/telemetry/specification/