Introduction
This wiki details out the architecture of the configurable reports for portal dashboard and also serves as an implementation guide to configure and set up a report.
Overview
Currently, the data team generates the data for the weekly reports in Diksha and the reports are being sent out as an html file. The configurable reports feature helps embedding these reports into the portal. This provides the capability to add a new report or modify an existing report configuration without any downtime to the Diksha systems. The power of the feature lies in the configuration of the reports that defines how the report will be visualised from the portal.
Reporting Architecture
The data team runs scheduled jobs to generate various daily and weekly reports such as total active devices over time, total number of sessions over time etc. The generated reports will be in csv format. If necessary, the csv reports can be converted to json formats using open source tools. The report data are then uploaded to a cloud storage and isolated by the channel to which the report belongs to. Channel is analogous to isolation of the report data for each State separately. Each report will have an unique identifier and also a report configuration. The path or location on cloud storage will be prefixed with the report identifier and also the channel identifier. The reports are rendered on the portal using the report configuration which also specifies the location of the data for the report.
Implementation Guide
Report Configuration Specification
The portal dashboard reports can only be accessed by the organisation admin users. The report configuration will be in JSON format and the schema for the report configuration is listed out below. A separate report configuration needs to be created for each of the organisation slug for which the report dashboards are being created. The multi-tenancy is achieved using the organisation slugh which will be part of the dataSource url specified in the configuration.
A sample configuration for a report with the required attributes is shown below.
{
id: "usage",
label: "Sunbird Usage Report",
title: "Sunbird Usage Report",
description: "The report provides a quick summary of the data analysed by the analytics team to track progess of Sunbird. This report will be used to consolidate insights using various metrics on which Sunbird is currently being mapped and will be shared on a weekly basis. The first section of the report will provide a snapshot of the overall health of the Sunbird App. This will be followed by individual org sections that provide org-wise status of Sunbird",
dataSource: "/usage/$slug/report.json",
charts: [
{
datasets: [{
dataExpr: "data.Number_of_downloads",
label: "# of downloads"
}],
labelsExpr: "data.Date",
chartType: "line"
},
{
datasets: [{
dataExpr: "data.Number_of_succesful_scans",
label: "# of successful scans"
}],
labelsExpr: "data.Date",
chartType: "bar"
}
],
table: {
"columnsExpr": "key",
"valuesExpr": "tableData"
},
downloadUrl: "<report_id>/$slug/report.csv"
}
- The report id should be unique
- For table columns and values are used if not available it will fallback to the columnsExpr, valuesExpr which will get the data from the key and tableData for header and values for the table respectively
- Multiple charts are configurable and follow the above config more details in setting up
Report data format specification:
The CSV file is used as downloadUrl by default name will be report.csv and it should contain the header and data which need to show as table and for graph as x, y axis and we need to use the CSV to JSON conversion open source tools to generate the json file which will have keys as headers and tableData which is array of arrays and graph data which should be collection array of objects as data below is the example
{
"keys": [
"Date",
"Number_of_downloads"
],
"data": {
"Date": [
"2018-12-01",
"2018-12-02",
"2018-12-03",
"2018-12-04",
"2018-12-05",
"2018-12-06",
"2018-12-07",
"2018-12-08",
"2018-12-09",
"2018-12-10",
"2018-12-11",
"2018-12-12",
"2018-12-13",
"2018-12-14",
"2018-12-15",
"2018-12-16",
"2018-12-17",
"2018-12-18"
],
"Number_of_downloads": [
"1850",
"2218",
"2312",
"1265",
"2113",
"8726",
"1763",
"2378",
"1764",
"8973",
"1224",
"1297",
"9872",
"1330",
"1799",
"2595",
"1815",
"1679"
]
},
"tableData": [
[
"2018-12-01",
"1850"
],
[
"2018-12-02",
"2218"
],
[
"2018-12-03",
"2312"
],
[
"2018-12-04",
"1265"
],
[
"2018-12-05",
"2113"
],
[
"2018-12-06",
"8726"
],
[
"2018-12-07",
"1763"
],
[
"2018-12-08",
"2378"
],
[
"2018-12-09",
"1764"
],
[
"2018-12-10",
"8973"
],
[
"2018-12-11",
"1224"
],
[
"2018-12-12",
"1297"
],
[
"2018-12-13",
"9872"
],
[
"2018-12-14",
"1330"
],
[
"2018-12-15",
"1799"
],
[
"2018-12-16",
"2595"
],
[
"2018-12-17",
"1815"
],
[
"2018-12-18",
"1679"
]
]
}
Naming convention for cloud storage upload directory
The default storage is azure blob store which is environment variable in portal(player service) sunbird_azure_report_container_name and it defaults to reports. As part of reports setup a private container should be created and name of container and environment variable name should be same and the container will have all the directories with organisation slug name and each directory contains the config.json and reports JSON and CSV file. The config will have relative path's to JSON and CSV files in directory as dataSource and downloadUrl.
Below is sample screenshot for storage format
