Business Requirement
Sunbird generates a number of datasets for reporting purposes. Some of these are downloaded by users via the UI (the Course progress exhaust, for example), whereas others are used by the Sunbird portal to generate reports and charts, which can then be viewed by users.
There is a need stated for making all such datasets that are generated by SB to be made accessible via APIs. This will permit for these datasets to be pulled via APIs by the adopters, for usage in whatever additional manner they see fit - and enable them to create custom reports/ visualisations of their own.
Dataset generation within SB currently happens for:
...
Custom reports such as the course progress exhaust or the user PII data - these are generated by data products that run on a daily/ on-demand basis
...
Portal based reports and charts - these are datasets generated from Druid, and power the charts shown on the ‘admin dashboard’ of the portal. These are generated based on the frequency of the reports as configured.
...
Other custom data files such as Consent files and master data files (geo data, for example)
Dataset download considerations:
Sunbird should enable an adopter to provision for data security - and allow access to datasets based on the nature of the data they contain. Consequently, a dataset that is not considered sensitive may be made available for public access, whereas sensitive data may require to be protected.
In order to enable different types of access, SB will support configurations that can make the datasets ‘PUBLIC’ or ‘PRIVATE’ - as the adopter may choose - this will determine whether a dataset can be accessed publicly using the API without any additional authorisation, or whether it needs additional approval from admins for access.
Current Implementation
Currently, Sunbird Observation supports Reports Service API to list and access all published reports. However, only metadata of a report is accessible through this API. There are associated data files that contain the detailed data of a given report. These are currently not accessible through API.
There is an internal endpoint at the portal backend layer which downloads the respective datasets from the azure reports container. This endpoint is accessible only by the logged in person(session based) having roles (REPORT_ADMIN , REPORT_VIEWER, ORG_ADMIN).
Moreover, there is a slug based validation so that they do not access other tenant’s data.
For parameterized reports, respective endpoints are injected based on the logged in user’s context.
As part of current implementation datasets cannot be accessed by non logged in person or any third party.
Supported parameters
$slug
$channel
$state
$board
Below Diagram explains the High Level Working flow on how the reports are rendered on the Portal Admin Dashboard.
...
Current Report Config Structure
...
title | Current Report Config |
---|
...
Business Requirement
Sunbird provides the capability to generate various datasets for reporting purposes. Some of these can be downloaded by users via the UI (the Course progress exhaust, for example), whereas others are used by the Sunbird portal to generate reports and charts, which can then be viewed by users.
There is a requirement for making all such datasets that are generated by SB to be made accessible via APIs. This will permit these datasets to be pulled via APIs, for usage in whatever additional manner they see fit - and enable them to create custom reports/ visualizations of their own.
Currently, following are the list of Datasets supported within SB:
Druid based Datasets/Reports (created via HE/APIs)
Custom datasets created as data products and configured manually
Ex: VDN reports, WFS, Collection Summary dataset
On-Demand Datasets
Expand | ||
---|---|---|
| ||
|
Dataset download considerations:
Sunbird should provide an ability to provision for data security and allow access to datasets based on the nature of the data they contain. Consequently, a dataset that is not considered sensitive may be made available for public access, whereas any sensitive data may require to be protected.
In order to enable different types of access, SB will support configurations that can make the datasets ‘PUBLIC’ or ‘PROTECTED’. The type of access will determine whether a dataset can be accessed publicly using the API without any additional authorization, or whether it needs to validate the user before providing access.
...
Current Implementation
Currently, Sunbird Obsrv supports Reports Service API to list and access all published reports. However, only metadata of a report is accessible through the Report Service API. The data files and associated data are not currently accessible through the Report Service API.
The Sunbird portal allows access to the configured datasets using the Sunbird portal backend APIs. However, the APIs make use of the authorization through the Sunbird portal’s session logins with the access controls that are controlled by different roles in the Sunbird portal such as REPORT_ADMIN, REPORT_VIEWER and ORG_ADMIN.
Data isolation is supported by performing a slug or org based validation so that the access to a specific tenant’s data is controlled using the user’s login context.
The Sunbird user’s login context is used for parameterized reports where the parameters are injected from the user’s login context. The list of supported parameters are:
$slug
$channel
$state
$board
Below diagram explains the high level working flow on how the reports are rendered on the Portal Admin Dashboard.
...
Roles which are supported in Admin Dashboard workflow
Report_Admin
Report_Viewer
Super_Admin - INSTANCE_ADMIN
Pages | REPORT_ADMIN <SUPER_ADMIN> | State REPORT_ADMIN | REPORT_VIEWER |
---|---|---|---|
List Page |
|
|
|
Report Detail Page |
|
|
|
...
Current Report Config
Expand | ||
---|---|---|
| ||
|
New Column Additions
visibility
type - varchar(10)
enum - [public, private, protected, parent]
Details - Link
accessPath
type - jsonb
Details - Link
Current Api Structure
...
title | GET- /report/get/:reportId |
---|
This API is associated with viewing and reading out the specific report on the Sunbird Platform. It returns only the metadata information for the report.
Response
...
language | json |
---|
...
|
New Column Additions
visibility
type - varchar(10)
enum - [public, private, protected]
Details - Link
accessPath
type - jsonb
Details - Link
visibilityFlags
type- jsonb
Description - Whether to show/hide the report on specific pages.
...
Access Control Spec
Controls who can access a report based on certain rules.
This can be achieved using two attributes visibility and accessPath.
Visibility
It can be defined both at the report level or it's children (i.e table, chart, map etc ) within a report.
Visibility | Access |
---|---|
public (default) | Accessible by all users. Anyone can discover and consume these Reports. |
protected | Accessible only to a limited set of users based on a criteria defined by the access path attribute. Default can be user with REPORT_VIEWER role belonging to the same channel or tenant. |
private | Similar to “protected” - accessible only by the creator of the report. |
AccessPath
This attribute is applicable for Reports with “protected” or “private” visibility only. This attribute can be used to restrict the access based on or more of the following criteria: organisation, role, group, user id, and location.
AccessPath interface is as follows :-
**Note - All keys are optional. Also very fine grained accessPath may not be required in this case. Replicating the actual schema defined for ActionSets as is and to be re-used for datasets as well.
Code Block | ||
---|---|---|
| ||
interface IAccessPath {
organisation: Array<string> | string;
role: Array<string> | string;
channel: Array<string> | string;
group: Array<string> | string;
userType: Array<string> | string;
framework: Array<string> | string;
isSuperAdmin: Array<string> | string;
board: Array<string> | string;
userId: Array<string> | string;
userLocation: {
state: Array<string> | string;
district: Array<string> | string;
block: Array<string> | string;
};
...anyOtherAttribute: any
} |
...
Current API Structure
Expand | |||||
---|---|---|---|---|---|
| |||||
This API is associated with viewing and reading out the specific report on the Sunbird Platform. It returns only the metadata information for the report. Response
|
Datasource Schema :-
id -: job_id
path -: endpoint to the portal-backend layer which download the dataset file - the path can be both parameterized and non parameterized . Portal backend populates the parameters using logged in user context details and downloads the respective file.
Proposed Solution
API to get Metadata + Data Files
There is a need to create API in report service that will provide access to the meta data as well as the report data files that are used to generate the reports in the 'Admin dashboards' page on the Sunbird portal with certain access controls.
Proposed API Structure to get the metadata + datasets.
METHOD - GET
URL: /report/datasets/get/:reportId
...
title | API to get meta data + datasets |
---|
...
language | json |
---|
...
|
...
|
...
|
...
|
...
|
...
|
...
|
...
|
...
|
...
|
...
|
...
|
...
|
...
|
...
|
Datasource Schema
Code Block | ||
---|---|---|
| ||
interface IDatasource { id: string path: |
...
string
} |
id: Job id for the dataset
path: Endpoint for downloading the dataset file(s) - the path can be both parameterized and non parameterized . Portal backend populates the parameters using logged in user context details and downloads the respective file.
...
Proposed Solution
API to get materialized data Files
There is a need to create API in report service that will provide access to the meta data as well as the report data files that are used to generate the reports in the 'Admin dashboards' page on the Sunbird portal with certain access controls.
Proposed API Structure to get the metadata + datasets.
METHOD - GET
URL: /report/datasets/get/:reportId?from=<>&to=<>&since=<>
Expand | ||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||||||||||
Proposed response structure - to get meta + datasets.
Explaination -
Unauthorized Access - Status Code 401
|
Access Control Spec
Controls who can access a report based on certain rules.
This can be achieved using two attributes visibility and accessPath.
Visibility
It can be defined both at the report level or it's children (i.e table, chart, map etc ) within a report.
...
Visibility
...
Access
...
public (default)
...
Accessible by all users. Anyone can discover and consume these Reports.
...
protected
...
Accessible only to a limited set of users based on a criteria.
Default can be users belonging to the same organisation or tenant etc.
...
private
...
Similar to “protected” - accessible only to a limited set of users defined by the “access path” attribute.
...
parent
...
Accessible only from within the parent object, i.e access to the users is based on the visibility of the parent object.
Can be defined at chart, table or map defined within a parent report.
AccessPath
...
Code Block | ||
---|---|---|
| ||
interface IAccessPath {
organisation: Array<string> | string;
role: Array<string> | string;
tenant: Array<string> | string;
channel: Array<string> | string;
group: Array<string> | string;
userType: Array<string> | string;
framework: Array<string> | string;
isSuperAdmin: Array<string> | string;
board: Array<string> | string;
userId: Array<string> | string;
userLocation: {
state: Array<string> | string;
district: Array<string> | string;
block: Array<string> | string;
};
...anyOtherAttribute: any
} |
...
Explanation -
|
...
Search Report API
This API is associated with Searching Reports on the Sunbird Platform.
...
Expand | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||
Example of Request payload with visibility and accessPath
Success Response - Status Code 200
Error Response - Status code 500
|
...