Consumption Metrics

Owned by Aniket Sakinala
Jul 03, 2019

This document details the setup processes required to get consumption metrics report generation scripts installed and running on a Virtual Machine. This document is split in two stage: a one time setup and maintenance tasks.

One Time Setup

  1. Git Fork the branch https://github.com/DIKSHA-NCTE/telemetryreports/tree/portal_dashboards to https://github.com/project-sunbird/sunbird-analytics/tree/release-2.2.0
  2. Spark 2.3.1 with JARs: 
    1. azure-storage-2.0.0.jar
    2. hadoop-azure-2.7.3.jar
  3. Credentials in bashrc for:
    1. Azure blob: ntpproduction
    2. Azure blob: ntpproductionall
    3. Google API's (Implementation team ID)(is a JSON file)
  4. Python 3.4.3 virtual environment with package listed in requirements.txt 
    1. pip install --user virtualenv
    2. virtualenv ENV
    3. source ENV/bin/activate
    4. pip install -r scripts/requirements.txt
  5. nodeJS 
    1. cd dags/tasks/node_csv_to_json
    2. npm install
  6. local data folder location

Instructions for Daily Metrics


  1. Git Pull branch
  2. python scripts/daily_metrics_refactored.py $local_data_folder_location

Instructions for Creation Metrics


  1. Git Pull branch
  2. python scripts/dialcode_exception_etb_reports.py.py $local_data_folder_location