Page Comparison

Introduction

...

SDK

Following are SDK's built into the container.

Global

// Plugin to register with the container on initialization.
register(String pluginId, Object config)

config = {
	apiToken: String,
	apiBaseURL: String,
	apiTokenRefreshFn: Function
}

Telemetry SDK

// API to register a plugin and it's corresponding sync url to dispatch telemetry to.
register(String pluginId, String syncURL)

// API to log/store a telemetry event
send(String event, String pluginId) : Promise;

// API to log/store an array of telemetry events
send(Array[String] events, String pluginId) : Promise;

// API to receive a batch of telemetry events in zip stream
sendBatch(String batchEvent, String pluginId) : Promise;

// API to receive a batch of telemetry events in zip stream
sendBatch(byte[] events, String pluginId) : Promise;

// API to force sync
sync(String pluginId) : Promise;

// API to get the status of events/batches for a given plugin id
// Response - {events: 3456, batches: 20}
status(String pluginId) : Promise;

File SDK

Import SDK

Export SDK

Settings SDK

...

Configuration

Database Schema

Folder Structure

Events

Download Manager

Below explains the process of downloading content when a user request for it, the steps are 

1. Download Manager is initialised with config and will be ready to take the request
2. When a user request for the download Sunbird Ed Plugin forward request to download Queue
3. If download manager is available it will download the content and once it is download
4.  It will raise an event and plugin will listen to that event
5. With event data it will extract the ECAR and index the content in content database

Image Added

At low level the download manager works as

1. When it starts downloading it creates <filename>.sud and <filename>.<n>.partial(s) files, <filename>.sud file contains the meta data about file.
2. If internet is disconnected or system got shut down  then it will pause the download
3. When the internet is connected again or system restarted it will read the file .sud  and continue the download from where it left
4. When the file is download completed then it combines all the partials and creates final ECAR file.

For more details you can refer su-downloader3

Image Added

Telemetry Sync:

Here we have two approaches to sync telemetry efficiently below will explain the them with pros and cons and concludes on one.

Solution 1:

This solution is having two steps to sync the telemetry in first step there will be a process which will execute every <x> minutes(configurable) in this method below steps are performed

1. Read the events form telemetry database
2. Events will be divided into equal numbers as batches of size <x> events(configurable)
3. The events will be compressed(.gzip) and stored in telemetry folder
4. Same events will be deleted from Telemetry database.

Image Added

Below diagram shows the second step process which is again a process which will execute in every <x> minutes(configurable)  below are the steps executed

1. Check the internet connectivity if available read all the event(s) files from telemetry folder
2. Sync the events to server and after each file is synced move them to telemetryArchive folder
3. The files in telemetry archive folder will be deleted in <x> days(configurable)

Image Added

Pros:

1. No dependency on database availability while syncing telemetry to server

2. There is clear separation in creation of compressed telemetry and syncing compressed telemetry to server

3. If the database is crashed or uninstalled by user still we  have telemetry in file system since we are reading and storing them in file system

Cons:

1. User can delete the telemetry files from file system

2. More number of I/O operation on file system

3. Possibility of telemetry sync failure is high

Solution 2:

This solution will have only one process which will be executed in every <x> minutes(configurable) . Below are the steps

1. Check for network connection if available ready the events from telemetry database
2. Events will be divided into equal numbers as batches of size <x> events(configurable)
3. Sync

...

1. the events to server and after each file is synced move them to telemetryArchive folder
2. The files in telemetry archive folder will be deleted in <x> days(configurable)
3. Same events will be deleted from Telemetry database
4. Above steps from step3 to step5 will be executed in series for each batch of events

Image Added

Pros:

1. Less chance of telemetry being lost due to user actions

2. Telemetry  sync failure is low

3. Less number of I/O operation which result in more performance

4. Easy to develop and test

5. There is single source of data which is not synced to server which will help in export process.

Cons:

1. Dependency on database availability

2.  Possibility of duplicate telemetry sync if the process execution time is less

Conclusion: We will go with the solution 2 since there is less chance of losing telemetry in the process of syncing it to server and possibility of  the user being able to easily access the file system compare to database more.

Telemetry Events

Logging

Log Sync

...