Data migration across tenants

Overview

As part of the ticket - SB-9803, we are required to merge user/content of 2 tenants into 1. Earlier the state had created 2 tenants, both have users as well as content. There may be user-org associations in both tenants as well. As part of this story we need to migrate the data from one tenant to other, and then retire/close/delete the empty tenant.


Data

FieldShift FromShift To
Root Org Id0124140824272322561401246375399411712074
NameETB Pilot RajasthanRajasthan
Total Content61444271
Sub-organisations00
Course Batches70
Users179237
Badge Extension Class00
channelpilotrajasthanrj

Approaches

Approach 1

We can provide a set of APIs using which one can migrate user, sub-org or content across the tenants (Root organization).


Pros

  • User will not require the server access, can trigger API
  • User can use different API's to build custom script, which is more flexible
  • We can apply user-access control if it is considered as an API end-point.
  • Does not require to bring the server down.

Cons

  • It is a one-time administrative task, so providing API is little risky for such cases
  • This API may not be useful for all users, and will bulk up the user-code unnecessarily.
  • difficult to implement and more over-head considering it is one-time job.


Solution

API to migrate organisation

POST /v1/tenant/migrate - will migrate the sub-organizations, to point to new root-organization.

{
 "sourceRootOrgId" : "0101201012",
 "targeRootOrgId" : "0101201013",
"entities" : ["user", "org", "content"]
"userIds" : ["id1", "id2" ] }


UserIds → if mentioned will be applicable to only those users, else to all users will be migrated.

entities → if mentioned only those will be migrated, else all entities will gt migrated.

Responses

200 OK - both valid root-org are provided and data can be migrated

400 Bad Request - Invalid sourceRootOrgId or targetRootOrgId

  • any of the org does not exist
  • any of the org is not active
  • any of the org is not a root-organization


Above API's will individually migrate the individual data


Approach 2

Provide a stand-alone script/migration job, which once invoked will migrate the data internally, with minimal user choices.

Pros

  • Simple approach, easy to implement
  • We do not have to expose such changes as API, as this is admin type of activity

Cons

  • Very specific solution - not much configurable from user perspective
  • User requires access to environment, as process will be run from background using environment variables.
  • This requires that server. instance must be down, while we are doing this migrations.

Data to be migrated (Identified as per database storage)

  • Child organization - Root org - may have other child organizations, we need to migrate those child organizations
  • User's root-org need to be updated to new root-org
  • Content data - channel name - need to be migrated according to new channel name.
  • User-org association, for old root-org need to be deleted, and new user-org association need to be added
  • geo_location data - which refers the root-org id also need to be migrated
  • badge_class_extension - also refers to the root-org
  • There are several storage structures which point to organisationid which need to be migrated if they are pointing to root-org which is being migrated.
    • course_management (organisationid, organisationname)
    • user_org(organisationid)
    • page_management (organisationid)
    • bulk_upload_process(organisationid)

Solution (Based on data analysis)

  • Get all user ids for Rajasthan Pilot Organisation
  • Update the rootOrgId, channel & slug for user ids fetched in above step.
  • Get all user-org entries where associated org id is Rajasthan Pilot.
  • Update the rootOrgId to point to Rajasthan root org.
  • Sync the users updated to ElasticSearch index
  • Get all course batches where createdFor is pointing to Rajasthan Pilot
  • Update the course batches and set createdFor to Rajasthan
  • Sync the course batch details to ElasticSearch index.
  • Mark the Rajasthan Pilot Org → inactive status.

Assumptions

  • Telemetry data will be pointing to according to old values for all the updates made so far, and will not be migrated.