/
Data migration across tenants

Data migration across tenants

Overview

As part of the ticket - SB-9803, we are required to merge user/content of 2 tenants into 1. Earlier the state had created 2 tenants, both have users as well as content. There may be user-org associations in both tenants as well. As part of this story we need to migrate the data from one tenant to other, and then retire/close/delete the empty tenant.


Data

FieldShift FromShift To
Root Org Id0124140824272322561401246375399411712074
NameETB Pilot RajasthanRajasthan
Total Content61444271
Sub-organisations00
Course Batches70
Users179237
Badge Extension Class00
channelpilotrajasthanrj

Approaches

Approach 1

We can provide a set of APIs using which one can migrate user, sub-org or content across the tenants (Root organization).


Pros

  • User will not require the server access, can trigger API
  • User can use different API's to build custom script, which is more flexible
  • We can apply user-access control if it is considered as an API end-point.
  • Does not require to bring the server down.

Cons

  • It is a one-time administrative task, so providing API is little risky for such cases
  • This API may not be useful for all users, and will bulk up the user-code unnecessarily.
  • difficult to implement and more over-head considering it is one-time job.


Solution

API to migrate organisation

POST /v1/tenant/migrate - will migrate the sub-organizations, to point to new root-organization.

{
 "sourceRootOrgId" : "0101201012",
 "targeRootOrgId" : "0101201013",
"entities" : ["user", "org", "content"]
"userIds" : ["id1", "id2" ] }


UserIds → if mentioned will be applicable to only those users, else to all users will be migrated.

entities → if mentioned only those will be migrated, else all entities will gt migrated.

Responses

200 OK - both valid root-org are provided and data can be migrated

400 Bad Request - Invalid sourceRootOrgId or targetRootOrgId

  • any of the org does not exist
  • any of the org is not active
  • any of the org is not a root-organization


Above API's will individually migrate the individual data


Approach 2

Provide a stand-alone script/migration job, which once invoked will migrate the data internally, with minimal user choices.

Pros

  • Simple approach, easy to implement
  • We do not have to expose such changes as API, as this is admin type of activity

Cons

  • Very specific solution - not much configurable from user perspective
  • User requires access to environment, as process will be run from background using environment variables.
  • This requires that server. instance must be down, while we are doing this migrations.

Data to be migrated (Identified as per database storage)

  • Child organization - Root org - may have other child organizations, we need to migrate those child organizations
  • User's root-org need to be updated to new root-org
  • Content data - channel name - need to be migrated according to new channel name.
  • User-org association, for old root-org need to be deleted, and new user-org association need to be added
  • geo_location data - which refers the root-org id also need to be migrated
  • badge_class_extension - also refers to the root-org
  • There are several storage structures which point to organisationid which need to be migrated if they are pointing to root-org which is being migrated.
    • course_management (organisationid, organisationname)
    • user_org(organisationid)
    • page_management (organisationid)
    • bulk_upload_process(organisationid)

Solution (Based on data analysis)

  • Get all user ids for Rajasthan Pilot Organisation
  • Update the rootOrgId, channel & slug for user ids fetched in above step.
  • Get all user-org entries where associated org id is Rajasthan Pilot.
  • Update the rootOrgId to point to Rajasthan root org.
  • Sync the users updated to ElasticSearch index
  • Get all course batches where createdFor is pointing to Rajasthan Pilot
  • Update the course batches and set createdFor to Rajasthan
  • Sync the course batch details to ElasticSearch index.
  • Mark the Rajasthan Pilot Org → inactive status.

Assumptions

  • Telemetry data will be pointing to according to old values for all the updates made so far, and will not be migrated.

Related content

User Initiated Account Merge
User Initiated Account Merge
More like this
ETL job to add state, district of non custodian user into user DB
ETL job to add state, district of non custodian user into user DB
More like this
Saving Course batch details inside user
Saving Course batch details inside user
More like this