Data migration across tenants
Overview
As part of the ticket - SB-9803, we are required to merge user/content of 2 tenants into 1. Earlier the state had created 2 tenants, both have users as well as content. There may be user-org associations in both tenants as well. As part of this story we need to migrate the data from one tenant to other, and then retire/close/delete the empty tenant.
Data
Field | Shift From | Shift To |
|---|---|---|
Root Org Id | 01241408242723225614 | 01246375399411712074 |
Name | ETB Pilot Rajasthan | Rajasthan |
Total Content | 6144 | 4271 |
Sub-organisations | 0 | 0 |
Course Batches | 7 | 0 |
Users | 179 | 237 |
Badge Extension Class | 0 | 0 |
channel | pilotrajasthan | rj |
Approaches
Approach 1
We can provide a set of APIs using which one can migrate user, sub-org or content across the tenants (Root organization).
Pros
User will not require the server access, can trigger API
User can use different API's to build custom script, which is more flexible
We can apply user-access control if it is considered as an API end-point.
Does not require to bring the server down.
Cons
It is a one-time administrative task, so providing API is little risky for such cases
This API may not be useful for all users, and will bulk up the user-code unnecessarily.
difficult to implement and more over-head considering it is one-time job.
Solution
API to migrate organisation
POST /v1/tenant/migrate - will migrate the sub-organizations, to point to new root-organization.
{
"sourceRootOrgId" : "0101201012",
"targeRootOrgId" : "0101201013", "entities" : ["user", "org", "content"] "userIds" : ["id1", "id2" ]
}
UserIds → if mentioned will be applicable to only those users, else to all users will be migrated.
entities → if mentioned only those will be migrated, else all entities will gt migrated.
Responses
200 OK - both valid root-org are provided and data can be migrated
400 Bad Request - Invalid sourceRootOrgId or targetRootOrgId
any of the org does not exist
any of the org is not active
any of the org is not a root-organization
Above API's will individually migrate the individual data
Approach 2
Provide a stand-alone script/migration job, which once invoked will migrate the data internally, with minimal user choices.
Pros
Simple approach, easy to implement
We do not have to expose such changes as API, as this is admin type of activity
Cons
Very specific solution - not much configurable from user perspective
User requires access to environment, as process will be run from background using environment variables.
This requires that server. instance must be down, while we are doing this migrations.
Data to be migrated (Identified as per database storage)
Child organization - Root org - may have other child organizations, we need to migrate those child organizations
User's root-org need to be updated to new root-org
Content data - channel name - need to be migrated according to new channel name.
User-org association, for old root-org need to be deleted, and new user-org association need to be added
geo_location data - which refers the root-org id also need to be migrated
badge_class_extension - also refers to the root-org
There are several storage structures which point to organisationid which need to be migrated if they are pointing to root-org which is being migrated.
course_management (organisationid, organisationname)
user_org(organisationid)
page_management (organisationid)
bulk_upload_process(organisationid)
Solution (Based on data analysis)
Get all user ids for Rajasthan Pilot Organisation
Update the rootOrgId, channel & slug for user ids fetched in above step.
Get all user-org entries where associated org id is Rajasthan Pilot.
Update the rootOrgId to point to Rajasthan root org.
Sync the users updated to ElasticSearch index
Get all course batches where createdFor is pointing to Rajasthan Pilot
Update the course batches and set createdFor to Rajasthan
Sync the course batch details to ElasticSearch index.
Mark the Rajasthan Pilot Org → inactive status.
Assumptions
Telemetry data will be pointing to according to old values for all the updates made so far, and will not be migrated.