Page Comparison

...

DB Changes

We will add one column cloud_storage_data in cassandra table → bulk_upload_process,which will store the data on upload. It will be a json format data, in following format.

Above data will be stored in encrypted format within database, so one cannot decipher the data at all.Data Definitions

Structural/Code Changes/AlgorithmAlgorithm for approach 1

• Before marking the status of bulkupload to completed, we will check the type - if it is Org or User or Location, it will continue the following process.

       Note: Currently only User/Org/Location will use this functionality, hence we will be enabling it only for this

...

Public API exposed3 functionalities.

• Create a temporary CSV file, with name <type>_<processid>.csv.
• Read the configuration for user-friendly names from system settings.
• Read the first row from results (success/failure) and map the columns to user friendly names to write the first row.
• Add one more column Result to mark the results, as Success or Failure
• Now start adding the result rows, from success results first. mark the last column as Success in each case.
• Now start adding the result rows from failure results, mark the last column as Failure in each case
• Invoke the utility, which uploads the file into cloud storage based on : storageType, container, objectId, with pre-configured timeout.
• Expected environment variable to connect to the cloud-storage. Currently only azure will be supported, which requires two environment variables AZURE_ACCOUNT_NAME, AZURE_ACCOUNT_KEY
• In our case it will be currently 
  • storageType: "azure",
  • container: "org" or "user",
  • objectId: "user_101010" or "org_101110"
• Once successful, store the generated URL into our db column.
• Delete the temporary file.
• Mark the status of upload to Completed.

Algorithm for approach 2

• Only step after upload changes, rest are same as approach 1:
  • Once successful, encrypt the data used for uploading in above step as described in DB Changes section and store the data into cloud_storage_data column.

Public API exposed

GET upload/statusDownloadLink/processId

Output:

{

 "id":"api.upload.status",

"ver":"v1",

"ts":"2018-10-29 17:03:51:282+0530",

"params":{

"resmsgid":null,

"msgid":"1e44fd1f-acad-4f37-a3c5-69928eec98ca",

"err":null,

"status":"success",

"errmsg":null

},

"responseCode":"OK",

"result":{

"response":"signedUrl"

}

}