Introduction:
In response to the dynamic landscape of cloud services and the evolving needs of our partner, we are proposing strategic enhancements to the The Manage Learn Service interacts with various components of the client-cloud-service npm package within the Manage Learn service. Amidst these changes, the focus is on accommodating the unique requirements of the Adoptor initiative. Adoptor, our valued partner, operates on NIC servers with a distinct S3 protocol-based storage system, necessitating the introduction of an S3 Storage Service. This pivotal addition not only aligns our services with Adoptors infrastructure but also ensures compatibility with AWS, reinforcing our commitment to versatility and client-centric solutions. This proposal outlines the tailored modifications required for the Adoptor, emphasizing seamless integration while maintaining the generic nature of the client-cloud-service package.
Objective:
The current client-cloud-service package is designed to manage file uploads, providing pre-signed URLs for AWS, OCI, GCP, and Azure. However, we've identified a need to enhance our services to cater to clients utilizing a custom storage service following the S3 protocol, hosted on NIC servers.
Problem Statement:
As we strive to meet the diverse needs of our stakeholders. Adoptor is looking to adopt our services and capabilities, specifically focusing on Observation & Survey functionalities. However, ML services will be hosted on NIC servers, and they do not utilize the standard set of service providers such as AWS, OCI, GCP, and Azure.
In the course of our engagement with Adoptor, it has become apparent that their storage infrastructure is based on a custom storage service adhering to the S3 protocol. Furthermore, our Proof of Concept (POC) using the aws-sdk library has confirmed the feasibility of integrating our services with their unique infrastructurefor diverse cloud interactions in the application. There is a need to specify that the Manage Learn Service will be deployed in the NIC cloud, utilizing NIC cloud storage for evidence uploads. The use case for Manage Learn involves users uploading their evidence, such as files, photos, videos, and PDFs, to the cloud as part of their submissions.
However, a limitation arises as the current client-cloud-service only supports AWS, OCI, AZURE, and GCP. The challenge with NIC storage is that it is implemented using the S3 protocol, but there is no requirement for a specific region since it is hosted in India. Additionally, NIC storage does not work with pre-signed URLs; hence, the evidence needs to be uploaded directly to the cloud.
To address this, we propose introducing an additional service file for S3 with cloud storage, specifically designed for NIC's private cloud, following the S3 protocol. Within this service file, we will introduce an upload function to handle file uploads. This addition is necessary because in NIC, there is no concept of signed URLs, and direct cloud upload is the preferred method.
Objective:
The primary objective of this proposal is to extend the functionality of the Manage Learn Service by addressing compatibility issues with NIC cloud storage. Currently, the service interacts seamlessly with various cloud platforms, including AWS, OCI, AZURE, and GCP, but encounters limitations when dealing with NIC storage due to its unique implementation using the S3 protocol.
Proposed Changes:
Introduction of S3 Storage Service:
We propose the addition of a new storage service within the client-cloud-service package specifically tailored for are introducing an additional service file designed to operate with S3 protocol-based cloud storage. This enhancement aims to accommodate clients utilizing NIC servers and their custom storage service while ensuring compatibility with AWS, eliminating the need for a specific region. While primarily intended for compatibility with NIC cloud storage, it is worth noting that this service file can also function with AWS S3, although it is not the recommended use case.
...
Custom Configuration Parameters:
The existing methods for AWS, OCI, GCP, and Azure require certain configuration parameters such as S3StorageService file requires the following parameters: access key, secret key, endpoint, and region. The new S3 Storage Service will be designed to accept custom configuration parameters relevant to the NIC server's S3 protocol-based storage.Additionally, considering the NIC S3 bucket does not work with region-specific configurations, the S3 Storage Service will be configured to handle this unique requirementbucket name. Notably, specifying a region is unnecessary, as the service will be exclusively hosted in India.
Library Compatibility:
To ensure seamless integration with the NIC server's storage service, a A Proof of Concept (POC) has been conducted using utilizing the aws-sdk library . The proposed S3 Storage Service will be developed with compatibility for this library, making it easy for clients to integrate our services with their existing infrastructure.
Generic Nature of the Client-Cloud-Service Package:
Recognizing the generic nature of the client-cloud-service package, the new S3 Storage Service will be implemented as an extension to maintain the package's versatility. Clients can easily include the S3StorageService and provide extended values through the index file, allowing for a smooth integration processto ensure smooth integration with the NIC server's storage service. The implementation will focus on two main functions: upload and generating downloadable URLs.
Implementation Steps:
Code Modification:
Implement the S3 Storage Service within the client-cloud-service package.
Update the package to accept custom configuration parameters for the S3 Storage Service, including adjustments to handle the NIC S3 bucket's unique characteristics.
Compatibility Testing:
Conduct thorough testing to ensure compatibility with the aws-sdk library, considering the NIC server's storage service peculiaritiesIn the code modification, a new service file, S3StorageService, will be added to implement the upload and downloadableUrl functions. The index.js file will be updated to incorporate the new service file and export functions from S3StorageService.
Documentation Update:
Revise documentation to include details about the new S3 Storage Service, its configuration parameters, and integration steps. Clearly outline the handling of NIC S3 buckets without region-specific configurations.
...