...
Installation and upgrade process will be easy
Cost of infra will be less as all the stateful workloads also run in k8s cluster
Step towards k8s native
Open source package unavailability issues will not occur as all the libraries are packaged as container image for non-containerised workloads
Scale up and scale down of the statefulsets are really easy
This will lead to adoption of newwer technologies such as tracing, service mesh, knative etc which can bring better visibility, security and scaling mechanisms
Challenges:
Need additional effort to setup operators for each stateful app and test
Database management/operations on k8s need to be taken care by SRE team post the installation
We need to do loadtest and benchmark the throughput
If statefulsets are scaled without thought, it can make or break the system
Needs good knowledge of Kubernetes and its ecosystem to ensure system is in a overall good shape
Some of the development activities will require changes in order to be able to run on Kubernetes
Approach2:
Leverage the Jenkins setup where containerised services are deployed to the kuberentes cluster and non-containerised services are deployed on to Virtual machines. Create wrapper jenkins jobs to provision & configure the services and deploy the services
...
Leverage existing ansible scripts to setup databases on Virtual Machines (Compute instances)
Jenkins based installation which user can visualise the builds and deployments
Most of the things are already in place and doesn’t require any new development effort
Jenkins UI provides good access management, audit and history of jobs that were triggered
Challenges:
Manage self managed servers
Always need to depend on package managers. If a old lib/package is removed, installation scripts will fail
Need to create Jenkins master jobs which in turn trigger other jenkins jobs of service provision, deploy and configure jobs in automated way
Users will need to switch context during installation, upgrade and the overall process is not a seem less experience
Two separate inventory and variable sources need to be maintained - one for the VM apps and the other for Kubernetes apps
Design discussion points:
We can go ahead with implementation of setting up containerised and non-containerised workloads on k8s cluster. Post the installation SRE team should be able to manage the infra including databases.
Add a disclaimer saying, if the SRE team is not capable enough to manage the database workloads on k8s cluster, it is recommended to setup the databases using cloud managed services or on self managed compute instances
Do the load testing and benchmark the setup for 2000 tps.
If the adopter has requirement of more than 2000 tps, it is recommended to setup the databases using cloud managed services
Enable configurability to connect to the managed database services from microservices and test with one CSP
Document the best practices to setup and manage the environment including security, scale, Monitoring monitoring and logging