...
The idea is to automate the build and deployment of various Jenkins jobs to a specific environment based on Github tags.
...
Jenkins jobs in staging environment will poll the repositories every 10 minutes to check if any new tags are created in the repo.
If new tag is found, the code will check if the tag is created in the deployment window. Lets say 16:00:00 to 18:00:00.
The start and end times are configurable using 2 variables in Jenkins - START_TIME, END_TIME
The tag will also be checked if it has a _RC string and whether it matchs the current release pattern. This is to prevent non RC tags to not trigger deployments and hotfix tags to not trigger staging deployements.
Example -
release-2.10.0_RC1 is valid
release-2.10.0, release-2.10.0_ABC, release-2.8.7_RC1 are invalid
Once the above check is passed, a new jenkins job is automatically created with the tag name.
Based on the job names, the respective build, upload and deploy jobs will be triggered.
A slack notification will be sent for successful build and successful deployements.
For intermediate jobs like uploading the docker image, artifacts etc, success notifications will not be sent and only failure notifications will be sent
...
Portal and Editors have a strong dependency. So during portal and editor tag creation, portal needs to wait till editors deployment is completed and then start as it downloads the editors from blob
Current design: Manual intervention to stop portal builds so that editor builds complete Dev team should co-ordinate and create editor tags first and ensure its deployed. After this, Portal should create the tag
Solution desired: Decouple these in someway
Planned change: Automated portal build in waiting state in case any editors builds are currently being built. If editor team creates tags, auto build portal again to include new editors
Certain builds when triggered parallely, causes overwriting of maven artifacts with a different code
Current design: Do nothing. If things break, it will be a reason to push teams to ditinguish distinguish their maven artifacts
Solution desired: Version the maven artifatcs properly in pom.xml and also use different naming conventions per service. Or separate submodules. Or even better, remove the submodules.
Few Yarn jobs have an option to select only a specific samza job to deploy
Current design: Deploy all samza jobs always as we dont have away to distinguish which samza job to deploy based on tag
Solution desired: Unknown
Planned Change: In Kubernetes, we will implement deployment of specific jobs
Depoyment tracker updates still need to be done by dev team for a couple more days for artifact promotion tracking and variables tracking. Also dev teams need to ensure variables are commited to private repo before creating tags
Current design: Variable updates are manual and cannot be automated
Planned change: A new summary job which will hold details on all the jobs that were run / triggered in that release. The summary file can be used for next environment promotion along with job details, tags etc.
Certain deploy jobs which dont have a corresponding build job needs to be triggered manually - Example: Admin dashboard jobs
Certain deploy jobs which have non standard parameters needs to be triggered manually - Example: Neo4jSyncTool
Different repos have different release version
Current design: Trigger those manually Planned change: Have a list of relase versions that we can match against to solve different repos having different release versions. Example - release-2.10.0, release-1.2.0, secor-0.25as long as they are in same version
Solution desired: Follow release cycle to start using the automated jobs
Submodule changes will not trigger a new build
Current design: Ask teams to create dummy tag in main repo
Soultion desired: Tagging of submodules and then updating submodule repository to point to these tags instread of branch / Removing the submodules all together
Tags for staging cannot be created outside the deployment window. Hotfix tag for preprod and above can be created any anytime of the day.
Updates - 28/04/2020
Deployment tracker will still be created for each release. This is to track only manual steps like:
New jenkins job from dev jenkins
Jenkins job configuration changes
Infra changes and provision jobs
Anything that needs to be run manually
Jobs which are not covered under the automated build and deploy process
For all ansible code repos like sunbird-devops, sunbird-data-pipeline, sunbird-learning-platform, branch will be used to deploy and no need to create tag for these repos in automated deployments
Final tag for ansible code repos like sunbird-devops, sunbird-data-pipeline, sunbird-learning-platform will be created before promotion to next environment.
Any new tags post promotion for ansible code will be done by respective dev teams
If there is a change in build step of Jenkinsfile, the dev teams should also ensure to make this change in a file called auto_build_deploy which is present in every repo.