Background 

When a deployment is in progress, we want to prevent users from using the system, and instead show them a maintenance message. However while the maintenance is on, the development team should be able to access Sunbird. 

The following need to be addressed

1. Show maintenance page when the deployments are happening with reason and duration.
2. Block all traffic to API's when site is down
3. Allow developers to use some way to access sunbird when maintenance mode is on

Solutions 

A. Node js middleware

Implement a NodeJS middleware to intercept all the requests coming to the site. When maintenance mode is on, all users will see the maintenance page when they try to visit any page. A separate link allows developers to key in a "access code" that lets them bypass the maintenance page.

How to turn on / off maintenance mode

# put app to maintenance
HTTP POST http://youapp.com/maintenance?access_key=[SECRET_KEY]

# back to normal
HTTP DELETE http://youapp.com/maintenance?access_key=[SECRET_KEY]

Pros

• The user interface is neater, and visitors will see the maintenance message. Only developers who know the bypass login link can bypass the maintenance mode. 

Cons

• Additional node module
• Will not affect any services that are not proxied via node, eg: Keycloak

Implementation details: 

The node "maintenance" module https://github.com/likeastore/maintenance will be enhanced to allow the bypass feature. Every time the request comes middleware in a node that checks the maintenance flag and if flag is set it will check for user cookie. If maintenance mode is ON and user cookie is not set then maintenance page will be displayed. The cookie gets set when a developer puts in the right access code. The access code can be a shared one.

B. NGINX Basic Auth

We can restrict access to website by implementing a basic auth. When maintenance mode is turned on any visitor to the site will see a basic auth prompt. This prompt will allow developers to access sunbird i.e. bypass maintenance. After successful login users can browse the site in normal mode. If the users click on cancel maintenance page will be displayed. Admin login credentials can be static and shared across the teams for sanity testing.

Other users may hit cancel, and they will be presented with the maintenance message.

How to turn on / off maintenance mode

Environment variable can be used to to set and unset the maintenance flag, Nginx will use this environment variable value to turn on/off basic auth. The proxy containers will need to be re-launched with the maintenance environment variable 

Pros

• Simple implementation, uses existing basic auth provided by NGINX
• Ensures all services are made offline, since all services go via proxy

Cons

• Not so great UX, since regular visitors will see a login prompt instead of the offline message until they click cancel

Implementation details

https://docs.nginx.com/nginx/admin-guide/security-controls/configuring-http-basic-authentication/