...
As per the above diagram, Router job should route all ASSESS events into Assessment Samza Job which compute and update the RDBMS table.The assessment data product which read from the table and joins the data from multiple tables and will generates the assessment report per batch and uploads to azure cloud storage.
Data product input tables :
| Code Block | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||
usr_external_identity // To get the user external id information
user // to get the user information
course_batch // To get the details about the batch enrolled to course
user_courses // To get the information about the user enrolled to courses
user_org // to get the user organisation details
assessment_history (new) // Assessment result |
Disadvantages:
- When pipeline is having huge lag,Then report will be wrong
...