Analysis Doc for P0 items

1. Upserting Dimensions and Events 

Dimensions can be upserted by modifying the ingestion API (/ingestion/dimension) to directly add/update the dimension into the database. 

Need Clarity

To upsert events we are not storing the events anywhere in the database, So how do we achieve the upsert functionality for the events?

2. Making Index as optional

Open

As seen in the above diagram, unique constraints are created for the columns “school_id” and “school_name”. The column “school_id” can be unique but  “school_name” alone cannot be unique because there may be more than one school with the same name as shown in the below example. Ex: school_id, school-name

1      ,  “Kendriya Vidyalaya”

2      ,   “Kendriya Vidyalaya”

As we have created an index for the “school_name” it is expecting to have unique values and there is an issue while data ingestion.

Example: 

PK,Index

String,string,

school_id,school_name

Approach to resolve the above issue:

To overcome the above scenario we can have composite unique constraints for the combinations of school_id and school_name. To specify the composite unique constraint remove the first row and add another row at the last in the dimension grammar 

Existing Approach:

PK,INDEX

String,string,  →  Data Types

School_id,school_name → Columns

Suggested Approach:

String,string,   → Data Types

school_id,school_name→ Columns

“school_id”, “school_id, school_name”→ Unique Constraints

As seen in the Suggested Approach we have another row at the end for unique constraints, so in the given example we need two unique constraints which is school_id and combination of school_id and school_name as composite keys.