Versions Compared

Old Version 3

changes.mady.by.user Sunil A S

Saved on

compared with

New Version 4

changes.mady.by.user Isha Wakankar

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Design changes

...

  • The current cache time for a device profile entry in Redis is 24 hours, which is updated by the Samza job by reading from the device profile Cassandra DB and add adding the device details to telemetry.
  • The same value is read from Redis instead of the updated value in Cassandra.
  • If any changes in the device location take place, they do not reflect in the Redis cache until 24 hours, even though the Cassandra has the updated value.

...

Changes to the device profile need to be a push mechanism from the Device Register API call instead of a pull mechanism after 24 hours. A Kafka event needs to be generated and the RedisUpdater a new Samza job needs to be created to update the cache entry with relevant details.

Design Flow:


Image RemovedImage Added


Solution:

The Device Register API writes the device profile changes to the Cassandra currently. Instead, the Redis Updater Job the DeviceProfileUpdater Samza job will now write the data to Redis as well as Cassandra by reading it from the Kafka topic.

...

  • The DeviceRegister Service writes the device profile details into a device-profile.log file.

    Code Block
    languagetext
    themeRDark
    titledevice-profile.log
    {
	"statecustomcode": "29",
	"country": "India",
	"iso3166statecode": "IN-KA",
	"device_id": "232455",
	"uaspec": {
		"agent": "Chrome",
		"raw": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.75 Safari/537.36",
		"system": "Linux",
		"ver": "68.0.3440.75"
	},
	"city": "Bengaluru",
	"countrycode": "IN",
	"state": "Karnataka",
	"statecode": "KA",
	"updated_date": 1566383863167,
	"devicespec": {
		"os": "Android 6.0",
		"make": "Motorola XT1706"
	},
	"districtcustom": "",
	"fcm_token": "asda",
	"firstaccess": 1565328282703,
	"statecustomname": "Karnataka",
	"producer_id": "dev.sunbird.portal"
}


  • Logstash will be configured to read from the log file and push the data into Kafka as an event.
  • A Secor process is run to take the device data from the Kafka topic and creates a backup.
  • RedisUpdater Samza DeviceProfileUpdater Samza job reads the Kafka event and:
       a) If device_id exists in cache, updates the cache and the existing device profile Cassandra DB.
       b) If device_id doesn't exist in cache, creates new entry in cache as well as Cassandra.

...