Versions Compared

Old Version 11

changes.mady.by.user Kartheek Palla

Saved on

compared with

New Version 12

changes.mady.by.user Kartheek Palla

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Expand
titleRepo: github.com/project-sunbird/knowledge-platform

/knowledge-platform/assessment-api (assessment-service specific code)

/assessment-api/ assessment-service (play application)

/assessment-api/ assessment-actors (actor implementation)

/assessment-api/ qs-hierarchy-manager (hierarchy-manager)

/knowledge-platform/ontology-engine (assessment-service uses entire ontology-engine)

/ontology-engine/graph-common

/ontology-engine/graph-core_2.11

/ontology-engine/graph-dac-api

/ontology-engine/graph-engine_2.11

/ontology-engine/parseq

/knowledge-platform/platform-core (assessment-service uses entire platform-core directly or via ontology-engine)

/platform-core/actor-core

/ontology-engine/schema-validator

/ontology-engine/platform-telemetry

/ontology-engine/platform-common

/ontology-engine/platform-common

/ontology-engine/cassandra-connector

/ontology-engine/kafka-client

/knowledge-platform/platform-modules (assessment-service uses only import-manager module)

/platform-modules/import-manager

/platform-modules/mimetype-manager

/platform-modules/url-manager

Expand
titleRepo: github.com/project-sunbird/knowledge-platform-jobs

/knowledge-platform-jobs/jobs-core (common module for all flink jobs)

/knowledge-platform-jobs/auto-creator-v2 (inQuiry BB job)

/knowledge-platform-jobs/publish-pipeline

/publish-pipeline/publish-core (common module for other publish job)

/publish-pipeline/questionset-publish (inQuiry BB job)

Dependency with Knowlg BB:

  1. Currently, all common modules are part of knowlg BB and are tightly coupled with knowlg/inQuiry services/jobs.

  2. inQuiry uses the below api’s

...

  1. Is separate infra monitoring (e.g: monitoring service, grafana dashboard) will be installed for inQuiry?

  2. How load testing will be performed for inQuiry components? Do we get common load test infra?

  3. Currently, all assessment APIs are onboarded with Content Role (e.g: Content Create or Content Update). Should we change the API roles to Question/QuestionSet Role (e.g: Question Create)?
    Note: this will have an impact on SunbirdEd/Diksha/etc)

  4. What will happen to service telemetry (e.g: info/error logs) dashboard such as Graylog? Do we need it? If yes, what all services/components need to be installed?

  5. Some basic functionalities around any objects (e.g: audit history, audit-event ) are currently with Knowlg. Should inQuiry also have these?

Proposed inQuiry BB Changes:

Proposed Git Repos:

Expand
titleSunbird-inQuiry/quml-service (for assessment-service )

/quml-service/assessment-service
/quml-service/assessment-actors
/quml-service/qs-hierarchy-manager
/quml-service/build (build script)
/quml-service/schema (question & questionset schema)
/quml-service/definition (sunbird supported primary category & its definition script for quml)
/quml-service/tests (API Functional Test Script/Code)

...

Expand
titleDevops repos

Sunbird-inQuiry/devops (devops script for provisining, deployements)

Sunbird-inQuiry/devops-cred or devops-private (private repo)

Deployment Structure:

  • inQuiry is dependent on the below services and jobs from Knowlg BB:

    • search-service

    • taxonomy-service

    • content-service

    • search-indexer flink job

  • inQuiry will install all dependent services/jobs in its own infra.

inQuiry & Knowlg using Same DB:

...

inQuiry portal will use Knowlg channel, category definition, asset & search APIs.

Micro Services

assessment-service:

  • assessment-service will be deployed with its own configuration.

  • assessment-service will be configured to make db calls for category definitions.

flink jobs

  • questionset-publish:

    • It will be installed with inQuiry specific configuration.

  • auto-creator-v2:

    • the job makes 2 api call (get hierarchy & add to hierarchy).

    • the api calls will be made using internal content-service url and end points (url & endpoint will be a configuration)

  • search-indexer:

    • No need to install search-indexer job for inQuiry, as graph db is same.

DB’s

  • Neo4j:

    • inQuiry doesn't need to install db extension, if graph db is same.

  • Logstash:

    • No need to install for inQuiry

  • Kafka:

    • assessment-service will use its own kafka topic for publishing question/questionset.

  • Cassandra:

    • inQuiry will have its own keyspace names.

  • Redis:

    • inQuiry will use db number assigned to BB.

  • ES:

    • inQuiry will use Knowlg search-indexer job and will write data to Knowlg Index (e.g: compositesearch)

    • inQuiry will use Knowlg search-service and consume data

inQuiry & Knowlg using Different DB:

...

inQuiry portal will use Knowlg channel, category definition, asset & search api's.

Micro Services

assessment-service:

  • assessment-service will be configured to make db calls to get the category definition as taxonomy-service will be installed.

flink jobs

  • questionset-publish:

    • It will be installed with inQuiry specific configuration.

  • auto-creator-v2:

    • the job makes 2 api call (get hierarchy & add to hierarchy).

    • the api calls will be made using internal content-service url and end points (url & endpoint will be a configuration)

  • search-indexer:

    • inQuiry will install search-indexer job with inQuiry specific configuration.

    • The job will write data to inQuiry Index (e.g: inquiry_searchindex)

DB’s

  • Neo4j:

    • inQuiry need to install db extension, because graph db server is different.

  • Logstash:

    • Need to install for inQuiry

  • Kafka:

    • a inQuiry specific topic need to create for search-indexer job also.

    • assessment-service will use its own kafka topic for publishing question/questionset.

  • Cassandra:

    • inQuiry will have its own keyspace names.

  • Redis:

    • inQuiry will use db number assigned to BB.

  • ES:

    • inQuiry will use its own search-indexer job and will write data to own Index (e.g: inquiry_searchindex)

    • inQuiry need to deploy search-service from Knowlg BB to consume the data. the service will point to inQuiry index.

Code Changes:

quml-service:

configuration change:

  • Cassandra's keyspace configuration will be updated as per the new keyspace format.

  • New Keyspace Format: [env_name]_[bb_name]_[keyspace_name] . e.g: dev_inquiry_hierarchy_store

  • Kafka topic configuration will be updated as per the new format. New format: [env_name]_[bb_name]_[topic_name]

...

  • Cassandra database script will be modified to have a building block name in keyspaces.

  • Kafka topic creation script will be modified to have a building block name.

...

  • all common modules listed in the Current Code Structure section will be pushed to either a common repo or different repo for each module - Need to decide by considering maven central movement for each component?

  • For resolving common module dependency, we have the below options:

    • option 1:

      • Check out all dependent repos along with the main repo and do a sequential build (dependent module first. e.g: platform-core then graph-engine) and then build the main repo.

      • A custom Jenkins build script with the parameterised branch will be written for the same. so that inQuiry can point to a specific branch/tag for the dependent component.

      • We need to have a similar build script for local checkout and development.

    • option 2:

      • Add all dependent modules as submodules.

      • Modify the Jenkins build script to checkout the main repo along with all submodules.

      • For all submodules, a specific branch (e.g: 4.9.0) will be configured.

  • For service level code movement, we have the below option:

    • option 1:

      • only assessment-service related code will be committed manually in the proposed structure because we need to take the code out from knowledge-platform/assessment-api folder.

    • option 2:

      • the entire knowledge-platform repo can be moved and then we can delete the unused code and modify the current service folder structure.

  • the graph-engine module needs to be enhanced to either use a DB connection or API call for fetching category definition. the default option would be DB call unless it is configured for API call. - for now, we can deploy taxonomy-service under inQuiry BB and use the DB call or can directly point to Knowlg keyspace. this enhancement can be taken up in the next release.

...