Page History

...

SUBJECT	DA - Harmonized Data Loading / Normalization
STATUS	Proposed
DECISION	Proposal: If the HDS is at rest, the loading of that data is the responsiblity responsibility of the member owner of the node. If the HDS is an API, the maintenance of that API is the responsibility of the Technical Steering Committee and the mapping to other data sources is the responsiblity responsibility of the member owner of the node.
DISCUSSION	Loading data will be via an API, IDM or ...? Will a direct SQL load be allowed? This will use the ingestion model. We should consider using graphQL as the extraction processing language or part of the extraction processing component to replace the current map reduce "extraction pattern"

SUBJECT	DA - Harmonized Data Format Governance
STATUS	Proposed
DECISION	Proposal: data schema, enumeration and the data dictionary will be standardized, and endorsed by the RRSC (and other groups per use case)
DISCUSSION	The Technical Steering Committee, Regulatory Reporting Steering Committee and the Data Model Steering Committee are all possible owners of this.

SUBJECT	DA - Harmonized Data Store
STATUS	Proposed
DECISION	Proposal: HDS can be persistent or transient. It's a member's decision. See next HDS can be persistent and can be used by member's as a "warehouse on the edge" for sharing data via openIDL
DISCUSSION	The data available for extraction must be normalized for multiple extractions across multiple use cases across multiple members. Is this one single model? Is this one database? Is this data at rest and/or available through an API

SUBJECT	DA - Harmonized Data Access
STATUS	Proposed
DECISION	Proposal: All access to "Harmonized Data" is through an API Member is responsible for the quality of the data retrieved and for certifying that a "request" for the data is supported.
DISCUSSION	If we determine that a standing harmonized store is not required, then we must establish an API with a standardized payload format that can be used to access the data. The member must "certify" that the data is available and quality in order to consent to a data extraction. The consent and certification can be captured on the ledger. The call to the API will come from the extraction processor. The extraction processor can run on the member node. Can the extraction processor run on the Analytics Node? If the extraction runs outside the member node, how does this work? Can it call the API directly? Must we use HLF to "transport" the data?

SUBJECT	DA - Harmonized Datastore DBMS Implementation
STATUS	Open
DECISION	Proposal: HDS will be a relational database. It cannot be a noSQL, graph, document DB etc. technical implementation of a HDS is non-prescriptive i.e. it can be MySQL, MS SQL, Oracle etc.
DISCUSSION	If data is at rest in the harmonized datastore, what is the technology? Does it need to be a single dbms? Should it be noSQL? Can it just be an interface?

...

Space shortcuts