Versions Compared

Old Version 15

changes.mady.by.user Sean W. Bohan

Saved on

compared with

New Version 16

changes.mady.by.user Sean W. Bohan

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Extraction error detection & handling

  • Everything needs to be edit-able
  • Fixes don't happen in current month (monthly correcting and then moving on)
  • Latency of error correction could be a year
  • need to make sure we have facility to capture corrections made while NOT bastardizing HDS
  • internal or architectural? DR is aware
  • SC - Errors:
    • missing information (on record provided)
      • current environment vs future
      • today - flat file from upstream, flat file submitted with missing limit, info passed to AAIS, flagged by AAIS, returned to carrier (can see instantly by state), these 2 states need fix made, go into SDMA to make fix then submit, AAIS approves, loaded by AAIS
        • it had already gone thru the edit
    • DH - load into SDMA, not approved yet, Susan makes corrections, goes thru edit again once Susan made corrections (see right away if fix worked), if in tolerance it is "approved" by AAIS
    • PA. - doing upload to SDMA, staging area, AAIS not running load until it is approved (edit package engaged)
    • SC - loading it to AAIS system, told to fix errors, fixes, then "officially submitting" and AAIS "approves"
    • PA - can't go to HDS until "approved"
    • DH - where within process is edit package? where is favcility to correct the errors, if HDS is supposed to be matching to source systems, then we shouldn't be making changes to HDS for other purposes beyond StatReporting - decision in ArchWG - how handle error corrections and fidelity of HDS
    • PA - direction, making update, go about making corrections of data already inside HDS, first example - data before HDS, different Error type
    • JB - case Dale mentioned, HDS is out of sync with source system, SS has error, needs time to fix, copies of DB with errors to be corrected - would suggest errors corrected get corrected in HDS but a log to inform source system of corrections as made - instead of lots of copies of collected data
    • JM - crossing boundary - doesn't care what carriers do - where do we stop caring - only thing, HDS has to be right, up to the carrier how they get it right
    • JB -yes but instead of making fix and a copy of DB it seems it should be ficec in HDS
    • SC - internal issue, AAIS needs to edit data, thats their job , if they say "2 errors" and they get fixed she says "done" and pushes to HDS - conflict with source system is something SHE deals with
    • JB transferred to AAIS for edit checks, 
    • PA - held before data lake until adter corrected
    • JM - cant occur until content is in it
    • PA - edit pre ETL
    • JB - do it 2x, if you correct HDS need to run edit in that environ
    • PA - how do we have chick-egg issue
    • JM - policy vs implementation ? - HDS is great cutoff point, everthing inside, up to the carrier to get it right in HDS - BUT Edits tell you whats right - carrier accountable up to HDS, if accountable on the carrier side and can verify before HDS, do the edits, send to HDS - what if I say "right, but edit stuff can't run iutnil other side - alrwady loaded to HDS - now what do I do? - accountability? where run edits is key question
    • PA - edit package run today, run on etl on load, no knowl on load - 2nd part AAIS does reconciliation after, sometimes errors arise
      • error type 1 - pre HDS , edit package fails on load - but what if loaded in HDS what is the recon process and what the process for that
    • JB - financial types of reconciliation
    • PA - yellowbook #s, compare #s submitted vs financial #s and due to granularity things come out wrong, financial reconciliation before stat reporting
    • JB - 1x year vs monthly
    • JM - reconciling financials? where?
    • SC - public info
    • PA - reach out to team with gap analysis, grey areas in codeing vs what they have, validate where /why numbers are off
    • SC - those arent errors, do reconcile, out of process doesnt become errors, differences and reasons why page 14 doesn't match - but NOT errors
    • PA - validity AAIS gets turning in reports on carriers - not only passed edit package but biz data matches fin data and a reason if it doesn't - why states listen to AAIS, how are we ensuring we are doing stuff correctly
    • JB - diff record exception 
    • JM - annual value add - edits? HDS needs two stage?
      • think its right but flag then run edits and get "ok/not ok" - question - who runs the eidts? in principle edits run on anyone centralized db
    • JB - copy of edits made avail to all
    • DH - one body resp for edits, not every single carrier 
    • JM - you put data in HDS, centralized code runs on all dbs, puot into HDS in some manner "this is not fully approved/edited" and decision: edit in place or is it a 2-stage thing?
    • SC - even if every carrier ran edit package themselvess, ult AAIS HAS TO RUN EDIT PACKAGE - resp lies with statistical reporting partner
    • PA - extract patterns to un T/F that a package was run - do test on clean or dirty data
    • JM - edits form of extractPattern, is it sufficient if it checks all the data
    • PA - regulator! 
    • JM - need feedback - run edits, if answer wrong, accountability to get it right
      • phys load or set flags
    • PA - should be running edits before load,
    • JM - WHERE? edits have to be consistent lang, thing needs to be well-defined structure
    • PA - rules engine, java, repackage rules engine as step in process going thru load (pass/no pass) 
    • JM - engine has to run against well define struct - b/c our data runs against well defines struct, now you are in HDS? put it into well def struct to run the rule that is the post-edit vers of that structure
    • PA - messaging format of HDS - stat plan, objects, run edit package against that 
    • JM - stat loading and knowl, if run edits against that, once passes - put it somewhere else or flag it - 2 concepts pre and post - saying to all carriers it needs to be PRE data but it has to have a shape - HDS? JM perceives when you demand "struct in diff way" and sees it as HDS
    • PA - diff pipleline but sees why it is outside of HDS
    • JB - data standard for saying how data will be considered, keep in mind dist arch, AAIS can't run anyting on db at carrier - raw, wont be sent to AAIS
    • PA - collections of stat records, running rules against them, if HDS is stat plan JSONified, run EPs, passed valdiation and legit extract
    • JM - HDS is JSONified stat stuff, edits, things all can see are ALL HDS in his mind - if prescribing shape b/c edits won't work, first place carriers have to do that
    • PA - pipeline A before HDS, where prescribed the data hits first
    • JM - widget shape here, then ep - prescribing shape, set of edits then HDS -  pipeline A is a prescribed shape, do whateve it takes to get it right, once edit passed drop into HDS
    • DH - wants to have DavidR weight in
    • PA - Pipeline A (infra before HDS), need to pull rules engine, before how much do we want to control creation? JM talkign about HDS being a larger thing, where does the baloon around openIDL begin? PipelineA is infra, carrier does all before? will still design load up to plugin
    • JB - think of pipeline A as data format
    • PA - wont process and give feedback
    • JB - need data format to be standard to run rules against, gives flexibiltiy to reconstruct design with same format (transit from flat file to whatever). 
    • PA - docker image with initial process? where is the official inbound point of openIDL community vs carrier
    • JM - one step at a time - HDS in the dark (far right), run extract patterns on - before HDS has to pass edits - edits need to be centrally maintaineed, id DRules expecting  something - pipeline A - already in that shape - sayig to carriers, prescribe format of HDS, to be right prescribe the edits, has to hit a prescribe shape here - carrier can do whatever to get into that form, that form is prescribed, java thing, json, all prescriptive, no flexibility
    • PA HDS, cna write queries against, layering other things not HDS
    • JM - centralized group do edits, carriers get it into that shape, must be part of standard of stuff to be prescribed
    • PA - meat of Drules, lot of it is testing stat plan, start ingesting as json, chekcing positionality
    • JM - thou shalt not load HDS until edits passed, edits maanged, approved format, carrier must get data into shape - reload until passed and THEN move to HDS
    • PA - can we have a bucket, fire lambdas against it, won't move to secondary bucket until passes
    • DH - suppose use HDS for other things, communicating with reinsureres, something outside of stat reporting, now that HDS not necessarily reflects source systems
    • JB - source consistent, take time to get corrected, logically - more correct vers HDS
    • PA - HDS more right than source system
    • JB - fixed at HDS but not at source
    • JM - policy, carrier accountaibility, edit finds something wrong, iterates on changes, if it takes 6 months to get back to source, for next 6 months other reports dont reconcile - accountability in governance statement "if you find an error you are accountable to reconcile"
    • JB - consolidated data in HDS for other purposes, if corrections were in HDS the right place to do it
    • JM - betteer that doesnt line up is wrong
    • JB - log for where / when changes done
    • JM - carrier accountability - more right data - where is accountability to carrier? whatever it takes upsteam - tell us changes you made requirement - lof that says "to get this loaded here are 7 edits" - accountability to make it transparent
    • PA - meta on each row with last update date and what changed
    • BH - if systems dont reconcile - BAD - what else are we doing with it? problem to be solved, may be a log, sounds painful
    • SC - reality - keep a log today (she does of every change made) - most cases data SC didn't get on her file (stat file) - is it really diff from soiurce system? she didn't get it on her file due to mapping upstream -know zip code is wrong or vin is wrong dont change things in her file or tell source system theres too many (agents inputting) - ok if under 5%
    • JM - pracical question - do edits - syntactially and symantically: find alpha, dont know if someone mistyped VIN, but no idea T/F in real workd - HOW RIGOUROUS DO EDITS NEED TO BE? - even if edits flag error? can we accpt it?
    • SC - happens all the time, might get edit "limint on policy is $1MM and you got somehting else - not an error"
    • JM - 2 levels of edits? showstopped (dead) and one we accept
    • SC - wont ignore fact error was received, will go and looks "did I have the right limit" - edits help und if there is a problem - is it internal edits ?
    • JM - what is the purpose of an edit? dont edit more than you have to - what is the purpose in this context - all sorts of mech for internal correction - dont edit more than you need to without purpuse - some things you have to fix, principle: only put in edits b/c hardcore reason to do it (not just clean data"
    • JB - work to be done - application and analysis and insight, not policy-level corrections
    • JM - do edits have levels? severity of error (which means will it be addressed)
    • JB - sanity check errors vs record format errors - can and will catch but WHERE in process
    • DH - gut check for AAIS as stat agent on how rigourous they need to be
    • JM - levels - showstoping and scary and "oughta check"
    • JB - accuracy in general (THRESHOLD)
    • JM - confidence scores from address cleansers - 
      • showstoppers (break system)
      • competency score (".7 good enough? yaaay")
    • JB - data quality scores, pick battles
    • SC - basic: does every field get a val - current and future, if not ABCD - if that field is filled? if so whats in there, nebulous - stat agents bear resp of "data is reasonable", know it is not garbage, how much has to be "good" - what does "good" mean (every field filled w/ reasonabvle value"
    • JM - mTable that does this - argument - for every field "type, table, range, = score"
    • SC, come across something, didn't meet the threshold, kick it back?
    • JB - levels determine responce
    • JM - governance ? - value, string, etc. - dont measure if you aren't gonna govern it - if you are gonna put a rule in there, must have govenance polity - arch has to provide for edit layer and series of thresholds to get a score and governance policies by score

Reconciliation (make sure report is correct based on request - reasonability check on the report - NOT financial reconciliation)

...