{"id":3116,"date":"2017-04-24T11:28:54","date_gmt":"2017-04-24T11:28:54","guid":{"rendered":"http:\/\/wiki.davelevy.info\/?p=3116"},"modified":"2019-10-11T18:49:55","modified_gmt":"2019-10-11T18:49:55","slug":"data-lineage","status":"publish","type":"post","link":"https:\/\/davelevy.info\/wiki\/data-lineage\/","title":{"rendered":"Data Lineage"},"content":{"rendered":"

About Data Lineage ….. my notes.Or more accurately links.<\/p>\n

    \n
  1. https:\/\/en.wikipedia.org\/wiki\/Data_lineage<\/a><\/li>\n
  2. https:\/\/en.wikipedia.org\/wiki\/Meta-data_management<\/a><\/li>\n
  3. https:\/\/en.wikipedia.org\/wiki\/ISO\/IEC_11179<\/a> a repository standard,\u00a0http:\/\/aristotlemetadata.com<\/a>, an implementation of the standard at apache.<\/li>\n
  4. https:\/\/en.wikipedia.org\/wiki\/Big_data<\/a><\/li>\n
  5. A white paper, “Lineage Tracing for General Data Warehouse Transformations<\/a>” from the wikipedia article on data lineage.<\/li>\n
  6. Practical Lineage Tracing in Data Warehouses<\/a>, by Cui & Widom, they pose a scenario and show how storage and inversion allows linegae to be captured and queried. The problem is exclusively defined in SQL.<\/li>\n
  7. Data lineage demystified\/<\/a> at dataversity.net, this is easy to read and mainly talks about why. It is a puff piece for ASG<\/strong>, who have tools in this market and have published a white paper, they host it at whitepapers.dataversity.net<\/acronym>, I have mirrored it here.\u00a0here…<\/a>. ASG advertise their Enterprise Intelligence and Data Lineage solutions hsted here…<\/a>, published here …<\/a>\u00a0and mirrored here<\/a>.\u00a0All this material is strong on why, and the coverage of the problem, weaker on explaining how its taps work, and how its reports meet business need. They published a case study with 5 examples<\/a> of the use of their tool.<\/li>\n
  8. The architectural scope of data governance<\/a>\u00a0a blog on Informatica’s site by Bob Kerel. He argues this is a process and needs a framework. He categorises the heterogeneity of the sources and repos, and classifies the problems as profiling, discovery, semantics (aka glossaries) and management\/lineage. I wonder if the semweb people have anything to offer.<\/li>\n
  9. Supporting Fine-Grained Data Lineage in a Database Visualization Environment<\/a>, authors are Woodruff & Stonebraker. Is this the paper that invented “lazy macro”? \u00a0Hover here for more.<\/acronym><\/li>\n
  10. https:\/\/www.timmitchell.net\/post\/2016\/05\/06\/etl-data-lineage\/<\/a><\/li>\n<\/ol>\n

    I used a picture from here <\/a>as the featured picture as graphs are a good visualisation of data lineage (it would seem)<\/p>\n

    See also<\/p>\n

      \n
    1. http:\/\/ilpubs.stanford.edu:8090\/525\/1\/2001-5.pdf<\/a><\/li>\n
    2. http:\/\/ilpubs.stanford.edu:8090\/403\/1\/1999-47.pdf<\/a><\/li>\n<\/ol>\n

      ooOOOoo<\/p>\n

      The more I look at this, why graphs and not ELHs<\/a>.<\/p>\n

       <\/p>\n","protected":false},"excerpt":{"rendered":"

      About Data Lineage ….. my notes.<\/p>\n","protected":false},"author":1,"featured_media":3117,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":"","_share_on_mastodon":"0"},"categories":[3],"tags":[1120,1109,667,911],"class_list":["post-3116","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-technology","tag-data","tag-data-lineage","tag-graphs","tag-technology"],"share_on_mastodon":{"url":"","error":""},"jetpack_featured_media_url":"https:\/\/davelevy.info\/wiki\/wp-content\/uploads\/2017\/04\/got-graph-w650.png","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/davelevy.info\/wiki\/wp-json\/wp\/v2\/posts\/3116","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/davelevy.info\/wiki\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/davelevy.info\/wiki\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/davelevy.info\/wiki\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/davelevy.info\/wiki\/wp-json\/wp\/v2\/comments?post=3116"}],"version-history":[{"count":5,"href":"https:\/\/davelevy.info\/wiki\/wp-json\/wp\/v2\/posts\/3116\/revisions"}],"predecessor-version":[{"id":4559,"href":"https:\/\/davelevy.info\/wiki\/wp-json\/wp\/v2\/posts\/3116\/revisions\/4559"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/davelevy.info\/wiki\/wp-json\/wp\/v2\/media\/3117"}],"wp:attachment":[{"href":"https:\/\/davelevy.info\/wiki\/wp-json\/wp\/v2\/media?parent=3116"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/davelevy.info\/wiki\/wp-json\/wp\/v2\/categories?post=3116"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/davelevy.info\/wiki\/wp-json\/wp\/v2\/tags?post=3116"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}