Five New Ideas From 2010 MIT Information Quality Industry Symposium

Here are some quick thought from the first day of the MIT Information Quality Industry Symposium. It’s my favorite event of the year. I refer to it as the “anti-boondoggle.” All academic theory and very little vendor fluff. I suppose that what you get when MIT and the University of Arkansas organize events. I’ll either post another top 5 tomorrow, or a full recap.

Please comment if you’d like me to dive further into any of these topics.

1) Cloud Is No Longer The Focus

Last year everyone talked about Governance in the cloud. This year it’s dead. Why? I think it may be that this group, unlike the Sales 2.0, is focused on Enterprise scale monolithic systems. Last year at MITIQIS, many presentations were focused on the cloud impact on large scale information quality programs. This year, it’s all about internal, installed systems. I find this facinating. Did this group try cloud, and not see the value? Or is it that there is still a duality of idealogies: One that prefers to keep things internal, and a second that wants to move their IT responsiblility to SaaS apps?

2) Master Data Management (MDM) Isn’t The Only Solution

I was surprised that among the Information Quality vendors and practitioners, MDM was no longer the focus. Joe Bugajski focused on it, but others merely touched on how they would interact with MDM rather than focus on MDM as the central system in an Information Quality focused environment. This year, many people talked about Information Quality at the system level, and fixing business process and human interfaces to eliminate dirty data at the source. This reminded me of the Data Warehouse to Data Mart paradigm shift of 10-15 years ago. I just felt old writing that.

3) Data Quality is a Dirty Word

“Information Quality” is now in vogue. I was corrected several times in conversations when I mentioned data quality. This is somewhere between a more highbrow way of marketing ourselves, and snobery. I don’t think this matters in the least bit, but others believe it’s more accurate and lends more credibility to our practice. As you’ll notice throughout my writing, I resist heavily the practice of pluralizing the word data. I never write, “data are,” which I believe is gramatically accurate. I feel the same way here. I do “Data Quality” work, regardless of who says that term is wrong. All right… I’ll use it in this post and try it on for size. This is the Information Quality Symposium after all.

4) Free Sources Drive Down R&D Cost

Data is available from government sources and tools are available from open source communities. No surprise there, but there was in increased focus on it here at MITIQIS. Why? Talend, an information quality vendor, builds their tools on the back of those open source libraries. They credited various shared data models, methodolgies and data sources that allow them to shortcut proprietary R&B spend. Trillium also spoke up, and mentioned that they leverage some of the same open-source thinking in their full price solutions.

5) 60-90% of Operational Data is Valueless

I won’t say worthless, since there is some operational necessity to the transactional systems that created it, but valueless from an analytic perspective. Credit to Kirk Amidon for this insight - he attended the session where this stat was quoted. Similarly, Steve Adler from IBM and others discussed it in their presentations. Data only has value, and is only worth passing through to the Data Warehouse if it can be directly used for analysis and reporting. No news on that front, but it’s been more of the focus since the proliferation of data has started an increasing trend in storage spend. That wasn’t discussed at the conference… just my opinion.