Cuisinarts, E-Commerce, and ... Controlled Vocabularies

By Lou Rosenfeld,  Dr. Dobb's Journal
Jan 01, 2002
URL:http://www.ddj.com/184412772


My friend related this interesting e-commerce experience to me recently.

He wanted to buy a food processor. Being the sensible guy that he is, he wanted quality but didn't want to pay an arm and a leg for it. A used Cuisinart would fit the bill quite nicely. So my pal fired up his browser and went on his happy way to eBay, where he proceeded to enter a search for exactly what he thought he was looking for:

"Cusinart."

Now, my friend is a pretty bright guy. He has a PhD and a host of publications to his credit. He's been a successful entrepreneur and academic. Nowadays he's helping the federal government get up to speed on e-commerce. So misspelling Cuisinart is obviously independent of intelligence—something that could happen to anyone.

It gets better. He submits his search and, voila, eBay has a "Cusinart" available. He bids successfully, and the misspelled "Cusinart" shows up on his doorstep a few days later.

You think that's the end of story. It's not. My friend eventually realizes his mistake and out of curiosity goes back to eBay. This time he searches for the correctly spelled "Cuisinart" and finds lots of them listed—as well as lots of bids pending. More bidders mean higher prices. The upshot is that if my friend had spelled the word correctly in the first place, he'd probably would have spent roughly triple the amount for a "Cuisinart" rather than his bargain-priced "Cusinart."

So what's my point? Well, it's not so much about how to make the most of your eBay experience (besides, there are already whole books on playing eBay to your advantage). Instead, it's just another illustration of how using controlled vocabularies can be the difference between a productive, efficient site and ... well, giving away your old Cuisinart at a third of its worth.

Controlled vocabularies and the guessing game

Whether you realize it or not, you're already familiar with controlled vocabularies. The Library of Congress subject headings and Yahoo's search criteria are a couple of examples. So, as you've probably guessed by now, controlled vocabularies are predetermined sets of terms that fit together to describe a specific domain such as kitchen appliances, nuclear engineering, or dirt biking.

The terms are standardized because language is ambiguous. People use different terms to say the same thing all the time. Or, worse yet, the same terms can mean different things. Sometimes folks just honestly screw up—like my friend did.

By predetermining the terms that make up a controlled vocabulary, and using those terms to describe your site's content, you can minimize the negative effects that variants, synonyms, and various other annoyances can have on your site and its users.

Here's an example. Let's say you're a webmaster at AT&T. Your site describes a huge host of products; one of them is the One Rate plan. There are many pages in your site that deal with One Rate. The problem is that there isn't a standardized spelling for it. So, some pages are about the "One Rate" plan, others describe it as "1 Rate" and on and on. Here are some of the possible references:

• One Rate Plan
• OneRate Plan
• 1 Rate Plan
• The One Rate Plan
• Dividend income
• Income returns
• Investment income
• Consider using multiple controlled vocabularies to describe the same content. Organize each around a theme like product names, subjects, processes, audience types, and so on. Avoid mixing these themes together because that's mixing apples and oranges, making them confusing for users to understand and more difficult for you to maintain.

• Balance the desire to use multiple vocabularies with the overhead involved in applying and maintaining each. Just as the domain changes over time, so should the terms used to describe it.

• Plagiarize (I mean, borrow from) good vocabularies that are already found in similar web sites, not to mention the examples you'll find linked from the ASI site. You might even find some useful candidates in your organization's printed materials.

• As described above, automatically enriching a search query behind the scenes can improve retrieval. But that doesn't mean it should be transparent to users who might not understand what's happening. Always provide at least a basic explanation of how your site's searching system works.

• Finally, remember that a good controlled vocabulary not only describes a domain's content, but should also reflect the language of users. If you don't have a good feel for the kinds of words your site's users commonly use, then analyze your search engine's query log.

Junk in, junk out

Looking around the Web and you'll find a wasteland of chaotic vocabularies. It's almost amusing: with all the hype about e-commerce these days (somehow I remember commercial web sites existing long before I ever heard that term), webmasters are going nuts for taxonomies to describe their products and services. But you don't hear too much about what terms should be used to actually populate those taxonomies.

Same thing goes for the corporate portal and the Yahoo-ized intranet. Vendors of portal software, XML-based approaches, and other products espouse metadata as solutions to the challenges of searching and browsing. Again, their solutions are only halfway there. They provide you with descriptive metadata fields, but you'll still need standardized terms to enter into those fields. Otherwise, it's simply another case of junk in, junk out.

 Taxonomies

Value of Organized Knowledge

by Jack Bryar
21-Jan-2002





An Old Problem Gets Worse

In recent years, the volume of news and information resources available to the typical corporate employee has grown exponentially. Corporate Web traffic has jumped by over 600% annually. Web-available content exceeds several billion items. Executives frequently receive more than 200 e-mails a day. The amount of corporate data generated per employee doubles every 18 months.

• 80% couldn’t/wouldn’t build a working Boolean search
• 87% used less than 3 words



A big part of the problem is that the same term can have different meanings to different people. Not knowing which terms will uncover sought-after information is a significant barrier for many knowledge workers. Any successful strategy for managing information has to overcome this problem.

In 1814, Thomas Jefferson was so dissatisfied with the ruined and disorganized state of the documents at the Library of Congress that he donated his collection and then personally reclassified the all the books there.
-- Source: Systems of Knowledge Organization for Digital Libraries, Gail Hodge

XML to the Rescue?

The Internet has been described as the world’s largest library, with the books thrown all over the floor. Many corporate information systems look just as disorganized. Information managers are convinced that the best solution to this clutter involves wrapping up all electronic document forms inside a common format, so that the content inside can be more easily found, and used by different applications.

The wrapper being used by most organizations today is XML. XML allows the tagging of a document with a description of what the document is about, and where it came from. Searching on XML meta-tags can certainly simplify the search process.

Unfortunately, XML does not solve the problem of finding information. It only standardizes the problem. It requires that any XML tagging system clearly understand what the document is about, and it needs to anticipate the search process someone might try to use to find it. This takes time, a great deal of sophistication, or both. Otherwise, the process results in hiding essential documents behind generic, idiosyncratic or meaningless tags, making the information management and retrieval problem even worse.

In order for XML tagging to be meaningful for search and retrieval, the terms used to tag content have to be intuitive enough to encourage their use by information-seekers. They should be structured in a standardized way; less as a set of variable keywords and more like a set of subject categories. These subject categories should be set up in a hierarchical fashion, with logical subtopics and overviews. This, in short, is a taxonomy.

Enabling an ability to search or manipulate content, "by category" is an essential benefit of a successful XML tagging process.

Taxonomies Defined

Taxonomies are sometimes called "classification schemes" or "categorization schemes." Each refers to grouping together similar items into broad "buckets" or "topics" which themselves can be grouped together in ever-broader "hierarchies." Examples of taxonomies include systems as diverse as the Dewey Decimal system found in small libraries, Yahoo’s Subject Index, and the massive taxonomic system proposed by Linneaus used by generations of biology students. Wherever they are used, they have the same goal -- to organize knowledge about a given subject.

A sample taxonomy from NewEdge:



Taxonomies and The Search Process

Perhaps the greatest benefit to taxonomies is improved searching.

Properly constructed taxonomies simplify the process of gathering "the right" information for daily business use by simplifying the vocabulary used in the search process. Tagging systems using raw key words or similar strategies are likely to generate search error rates approaching that of straight text searches. For example, while a search on the word "DSL" will find stories on a particular type of broadband technologies, it will miss others, and may accidentally find content referring to Dutch sign language or Data SubLanguage.

A better approach would be to define these documents as belonging to the subject category, "Digital Subscriber Line." If the searcher can focus on a proven set of categories rather guess at keywords, chances of finding the right content, are far greater, and the process will be faster and more reliable.

The most important contribution of taxonomies to the search process is that they work.

Even using a relatively primitive taxonomic system, Microsoft reported a 40% improvement in hit rates. Satisfaction metrics doubled. In addition, the time spent trying to find a given document was significantly reduced. The success rate of taxonomic-based searching reduces the strain on systems and on the people who use them.

Business is Complicated



Naturally, one of the most important criteria for taxonomy is that it should be easy to navigate. But building solid taxonomies is much easier said than done. Consider, for example, a taxonomy of business subjects.

Businesses vary in size and have multiple points of focus. Business activities involve an array of subjects that do not always fall into logical groupings. Subject boundaries are often fuzzy.

Subject hierarchies can feel artificial, as content, particularly business critical content, may fall into multiple categories. Indeed, most executive-level business documents involve several categories. Traditional indexing schemes dissolve in complexity as the number of unique concepts grows.

So, while some subjects are relatively easy to categorize, most business functions are not. (I should know: NewsEdge has spent several years developing a proprietary business taxonomy). Nevertheless, you should seriously consider developing a taxonomy for the content management system residing underneath your e-business efforts in general, and your Intranet in particular. Your content contributors and end-users alike will be grateful.

 

Next:

Send Feedback

See all ECM Channel feature articles.

Need to select a technology vendor, but confused about your choices? See our vendor-neutral technology reports.

Join the conversation

Digg This | Search Technorati | Tag it on Del.icio.us




About the Author



Jack Bryar is a Practice Leader in the Knowledge Management Consulting and Editorial Services unit of NewsEdge Corporation. Bryar has helped NewsEdge clients to determine their information needs in industries ranging from energy production to international banking.  Prior to NewsEdge, Bryar led the I.T. Practice of a corporate advisory services consultancy based in Toronto, Canada.

Taxonomy Design Types

---


Barbara Blackburn
5/31/2006

So you’ve decided you need a taxonomy to categorize and organize your documents and records. But how do you decide what type of taxonomy to design? The type of taxonomy you choose is as important as the taxonomy itself. If the design doesn’t meet the needs of the users; it will not be used.

Taxonomy Types
Taxonomies are usually hierarchical. Categories (nodes) in the hierarchy progress from general to specific. Each subsequent node is a subset of the higher level node. There are three basic types of hierarchical taxonomies: subject, business-unit, and functional.

A subject taxonomy uses controlled terms for subjects. The subject headings are arranged in alphabetical order by the broadest subjects, with more precise subjects listed under them. An example is the Library of Congress Subject Headings (LCSH) used to categorize holdings in a library collection (see the example to the right). The Yellow Pages could be considered a subject taxonomy.

It is difficult to establish a universally recognized set of terms in a subject taxonomy. If users are unfamiliar with the topic, they may not know the appropriate term heading in which to begin their search. For example, a person searches through the Yellow Pages for a place to purchase eyeglasses. They begin their search alphabetically by turning to the E’s and scanning for the term “eyeglasses.” Since there are no topics titled “eyeglasses,” the person consults the Yellow Pages index, finds the term eyeglasses, which provides a list of preferred terms or “see also” which directs the person to “optical – retail” for a listing of eyeglass businesses.

LIBRARY OF CONGRESS SUBJECT HEADINGS


• H -- SOCIAL SCIENCES
• J -- POLITICAL SCIENCE
• K -- LAW
• L -- EDUCATION
• M -- MUSIC AND BOOKS ON MUSIC
• N -- FINE ARTS
• P -- LANGUAGE AND LITERATURE
• Q – SCIENCE
• R -- MEDICINE
• - Subclass RA Public aspects of medicine
• - Subclass RB Pathology
• - Subclass RC Internal medicine
• - RC31-1245 Internal medicine
• - RC49-52 Psychosomatic medicine
• - RC251 Constitutional diseases (General)
• - RC254-282 Neoplasm. Tumors. Oncology
  Subject Taxonomy Sample

  COUNTY GOVERNMENT BUSINESS-UNIT TAXONOMY
  

• Assessor
• Building
• Commissioners
• Coroner
• District Attorney
• Finance
• Health and Environment
• Human Resources
• Human Services
• Motor Vehicle
• Clerk and Recorder <------------
• Department
  
  • Elections <------------ Divisions
  • Motor Vehicle
  • Recording
  • - TD1000 <-------- Records
  • -- Warranty Deed
  • -- Quit Claim Deed
  • -- Subdivision Plat
    
  • Sheriff
  • Treasury
    Business-Unit Taxonomy Sample
    In both examples (LCSH and Yellow Pages), the subject taxonomy is supported by a thesaurus. A thesaurus is a controlled vocabulary that includes synonyms, related terms, and preferred terms. In the case of the Yellow Pages, the index functions as a basic thesaurus.
    
    

    In a business-unit taxonomy, the hierarchy reflects the organizational charts (e.g., department/division/unit). Records are categorized based on the business unit that manages them. The example above and to the right shows the partial detail of one node of a business unit taxonomy that was developed for a county government.

    One advantage of a business-unit taxonomy is that it mimics most existing paper- filing system schemas. Therefore, users are not required to learn a “new” system. However, conflicts arise when documents are managed or shared amongst multiple business units. For example, in the county government example above, a property transfer document called the “TD1000” is submitted to the recording office for recording and then forwarded to the assessor for property tax evaluation processing. This poses a dilemma as to where to categorize the “TD1000” the taxonomy. Another issue arises with organizational changes. When the organizational structure changes, the business-unit taxonomy has to change as well.

    In a functional taxonomy, records are categorized based on the functions and activities that produce them (function/activity/ transaction). The organization’s business processes are used to establish the taxonomy. The highest or broadest level represents the business functions. The next level down the hierarchy constitutes the activities performed for the function. The lowest level in the hierarchy consists of the records that are created as a result of the activity (e.g., transactions).

    The example above shows partial detail of a functional taxonomy developed for a state government regulatory agency. The agency organizational structure is based on regulatory programs. Within the program areas are similar (repeated) functions and activities (e.g., permitting, compliance, enforcement, etc.). When the repeated functions and activities are universalized, the results are a “flatter” taxonomy. This type of taxonomy is better suited to endure organizational shifts and changes. In addition, the process of universalizing the functions and activities inherently results in broader and more generic naming conventions. This provides flexibility when adding new record types (transactions) because there will be fewer changes to the hierarchy structure.

    One disadvantage of a functional taxonomy is its inability to address case files (or project files). A case file is a collection of records that relate to a particular entity, person, or project. The records in the case file can be generated by multiple activities. For example, at the regulatory agency, enforcement files are maintained that contain records generated by enforcement activities (Notice of Violation, Consent Decree, etc.) and other ancillary, but related activities such as contracting, inspections, and permitting. To address the case file issue at the regulatory agency, metadata crossreferencing was used to provide a virtual case-file view of the records collection.

    ARTWORK GOES HERE Which Taxonomy Type Should You Use?
    Each taxonomy type has its pros and cons. In most cases, a Hybrid approach combining the taxonomy types is the most appropriate.

    In choosing a taxonomy type, consider the following:

    • Gain an understanding of your organization and how the business units function and interact.
    • What are the needs of the users (both internal and external users)? Will you need multiple “views” or methods for records searching and categorization?
    • Where will the taxonomy be applied and what are the operating parameters or limitations of those systems? (electronic content or records management system, paper files, shared network drive, website, etc.)

Barbara Blackburn is a senior consultant with IMERGE Consulting (www.imergeconsult. com). She specializes in document and records management technology planning, selection, and deployment. Ms. Blackburn is also an instructor for AIIM’s ERM Certificate program. Contact her at (barbb@imergeconsult.com).

Data stewardship is a multi-faceted function which places the ultimate organizational responsibilities for data quality. Frequently, people refer to a data steward, as though a single person can perform all the activities. While a single person or group of people may ultimately be responsible for data quality, the stewardship functions are carried out by several people. The Zachman Framework perspectives are useful for determining the organizational responsibilities for different aspects of data stewardship.

The planner perspective provides an enterprise view. In this view, the major subjects of the organization are identified. One way of identifying the major subjects is to bring together business people from each of the major functional areas of the company so that the major subjects can be identified. A typical list of major subjects would include customers, products, competitors, facilities, human resources, materials, etc. Agreement on the list of major subjects, and the scope and definition of each, needs to be gained at an enterprise level. The business people who were instrumental in defining the major subject areas can form the nucleus of a stewardship council which would provide an enterprise view toward the data.

In addition to defining the subjects and their content, this council would also set the policies concerning the management of data as an enterprise resource. These policies would address data creation, maintenance, dissemination (including security) and disposition from a generic viewpoint, with the responsibility for executing the policies for individual data elements or subjects being under the control of the appropriate people.

In the owner perspective, the entities and attributes within each subject area are identified and defined from a business perspective. The relationships between pairs of entities and the domain of business values are also defined in the owner perspective. To a large extent, the work performed in this perspective is done within individual subject areas. Hence, it is appropriate to designate, for each subject area, a business person as the data steward to oversee the development of the owner perspective. The stewardship council would provide guidance for matters involving multiple subject areas or if consensus cannot be reached on the subject area within which an entity belongs.

Data analysts within data administration should be responsible for developing the (system) data model which governs the technical implementation of the business model. Data administration defines the overall naming standards and, therefore, may need to rename some of the entities for representation within the physical databases. Interaction with the business people is warranted to ensure that the resultant model is semantically consistent with the business model. If discrepancies arise, both parties need to be part of the resolution.

Once the system model is defined by the data analyst, a person within database administration must define the physical tables which preserve the integrity of the system model and meet the designated performance requirements. This is done in the builder view.

In the builder view, physical databases and tables are defined. The technical model used in this view may differ from the system model because of access or performance considerations. As with other views, some collaboration may be needed between data administration and database administration. Database administration needs to ensure that the referential integrity implied in the system model is preserved. The database administrator is then ready to create the physical databases and tables in the subcontractor view.

The physical databases are created by database administration in the subcontractor view. With modern repositories and CASE tools, consistency with the builder view is easy to guarantee, as the creation of the physical environment can be automated. The product of the process is the actual database which is eventually populated.

The final product is the set of populated tables. Data for these tables is often provided by individuals, and these individuals need to be accountable for the data entered.

The data stewardship roles exist to ensure the integrity of the product--the populated databases and tables. Without this integrity, the value of any database is very limited. Throughout the process, the stewardship functions are performed by different people. First, a group of people defines the major subjects of interest. These people need to have an enterprise-wide business perspective. Then the contents of each subject area of interest are defined by a person with a specific business perspective. The result is translated into an electronic representation of the business by a person with business and data analysis skills who applies information resource management principles to make the transformation. Finally, a person with technical database skills defines and creates the physical tables.

These functions are designed to help ensure that the data entered meets the enterprise needs. However, unless the person entering the data is also concerned with the data quality, the data may pass all the electronic validation rules and still be invalid.

This column focuses on the data stewardship responsibilities within the data dimension of the Zachman Framework. The steward also addresses the other dimensions. For example, the processes which are defined for ensuring the data quality are declared within the process dimension, the physical distribution of the data is declared within the locations dimension, the security rules are declared within the people dimension, the currency requirement and retention rules are declared within the times dimension, and the governing business rules are declared within the motivations column.

Jonathan G. Geiger is executive vice president at Intelligent Solutions, Inc.  Geiger has been involved in many corporate information factory and customer relationship management projects within the utility, telecommunications, manufacturing, education, chemical, financial and retail industries.  In his 30 years as a practitioner and consultant, Geiger has managed or performed work in virtually every aspect of information management.  He has authored or co-authored numerous articles and three books, presents frequently at national and international conferences, and teaches several public seminars.  Geiger may be reached at JGeiger@IntelSols.com.

Copyright 2007, SourceMedia and DM Review.

DATA STEWARD ROLES & RESPONSIBILITIES
Written by Robert S. Seiner
KIK Consulting & Educational Services, LLC & TDAN.com
[Return to the Article Archive]

© Copyright 2005 - Robert S. Seiner

Published in TDAN.com July 2005

Introduction

If you define Data Stewardship the same way that I define Data Stewardship, then we are apt to agree that defining data steward roles and responsibilities is at the heart of best practices for implementing a successful data stewardship program.

The definition that I use for Data Stewardship is “the formalization of accountability for the management of organizational data”.  By stating “formalization of accountability” I mean that there are already “de facto” stewards of data in every organization that informally steward the data from several perspectives.  The way I see it, you wouldn’t have any data if “someone” wasn’t responsible for defining what data was/is needed, “someone” wasn’t responsible for producing the data per the definer’s specifications, and “someone” wasn’t going to use the data in the first place.  These people that define, produce and use the data are the “TRUE” data stewards of the organization.

The questions are: Do they recognize themselves as data stewards?  Are they recognized as data stewards?  If they are already recognized as data stewards – do they know what it means to be a data steward or what a data steward is “really” supposed to do?  Is there anybody to assist them in becoming the “best” steward they can be by providing them a framework, discipline and education on how they and others can best manage the data of the company?  Is there anybody to guide them through paying specific attention to stewarding the data that they are accountable for?  Data steward roles and responsibilities are a critical component to successfully implementing a data stewardship program.

As an easy way to articulate data steward roles and responsibilities, I typically break the roles and responsibilities into two different categories – 1) Data Stewardship Program Roles – pertaining to those individuals that define, design and execute the administration of the data stewardship program and 2) the Participating Data Steward Roles – pertaining to individuals that are responsible for daily operations and the management (definition, production and usage) of enterprise data.  The Program Roles are often overlooked and seldom become a focus of a data stewardship program.  I believe this oversight is a mistake and thus in this article I have paid specific attention to defining both the Program and the Participating data steward roles and responsibilities. 

There is also a role for a person(s) that you may have initially identified or assigned “THE” data steward for a specific datum or set of data in your company.  Read on for a further description of where I see “THE” data steward fitting into the picture (and believe me, they play a critical role).

Above the Line, Below the Line

The next portion of the article defines the different type of roles and responsibilities that you may want to consider to include in the definition of both the program and participating roles.  Figure 1 below displays a separation of the Program Roles and the Participating Steward Roles - which I refer to as the "above the line" roles and the "below the line" roles.


Figure 1 - Steward Roles Above and Below the Line

Data Stewardship Program Roles

The Data Stewardship Program Roles focus on setting up the Data Stewardship Program.  The individuals that play the program roles are not the Data Stewards themselves and most often the only data that the program roles are responsible for is the data about the stewards and the data about the data they steward.

I describe these roles to be "above the line" drawn across the middle of Figure 1.  These roles focus on understanding the requirements for the data stewardship program, understanding the desired outcome of the data stewardship program, and they focus on architecting the best possible data stewardship solution for the organization.  More often then not the program roles originate in the IT organization even though the data stewards themselves reside in the business organizations.  The Data Stewardship Program Roles are responsible for finalizing participating steward roles and responsibilities, defining how data stewards will be "recognized" (not assigned), defining the database for collecting information about data stewards, and disseminating the data steward information (meta-data relating stewards to the data they steward) to those individuals that will benefit from using the stewardship meta-data.

Data Stewardship Program Manager

As part of the Program Roles, the Program Manager operates, manages and oversees the entire Network and Program.  The Program Manager coordinates the efforts of the Project Manager, Subject Matter Managers, and the Participating Data Stewards (see below for descriptions of these roles).

Program Manager Responsibilities (may delegate to Project Manager defined below):

• Working with the Project Manager to design and implement the Program for the company according to the company’s best practice definition of their data stewardship program.

• Working with the Project Manager to create and provide educational materials and guidance to Subject Matter Managers and Participating Data Stewards.

• Working with the Project Manager to identify (with the Subject Matter Managers) the appropriate Participating Data Stewards including Data Definers, Data Producers and Data Users as well as Subject Matter Managers.

• Working with the Project Manager to ensure that the data stewardship information is available and accessible through the a stewardship meta-data repository that contains information that links Participating Data Stewards to the data they steward.

• Working with the Project Manager to ensure that the processes pertaining to data stewardship information in the are followed to assure that the program is current/active and that the data stewardship information maintained in the is kept up-to-date.

• Working with the Program Manager to design and implement the Program for the company according to the company’s best practice definition of their data stewardship program.

• Creating and providing educational materials and guidance to Subject Matter Managers and Participating Data Stewards.

• Identifying (with the Subject Matter Managers) the appropriate Participating Data Stewards including Data Definers, Data Producers and Data Users as well as Subject Matter Managers.

• Ensuring the data stewardship information collected is available and accessible through the repository.

• Ensuring the Data Stewardship Change Management procedures are followed to assure that the program is current/active and that the data stewardship information recorded in the stewardship repository is kept up-to-date.

• Identifying (with the Program Manager) and coordinating the activities of the appropriate Participating Data Stewards including Data Definers, Data Producers and Data Users of data that falls under the subject matter.

• Ensuring the Data change management procedures are defined, followed and enforced to assure that the program is current/active and that the stewardship information in the repository is kept up-to-date.  This may involve periodic reviews of the procedures and enforcement of these

• Identifying the specific data that is needed to operate the business processes of their areas.

• Recording business definition and appropriate meta-data (information about the data such as business name, business description, valid values, etc.) for the data they define by utilizing the tools and processes that have been identified and that are supported by the Data Stewardship Program Manager's resources (above the line).

• Identifying opportunities to share and re-use data. 

• Identifying data quality standards. 

• Participating in the enforcement of data quality standards and identifying and resolving to data quality issues pertaining to the data that they define.

• Ensuring the the quality, completeness and accuracy of data definition.

• Identifying the specific data that is needed to operate the business processes of their business areas.

• Recording business definition and appropriate meta-data (information about the data such as business name, business description, valid values, etc.) for the data they define by utilizing the tools and processes that have been identified and that are supported by the IT organization.

• Identifying opportunities to share and re-use data. 

• Identifying data quality standards. 

• Participating in the enforcement of data quality standards and identifying and resolving to data quality issues pertaining to the data that they define.

• Ensuring the quality, completeness and accuracy of data definition.

• Responsible for communicating new and changed business requirements to individuals who may be impacted

• Responsible for communicating concerns, issues and problems with data to the individuals that can influence change

• Producing (inserting, updating, deleting) business and technical data in the IT systems that support the business processes.

• Validating data that enters and exits business processes.

• Coding and editing accurate data quality standards (including format, content, and data dependency) for the data they produce.

• Ensuring the quality, completeness and accuracy of data production according to the definition of the data provided by the Data Definition Stewards.

• Responsible for communicating new and changed business requirements to individuals that may be impacted

• Responsible for communicating concerns, issues, and problems with data to individuals that can influence change

• Accessing and Using the data for its intended purpose.

• Accessing information (meta-data) available about how the data was defined for the business and how the data was produced in the information systems in order to use the data for its intended purpose.

• Ensuring the quality, completeness and accuracy of data usage according to the definition of the data provided by the Data Definition Stewards.

• Responsible for communicating new and changed business requirements to individuals that may be impacted

• Responsible for communicating concerns, issues, and problems with data to individuals that can influence change

There are typically more Data Usage Stewards then there are data definers and data producers.

The last two bullets in each of the Participating Data Steward Roles described above are identical and they are very important.  The Participating Data Stewards are the “eyes and ears” of the organization when it comes to knowing the data.  If a Participating Data Steward identifies a problem with the definition, production and/or usage of data, they must be responsible for bringing that issue to the attention of someone who can bring on change.

Conclusion

Flying in the face of my whole 3D approach to data stewardship (click here to see an old article I wrote called Data Stewardship in 3D) I am sad to say that many of the companies that I have worked with have initiated data stewardship programs by identifying a person (singular) as an “owner” or “THE” data steward of a particular piece or set of data.   Does this sound like you?

My approach for defining data steward roles and responsibilities states that, more often than not, one participating steward per datum or data set is not effective or enough.  Especially if “THE” data steward is significantly far removed from the daily definition, production and usage of the data they are stewarding.  “THE” data steward may be formally accountable for the data – but this article suggests that companies should consider identifying the data definers, producers and users of the data and educate them on the roles and responsibilities defined here.  You will not be sorry you did.

This short article has described a very practical, pragmatic and most importantly an actionable approach to defining data steward roles and responsibilities in your organization.  This article should help you to determine who the “TRUE” data stewards are.

Once your organization has identified the data stewards the next question becomes – “what do we do next?” or “how do we get these stewards to truly participate in the data steward role?”  That is certainly a very good question and one that will be discussed in a future TDAN.com article.  If you need to know sooner, contact me at rseiner@kikconsulting.com and we can talk about it.

© Copyright 2005 - Robert S. Seiner
  

---

Robert (Bob) S. Seiner is recognized as the publisher of The Data Administration Newsletter, LLC – www.TDAN.com - an award winning electronic publication that focuses on sharing information about data, information, content & knowledge management disciplines. Mr. Seiner speaks often at major data management and meta-data management, business intelligence and knowledge management related conferences and user group meetings across the U.S. He can be reached at the newsletter at rseiner@tdan.com or 412-220-9643.

Mr. Seiner is the President and Principal Consultant of KIK Consulting & Educational Services, LLC – www.KIKconsulting.com - a company that focuses on consultative mentoring, a blend of consulting and educational services to enable companies to improve and gain value from the management of their data and information resources. Mr. Seiner offers 2-day in-house and public courses on how to build and implement data governance / stewardship programs and meta-data programs.  He can be reached at the consulting firm at rseiner@kikconsulting.com.

 

 The Case for Data Stewardship
 
by William Laurent



Summary: Having first-class data significantly helps all aspects of your organization's business - helping to effectively gauge and manage risk, avoid redundant work loads, increase customer satisfaction and provide better business intelligence and decision-making support.

The Sarbanes-Oxley Act of 2002 (particularly Section 404), has created a sense of urgency for senior executives to take an active interest in the accuracy, consistency and timeliness of their data. Now more than ever, corporate compliance, audit and governance issues must be efficiently resolved. Yet it is widely believed that less than half of large companies have a formal data stewardship or data quality program that protects and leverages their unique strategic asset of data. Smart businesses are proactive, not reactive; they comprehend the business need for quality data. Aside from compliance and governance agendas, having first-class data (and the ability to unlock it) significantly helps all aspects of your organization's business - helping to effectively gauge and manage risk, avoid redundant work loads, increase customer satisfaction (on the front and back ends) and provide for better business intelligence and decision making support. Conversely, poor data quality will have injurious consequences across all enterprise operations - negatively impacting the most critical customer service obligations (such as billing and remittance processing), skewing decisions and fulfillment obligations, and creating a host of other tactical problems. Numerous hidden business costs such as revenue leakage and higher capital expenditures may all be maladies that stem from lack of data integrity. While there are tools that can help with and augment your formal methodology, capable data stewardship requires the symbiotic merging and integration of the automated (technology) with the manual (people). For many organizations, data problems remain secreted or hidden until issues are uncovered that result from a formidable event of costly business impact, such as an external audit, fraudulent activity, subpoena or valued customer satisfaction issue. A prolific variety of complex legacy data (which may go through many iterations of manual reconciliation on its way up to senior management's eyes) lingering in cloaked and "siloed" structures will always be poised to wreak havoc when least expected. Today, with the advent of widespread Internet and e-commerce applications, the data problems of an enterprise can be exposed to the entire consumer world. If you have bad data on your customers (e.g., not recognizing them across accounts), they will know - often long before you catch on. The time to establish a formal data stewardship program is now!

When implementing a formal stewardship program or policy for your organization, it will be important to appoint the right person as the lead steward and assign the steward an appropriate title. This person will understand and approach duties from the perspective that data quality is a collaborative business and IT matter as he/she champions high-quality data across all multinational systems - from repositories to reports. The most effective stewards will be familiar with core business values and practices but should also be able to understand data models, tech-speak and data storage topics from a high and low level. (A systems analyst background with relevant industry understanding and outstanding communication skills will be ideally suited to bridge this sometimes giant gap between technology and the business.) A strong leader and a people person, the lead steward will educate and expand many people's horizons about proper data governance and the consequences of unreliable data on business objectives; the steward will make others accountable for continuous improvements in the caliber of data. Lead stewards should be visible senior-level people who are respected and well liked in the organization, with the ability to motivate and envision change from a high level. They should be empowered by senior management and steering committees to directly address issues and manage standards-based implementations from both a business and technology-centric view, brandishing their "data police badge" when staff members resist data standards and the added responsibility or loss of control that come with such regulations. A VP-level title of data steward is not out of the question.

Stewards must have specific and measurable goals for data quality, making sure that public data helps enforce and promulgate vital business rules and processes. A viable formal stewardship policy will be rooted in ongoing standards that identify goals, priorities and quality metrics within all systems infrastructure elements (from data warehouses to OLTP applications) and business functions that touch or affect data. The steward will tie business strategy to data strategy, applying generally accepted qualitative metrics and heuristics (risk management, cost benefit analysis, change management, etc.) to the measuring and enforcement of data quality. With increasingly aggressive life cycle timelines and rapid application development (RAD) paradigms, organizations should not have to keep worrying about the veracity of data from conversion to conversion, re-inventorying and re-reconciling data elements piecemeal every time a new systems integration project is tackled. Without ongoing business driven tools and touchstones for data assessment and improvement, your data stewardship program is doomed to fail. Only a continuous and structured methodology will bring long-term benefits, not a sustained culture of quick data fixes. For larger enterprises, this methodology will have functions that leverage time-honored calculations, derivations curves and benchmarks for measuring quality. The business rules of the corporation should be documented and maintained alongside data models, data dictionaries, meta data repositories, etc., so that a spirit of common responsibility, knowledge and ownership is cultivated. The formal stewardship policy will reside in a public place (corporate intranet) along with other operating policies so that employees who create, use and own the data develop a sensitivity to issues of data rectitude.

If data quality improvements are to continue to be the norm, the impetus must come both through technology architecture and business requirements. It must be understood and evangelized throughout the company that many times business continuity is data continuity: data problems invariably turn into business ones, increasing a firm's exposure to various unanticipated risks. In contrast to purely technical issues, a data steward will sometimes have to address and analyze data problems that are attributable to business routines that have a large human/manual element, and thus cannot be changed easily. The steward will need to have a business understanding and the wherewithal (support of senior management) to carry out the reengineering of various business processes. This may necessitate that they have a cursory understanding of various types of unstructured data (such as that found in electronic document management systems) and data that is not stored digitally, as well as the integration challenges associated with them. In such cases, a steward may have to help engineer a balance between the manual processes - routing paper requests, bar-coding, manual archiving - of a company's information center and records management databases and software.

Under a stewardship program, everybody shares responsibility and accountability for data excellence; information belongs to everybody, and like other (more empirical) corporate resources, it must be scrupulously managed and cared for. The stewards do not own the data; they define and bestow ownership and accountability accordingly, as they train, guide and mentor others in data quality best practices, rewarding compliance and adherence to quality specifications. Everybody that touches data throughout the organization must understand their role in data quality and be able to provide a feedback loop that will help stop bad data habits from propagating throughout the business. There will be pockets of resistance. However, a good lead data steward must know how to deal with the political side of the job. Drawing up a data consumer matrix, which describes who uses what data elements (and what actions are taken with that data), is always an invaluable exercise for assigning data trusteeship. This will help create a culture of data stewardship along all parts of the information life cycle and data supply chain - from the most important reference data to the most transitory of meta data.

In order to accomplish a firm foundation for continuous and measurable integrity of data, stewards should create and foster various partnerships and cross-functional teams (from business and technology) based on pre-approved agreements that govern systems not only inside, but also outside the aegis of enterprise control. Studies have shown that employee data entry errors are responsible for more than 75 percent of bad quality data. However, due to intensified e-commerce activity and broadening reliance on third-party data feeds, potential sources for substandard data are rising exponentially for many businesses. External third-party data sources that will commingle with (or feed) your internal system schemas should meet predefined standards of information quality and management before reaching the sanctity of your organization's gateway. Data suppliers must be held to a service level agreement (SLA) that states their responsibility and interest in providing quality data from a value-added information environment.

There is no one-size-fits-all job description for a data steward; stewardship roles will evolve according to the topology of IT infrastructure/architecture and the principal nature of the business. Care should be taken so that the steward is not saddled with more responsibility than he/she can handle. It will sometimes be good to have a hierarchy of steward roles or spheres of influence, where co-stewards oversee data probity issues along predefined systems or business segment domains in order to make data policy more manageable. Another possibility is to create a data stewardship committee that will share general responsibility for defining quality metrics; assessing acceptable and permissible use patterns of information assets; and assigning levels of stewardship responsibilities accordingly in order to limit data exposure and integration risks across all platforms. The committee will meet periodically to discuss all data governance issues, reporting on gaps in quality control and releasing/publicizing their findings. Items for committee consideration may include the following and beyond:

• Maintenance and storage strategies for corporate reference data.
• Comprehensive standards for data dictionaries, data models, calculation engines, business rules, ETL deltas and more.
• XML strategy: While XML and associated namespaces have been a boon for system-to­system connectivity and communication exchange, standardizing the syntax, format and relationships of information objects -- a semantic consistency that reconciles business data and business-speak ­ will not always be easily achieved. People must know what XML tags mean in the real world business sense, across all of an enterprise's data structures, whether they are financial, manufacturing, etc. The stewardship committee will organize and establish correct linkages for all vital XML elements, artifacts and attributes via a meta data registry (MDR) that will document and define (i.e., capture meanings about) all XML namespaces by storing descriptions of data units and values across systems and data contexts.
• Data privacy and data security policies.
• Web-enabled universal "data shopping" on corporate intranet.
• Auditing of service level agreements to ensure proper data availability.
• Data standards: documented data domains, validations rules, data dependencies; source to target mappings; semantic management, naming conventions, data-typing, etc.
• Replication, backup and recovery policies and procedures.

With the relentless push toward federated-access systems that offer complex real-time (or near real-time) distribution and availability/exposure of data services, data volumes and information proliferation have grown to historically high levels. For many, data integrity problems have become worse than ever, establishing huge roadblocks for the timely delivery and sharing of quality data. An unceasing program of data governance and management, established by carefully selected people, roles, responsibilities and technologies, may be long overdue. Address your issues early (now!), not when you are ready to expose data to share. Address your enterprise-wide data quality issues with respect to accessibility, accuracy, consistency and completeness. Get executive buy-in and start your data stewardship program today! 





---

William Laurent is practice director for Business Intelligence and Corporate Governance at Global Passage LLC. Laurent has a diverse systems background, successfully designing and managing implementation of projects for the insurance, banking, finance, government, technology and entertainment industries.  He would enjoy receiving your comments at wlaurent@gpassllc.com.

Copyright 2007, SourceMedia and DM Review.

RM 101

Why Records Management?

by Priscilla Emery
30-May-2005 --<script></script> Comments -- none yet<script src="http://www.intensedebate.com/js/getCommentLink.php?acct=4f391d025c7a274f56ab1bba3cf65df1&postid=http://www.cmswatch.com/Feature/127-RM-101&posturl=http://www.cmswatch.com/Feature/127-RM-101&posttitle=" type="text/javascript"></script><script src="http://www.intensedebate.com/js/genericLinkWrapperV2.js" type="text/javascript"></script>



 

Ed.'s Note: This article is excerpted from the Records Management Report, published by CMS Watch.

To some, managing records represents one of the most boring and onerous business functions that anyone could possibly undertake within an organization. Of course, most people don’t even understand what records management is -- making it easy to malign an activity that is so misunderstood.

Indeed records management crosses numerous disciplines. Did you know…

• That categorization and indexing are two elements that are critical to the success of a records management program? Sounds a little like knowledge management.
• That vital records preservation is one of the key steps in developing a disaster recovery plan? Sounds like infrastructure management.
• That by ignoring records management policies employees and their companies can potentially end up facing criminal penalties? Sounds like a legal profession.
• What led to Arthur Andersen’s downfall? Shredding — inappropriate shredding — shredding of records that should have been retained according to the policies of both Andersen and Enron.
• Does the information document your daily business process?
• Does the information provide input to a mission-critical business decision?
• Does the information provide evidence as to whcy a business decision was made?
• Is the information required for legal, fiscal, audit, or tax purposes?
• manage retention schedules,
• store records in their appropriate classifications along with the prescribed metadata,
• search for records when required,
• destroy records according to prescribed schedules,
• put holds on records so they don’t get destroyed while the company goes through an audit or investigation, and
• track the access of the record over its active and in-active life cycle.
• This blog post
• All blog posts
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• Logged in as
• No anonymity - you must indicate your name and affiliation in the text of your comment
• Disagreements welcome, but no marketing by vendors or other firms, and please mind your manners
• Our critical analysis exposes product weaknesses as well as strengths
• We deliver unrivaled technical depth and comprehensive project advice
• Our research is led by international topic experts
• We only work for buyers -- never for vendors

 

<!-- goal -->

Contact us

CMS Watch

info@cmswatch.com

3470 Olney-Laytonsville Road Suite 131

Olney, MD USA 20832

1 800 325 6190 (customer service)

+1 617 763 5336 (int'l customer service)

Fax: +1 214 242 3048

<!-- info -->


---

<!-- ****************************************************************** --><!-- * FOOT AREA * --><!-- ****************************************************************** -->
<!-- bnav -->

<!-- noindex -->Mail This Page<!-- index --> | About | Feedback | Customer Support | Free Report Samples
Copyright © 2001-2009 by CMSWorks, Inc.   Terms of Use | Privacy Policy

<!-- #### END: FOOT #### -->
<!-- bnav --><!-- sbot --><!-- sbox -->

Policy retention still is an issue for many businesses, and electronic
technology has made large-scale document storage easier. However, the advent of
claims-made policies has reduced the value of expired liability policies.