Business Analyst Handbook

Metadata will rock your world

Today, most places you look, metadata is scarce.  Try to find definitions within most corporate databases.  Ask to see the data dictionary.

When you find that isolated corporate application built with a data dictionary, other applications in the same environment duplicate the data, and have no metadata.

These conditions are true at every scale.  Small businesses have less need and fewer resources to invest in organizing their data.  They find it cheaper and faster to build scarcely-defined data buckets, lacking constraints.  As these businesses grow, acquire other companies or merge into major corporations, their legacy applications still work and so remain isolated.  Within most enterprise architectures, undefined, unconstrained, duplicate data limits knowledge available for analysis, while constraining daily operations.

Metadata has only begun to surface within and between businesses.

It surfaced in 1996 when two Sanford students were convinced that the knowledge within the internet could be better accessed.  They devised a scheme for cataloging and searching it.  This led to Google.

At the user interface, Google is a rectangular box and a push button.  Type anything into the box, and Google searches its metadata and retrieves web pages for you, really fast.  By providing this free service, Google runs a 21 billion dollar advertising business.

The Google rectangular box holds nothing but metadata.  Their metadata is the largest information catalog that ever existed.  It tells what information exists on which web pages.  It keeps track of how the information is accessed, and from where.

Look near the top of any page on Amazon.com, and you will find a similar rectangular box.  Step back from your computer, and drive through any city looking for the vast warehouse that holds every Amazon.com book, DVD, groceries, toys, appliances, clothing, and office supplies.  There is no such warehouse.  Amazon.com uses metadata to sell merchandise, quickly, abundantly, from warehouses worldwide.  Because they profit from using metadata, they don't mind selling from warehouses of other companies.  In fact, they encourage their customers to re-sell books after reading them, along with outgrown toys and anything else with market value.

Wikipedia.com has a rectangular box.  When you type just about any noun (person, place or thing) into that box, Wikipedia retrieves an article.  Last year, it provided about 2.5 billion page views per month, governed by a non-profit organization with about 7 million dollars revenue.

Where is the rectangular box in your organization?  Where do your leaders find accurate strategic information as quickly as Google finds 9,500 yackadoos? - as conveniently as Amazon.com finds 184 varieties of potato peeler? - and as thoroughly as Wikipedia digs up Machu Picchu?

Most organizations - large and small - lack the metadata to power such a rectangular box.  When its time to define a "customer", someone writes a definition and that information is distributed to a few departments, read and understood by only a few employees.  When its time to build a new application, the wheel is re-invented, a new silo is constructed and the metadata is left for later.

This will change.  As the pioneering Google, Amazon.com and Wikipedia have abundantly shown us, there is gold in constructing a means of organizing corporate data.  One day, this will be the corporate norm.  The standard.  Just as little bookstores with jumbled shelves have been replaced by big bookstores with workstations where any book can be found, counted, priced, sourced and described (that's all metadata)... I.T. shops lacking that kind of organization will one day acquire it, or will be acquired.

One day, a common taxonomy will link I.T. shops across lines of business and across international boundaries.  The well-paid wheel re-inventors and silo constructors will have moved on to other work.

Business users in cubicles, conference room and corporate offices will have that little rectangular box which holds all the information that they need to make their organizations prosper.  Metadata will have reshaped their world.

Feel the rumbling underfoot, just a little?

It's all about the data

A wise Manager once told me... actually told me several times that "It's all about the data."  He had been in corporate Information Technology for 20-some years and now led a team of DBA's and Data Modelers, so may not have been the most objective source.

As Business Analysts, we bridge the sometimes scary gulf between a customer and a development team.  I say sometimes, thinking of those occasions when the developers were all too eager to code something, anything, when the customers had just begun to think about the problem.

Sometimes, it seems like a developer may have a hidden agenda; some program he or she has wanted to code and is just waiting for a budget to write that thing.  That's the BA perspective; I've heard the developers express frustration over waiting weeks for documentation... and in the meantime having no available charge codes... and then receiving wildly inaccurate specifications.  After developing to those, it was of course up to them to implement the necessary design changes on their own time and budget.

We Business Analysts can make things better for both groups we serve by getting the requirements correctly and quickly.  One key factor for accomplishing that is understanding that requirements are more than function lists.  We sometimes have to tweeze out business rules... even recommend that the application allow for evolving changes.  And we certainly have to get a handle on the data.

A Data Architect or Data Modeler may join us on the team.  That's fine, as long as we can stay in tandem with them.  Many have highly refined skills in drawing third-normal-form diagrams, then adroitly changing them into physical databases. 

When we work with them, our responsibility as BA is simply to be sure they have opportunity to draw out all the necessary nouns from our mutual clients.  Those nouns become entities and attributes, and later become tables and columns in the database supporting the technology solution our clients seek.

If you know a bit about third normal form and data modeling, you might want to draw a preliminary Conceptual Data Model (CDM) or more detailed Logical Data Model (LDM). If you have license to such diagramming powerhouses as Embarcadero ER/Studio or CA ERWin, you can quickly draw an impressive Entity Relationship Diagram (ERD).  If you really want to be helpful, include definitions, relationship names and data types with those diagrams.  Otherwise, your partnering Data Architect may cry, "Where's the metadata?"

I am biased towards ER/Studio because it draws beautiful diagrams so easily and even will untangle all the relationship lines for you with one click.  ERWin now does that, but with far fewer options and not so elegantly.

Recently, I needed a data modeling tool for another machine and another client and had no budget.  Fortunately, Dan Horn had posted a very handy chart online, listing 65 products, each with a link to the vendor website, a price quote and comments. Prices range from free to $14,000 with many in the $100 - $200 range. 

After looking through that chart, I settled on Power Architect.  It wasn't as easy to learn and use as the $2K+ enterprise tools I had used before, but it did the job.  It did at least the part of the job where the model was drawn and then forward engineered into a database.  The other part of the job was meetings with the client to review iteration after iteration of the paper diagram until I really had drawn the data elements they needed. 

The work was all very much in line with what a BA must do.  And in that case, it was all about the data.

Rules not made to be broken

Business Rules were made to change!

When was the last time you worked for or consulted at a company with stable business rules?  If such a company existed, no one would ever transfer to another department.  There would be no promotions (actually, I have heard that a few times).  The company would always use the same suppliers, and they would have the same customers - year after year - with their needs and interests never changing.

Consider these examples:

• Ann Jones is responsible for the I.T. department (governance)
• The database must be backed up between 2:00 and 3:00 a.m., 7 days each week.  (systems)
• State and county sales tax of 6.5 percent is to be collected for all merchandise shipped to Florida addresses.  (calculations)

But people move from job to job, and software is constantly upgraded, and it has been a while since tax laws were carved on stone tablets.

This volatility means two things to the Business Analyst.  First - that beautiful requirements document with all of the detail you so faithfully captured will be out of date soon after the final signature is scribbled on the last page.  In fact, the signature page is probably out of date as well.

If you must produce a document, produce it from a repository.  Select the always-volatile rules from some form of database... even a spreadsheet... and if possible distribute it via an intranet rather than killing yet another tree.

If volatility causes the need for a repository, then it also brings the need for organization.  If the repository is not organized, you won't be able to search and update and extract business rules.  If it is poorly organized, the business rules will not trace to data and functional requirements.

The suggestion is, then, record your business rules as data, and organize them using some sort of taxonomy.  Maybe develop a hierarchy based on the company organization, or based on your enterprise data model, or simply a hierarchy of tasks.  Regardless of the form of organization, find a way to link business rules to functional requirements and to data elements.  The home page of my web site, gpond.com kind of illustrates this.