Expert: Chuck Cosby - 10/14/2008

Question
Hi,

I am working on a few databases which have large amounts of data with some of them wrong. I want to develop a tool or use an existing tool which would help me analyse and correct the data with respect to a certain pattern. for eg an attribute might have values of between 1 to 20 normally but suddenly it has an abnormal value of 56 which has to be detected and notified or modified.

Any ideas

Regards Thiru

Answer
There are such tools in the 'Data Warehousing' technology areana.  They are called 'ETL' for 'Extract, Transform, and Load.  Extract, as in extract data from from a production database.  Transform, as in fix the problems with the data, and Load, as in Load the data into the data warehouse.  You are interested in the 'Transform' aspect.   But i suspect the entire data warehousing concept might be of interest to you.  do an internet search on 'data warehouse'  or 'ETL'