Archive

Archive for January, 2010

SSIS package to Remove Duplicates from any Database

January 19th, 2010 Arun Mallick No comments


Here comes one more challenging yet interesting topic to tide over. The requirement goes something like this: You have some sources, let it be some sales data or some Call center data coming from different sources. Data can be of different media. But you are able to load it in your staging tables i.e. one staging database you are maintaining like StageDB for storing these incremental data.

So, here we have both master i.e. Dimensional data and Detail i.e. Fact data in our stage database. But before processing further and loading it into our Mart or Data warehouse, we need to check if there is any redundancy at the row level for each of these tables in the staging database. Here comes the main problem, what if you don’t know how many tables are there in the staging database and how many columns are there for each table in the staging database but still you have to keep only unique records for each table.

Read more…

Categories: SSIS

ROLLUP with CURSOR for grouping

January 3rd, 2010 Arun Mallick No comments


Problem: I have a table for customer information with Customer Ids and Customer Type. The customer may fall into different types i.e. the customer may belong to Type A or Type B or may belong to both the groups. The requirement is to find out the count of the customers as per the type in each group and if the customer falls under more than one group, it must not be counted under the individual group rather it should be counted under the group with both the types.

Read more…

Categories: T - SQL