In hash user keys partitioning, the Integration Service uses a hash function to
group rows of data among partitions based on a user-defined partition key. You
choose the ports that define the partition key:
SQ->FIL->SORT-> AGG->TGT
SQ->FIL->SORT-> AGG->TGT
When you specify hash auto-keys partitioning in the preceding mapping, the
Sorter transformation receives rows of data grouped by the sort key, such as
ITEM_DESC. If the item description is long, and you know that each item has a
unique ID number, you can specify hash user keys partitioning at the Sorter
transformation and select ITEM_ID as the hash key. This might improve the
performance of the session since the hash function usually processes numerical
data more quickly than string data.
If you select
hash user keys partitioning at any partition point, you must specify a hash key.
The Integration Service uses the hash key to distribute rows to the appropriate
partition according to group.
For example, if
you specify key range partitioning at a Source Qualifier transformation, the
Integration Service uses the key and ranges to create the WHERE clause when it
selects data from the source. Therefore, you can have the Integration Service
pass all rows that contain customer IDs less than 135000 to one partition and
all rows that contain customer IDs greater than or equal to 135000 to another
partition.
If you specify
hash user keys partitioning at a transformation, the Integration Service uses
the key to group data based on the ports you select as the key. For example, if
you specify ITEM_DESC as the hash key, the Integration Service distributes data
so that all rows that contain items with the same description go to the same
partition.
To specify the
hash key, select the partition point on the Partitions view of the Mapping tab,
and click Edit Keys. This displays the Edit Partition Key dialog box. The
Available Ports list displays the connected input and input/output ports in the
transformation. To specify the hash key, select one or more ports from this
list, and then click Add.
Hi.. I have a doubt on Pass through partition.
ReplyDeleteSQ-->EXP-->TGT
I have SQL Override statement in SQ. And I configured SQ as partition point with 2 Pass through partitions. So do I need to write separate SQL Override statements with separate where conditions per each partition?
Plz help. THANK YOU!