TI Process gives duplicates into asciioutput

[mincharug.shulft]

Hi All Gentle mans ,its pleasure to write an email bcz i see, well prompt responses each others...really i say thank to this forum....

I have a problem in TI Process:

I have users data want to store into text file,so ,i am writing TI Process script using asciioutput function into text file,but the information (users ids) that i am storing into text file has many duplicates bcz of my source.i donot want to load or store it in my text file all my duplicate users information,is there any way to check whether already same user exist in the text file or not before load it bcz of this i am not able to open the text file in excel to remove the duplicates since it stored in more than 500 MB....really appreciated your prompt response.

[Wim Gielis]

How many such different ID's do you have ?
Because you could use a temporary dimension, do a DIMENSIONELEMENTINSERT, and then check (DIMIX = 0 or not ) if the element (ID) in the temporary dimension or not.
In the Epilog you remove the temporary dimension again.

[TrevorGoss]

How many such different ID's do you have ?
Because you could use a temporary dimension, do a DIMENSIONELEMENTINSERT, and then check (DIMIX = 0 or not ) if the element (ID) in the temporary dimension or not.
In the Epilog you remove the temporary dimension again.

A temporary dimension is a good way to achieve this, another way would be to have a string variable and attach the IDs to the string and then do an if condition to check if the ID exists within the string

[Wim Gielis]

A temporary dimension is a good way to achieve this, another way would be to have a string variable and attach the IDs to the string and then do an if condition to check if the ID exists within the string

For small numbers of IDs I could do your approach, but I doubt this is the case here if the output file is 500 MB.

[mincharug.shulft]

Thanks a lot for your prompt reply Wim Gielis and TrevorGoss .

My source is my transaction logs converting into text file and using that a source.
from my transaction logs (.txt) i need to get users ids...so still i can create temp dim.....Please help on this.Thanks in advance

[mincharug.shulft]

How many such different ID's do you have ?
Because you could use a temporary dimension, do a DIMENSIONELEMENTINSERT, and then check (DIMIX = 0 or not ) if the element (ID) in the temporary dimension or not.
In the Epilog you remove the temporary dimension again.

A temporary dimension is a good way to achieve this, another way would be to have a string variable and attach the IDs to the string and then do an if condition to check if the ID exists within the string

Thanks a lot for your prompt reply Wim Gielis and TrevorGoss .

My source is my transaction logs converting into text file and using that a source.
from my transaction logs (.txt) i need to get users ids...so still i can create temp dim.....Please help on this.Thanks in advance

[Wim Gielis]

so still i can create temp dim

Yes, correct.