DimensionDeleteAllElements Issues

[Steve Vincent]

has anyone found in 10.2.2 that DimensionDeleteAllElements no longer works when used as part of a single TI to rebuild a dim?

On it's own it works just fine, but when used as part of a rebuild it's doing nothing. It means if a structure is altered its leaving the old parts around as well as building the new bits via metadata. The result is a screwed dim, duplicates in consolidations etc. Just wanted to know if anyone else had seen that issue and if it's already logged with IBM. It's causing me a major headache and i'm trying to find a way around it.

[lotsaram]

What else is happening on the prolog? Specifically are any other processes being called that do any edits to the dimension?

Each TI process has its own shadow dimension copy and commits independently so if combining TI processes for metadata updates it's quite a different proposition to data updates and you need to be very careful to avoid "last save wins" type scenario.

[Steve Vincent]

its a complicated chain in total (replicates structures / data across multiple servers) but the TI itself is the only thing making the dim changes and calls no others. The code has been in place & working flawlessly for years but we've recently moved from 9.5.2 to 10.2.2 which is why i think the function has been screwed up in this release

[Steve Vincent]

Seems like i've been caught out by the change to single/multiple commit of chores.

Previously everything run in the way single commit is supposed to behave, chores as well as TI. But 10.2 defaults to multi commit which screws up anyone trying to chain TIs together when some rely upon the actions of others, such as rebuilding dimensions. Despite being able to change that in a chore, all TIs now seem to behave in a multi commit manner, hence my once perfectly working replication procedure is now stuffed.

I'm being tripped up where structures are changed and start to clash with aliases. It's supposed to empty the attributes cube before rebuilding the dim, but the TI calling this is behaving as multi commit so TM1 still thinks the cube is populated, so fails on update due to alias clash. Is there any server setting that can control this mode too?

[rmackenzie]

Perhaps try changing the option in the chore dialog?

[Steve Vincent]

first thing i tried, doesn't help when TIs are called from other TIs

[BrianL]

In this case multiple commit mode is a red herring. Multiple commit mode ONLY applies to chores. It inserts a commit between each TI that is explicitly defined as belonging to the chore. It has no impact/relevance to processes called by ExecuteProcess. They still run in the parent TI context.

Also, multiple commit mode still defaults to off. Even if the default had changed, TM1 should not be editing your existing chores (to change value) during an upgrade.

Further, all multiple commit mode does is make your changes public to ALL users at each commit point instead of keeping the changes local to the chore/TI that's running until the entire thing finishes running. It should not effect the execution semantics at all.

Seems like you've already narrowed this down to the attributes cube not being empty. What's your code that's supposed to do that?

[Michel Zijlema]

Do you have a DimensionSortOrder statement directly before the DimensionDeleteAllElements statement? Than you're probably caught by this bug...
In that case reversing the order of the statements would solve the issue.

Michel

[lotsaram]

Actually ironically I ran into a bug to do with DimensionDeleteAllElements on Friday with 10.2.1

The situation was if DimensionDeleteAllElements is called and then subsequently there are no further metadata actions in the process then the DimensionDeleteAllElements has no effect. (Say for example the data source is empty so non code from the metadata or data tabs is executed). I would have expected the dimension to be empty in such a scenario.

Can anyone else confirm whether this is "expected behaviour" and therefore according to IBM a "feature" vis a vis a "bug"? IMO if the function doesn't do what it says on the can then it's a bug but I hardly ever use DimensionDeleteAll elements and normally prefer to use an unwind approach, but this was for a temp kind of dimension so delete all elements was appropriate.

[Steve Vincent]

In this case multiple commit mode is a red herring. Multiple commit mode ONLY applies to chores. It inserts a commit between each TI that is explicitly defined as belonging to the chore. It has no impact/relevance to processes called by ExecuteProcess. They still run in the parent TI context.

Also, multiple commit mode still defaults to off. Even if the default had changed, TM1 should not be editing your existing chores (to change value) during an upgrade.

Further, all multiple commit mode does is make your changes public to ALL users at each commit point instead of keeping the changes local to the chore/TI that's running until the entire thing finishes running. It should not effect the execution semantics at all.

Seems like you've already narrowed this down to the attributes cube not being empty. What's your code that's supposed to do that?

I agree, what you state should be true but so far it's not what i'm seeing when running processes that have been solid & reliable for at least 4 years now.

DATASOURCE
a cma file of the dimension structure in standard TM1 format

PROLOG

Code: Select all

### SET THE DATASOURCE ###
sFilePath = CELLGETS ( '_Replication' , '_Replication' , 'Rep Filepath' );
sFile = sFilepath | 'DIM_' | DimName | '.cma' ;

DatasourceNameForServer = sFile;
DatasourceNameForClient = sFile;

Cube = '}ElementAttributes_' | DimName;
DimensionDeleteAllElements ( DimName );

### ONLY RUN IF ATTRIBUTES ALREADY EXIST ###
IF ( CubeExists ( Cube ) = 1);
    CubeSetLogChanges(Cube, 0);

### CLEAR ALL ATTRIBUTES TO AVOID ALIAS CONFUSION WHEN REBUILDING STRUCTURE ###
  CUBECLEARDATA ( Cube );

ENDIF;

METADATA

Code: Select all

ElementExists = DIMIX ( DimName , Element );
TopLevel = ELPARN ( DimName , Element );

IF ( type @= 'C' & TopLevel = 0 );
    DimensionElementInsert ( DimName , '' , Element , 'C' );
    Parent = Element;
ELSEIF ( type @= 'N' & TopLevel = 0 );
    DimensionElementInsert ( DimName , '' , Element , 'N' );
ELSEIF ( type @= 'S' & TopLevel = 0 );
    DimensionElementInsert ( DimName , '' , Element , 'S' );
ELSEIF ( type @= 'C' & ElementExists >0 );
    Parent = Element;
ELSEIF ( type @= '' );
    DimensionElementComponentAdd ( DimName , Parent , Element , Weight );
ENDIF;

DATA
Empty


EPILOG

Code: Select all

DimensionSortOrder ( DimName , 'ByName' , 'Ascending' , 'ByHierarchy'  , 'Ascending' );

### ONLY RUN IF ATTRIBUTES ALREADY EXIST ###
IF ( CubeExists ( Cube ) = 1);
    CubeSetLogChanges(Cube, 1);
ENDIF;

DimensionDeleteAllElements is not being "commited" at this stage, so when completing the dim TM1 is being confused and merging the old dim with the rebuilt one.

Our version of TM1 is actually bespoke - a hotfix was provided to us to clear a specific issue that was stopping us deploying to the customer. I have a sinking feeling that some of these problems might be caused by that "fix", especially if no one else is seeing the same results as i am.

[Steve Vincent]

Actually ironically I ran into a bug to do with DimensionDeleteAllElements on Friday with 10.2.1

The situation was if DimensionDeleteAllElements is called and then subsequently there are no further metadata actions in the process then the DimensionDeleteAllElements has no effect. (Say for example the data source is empty so non code from the metadata or data tabs is executed). I would have expected the dimension to be empty in such a scenario.

Can anyone else confirm whether this is "expected behaviour" and therefore according to IBM a "feature" vis a vis a "bug"? IMO if the function doesn't do what it says on the can then it's a bug but I hardly ever use DimensionDeleteAll elements and normally prefer to use an unwind approach, but this was for a temp kind of dimension so delete all elements was appropriate.

That is a bug - as part of the analysis on my issue i ran a TI with nothing but DimensionDeleteAllElements in it and it cleared the entire dim as expected.