Change to DimensionElementComponentAdd in PA vs 10.2?

Post Reply
lotsaram
MVP
Posts: 3251
Joined: Fri Mar 13, 2009 11:14 am
OLAP Product: TableManager1
Version: PA 2.0.x
Excel Version: Office 365
Location: Switzerland

Change to DimensionElementComponentAdd in PA vs 10.2?

Post by lotsaram » Mon Jun 24, 2019 2:45 pm

In the latest TM1 server version 11.5.00000.23 we have detected a change in the behaviour of the TurboIntegrator function DimensionElementComponentAdd. In previous versions if the element supplied as the component to DimensionElementComponentAdd did not already exist in the dimension then it would be automatically added as a leaf element without need for explicitly using the DimensionElementInsert function to create the child element first.

This new stricter behaviour may cause any previously error free (and productive) processes to throw errors and no longer work.

This change might have come in an earlier 2.0.x and not in 2.0.7. Is anyone able to confirm this behaviour and which 11.x version it appeared?

Many people may have exploited the more lenient behaviour of DimensionElementComponentAdd so although dead easy to patch this is definitely something to be aware of.
Please place all requests for help in a public thread. I will not answer PMs requesting assistance.

Wim Gielis
MVP
Posts: 2089
Joined: Mon Dec 29, 2008 6:26 pm
OLAP Product: TM1
Version: PAL 2.0.7
Excel Version: Office 365 - latest
Location: Brussels, Belgium
Contact:

Re: Change to DimensionElementComponentAdd in PA vs 10.2?

Post by Wim Gielis » Mon Jun 24, 2019 8:04 pm

I haven’t used that functionality that much in the past, I always specify 2 DimensionElementInsert and 1 DimensionElementComponentAdd functions. I can’t help with exact versions.

On a related note if I may. What I found out in the not so distant past is that if DimensionElementComponentAdd is used to add a child to a leaf element (which should change the leaf to consolidated), we get an error. I was almost sure this was not the case in the more distant past - but I’m not100% sure.
Best regards,

Wim Gielis

Excel Most Valuable Professional, 2011-2014
http://www.wimgielis.com ==> 109 TM1 articles and a lot of custom code
Newest blog article: Set up a TM1 service with AutoHotKey

lotsaram
MVP
Posts: 3251
Joined: Fri Mar 13, 2009 11:14 am
OLAP Product: TableManager1
Version: PA 2.0.x
Excel Version: Office 365
Location: Switzerland

Re: Change to DimensionElementComponentAdd in PA vs 10.2?

Post by lotsaram » Mon Jun 24, 2019 8:34 pm

Wim Gielis wrote:
Mon Jun 24, 2019 8:04 pm
if DimensionElementComponentAdd is used to add a child to a leaf element (which should change the leaf to consolidated), we get an error. I was almost sure this was not the case in the more distant past
Yes definitely that used to work. (Not actually a good thing given that converting elements from N to C results in data loss! Good that this has changed). My guess is that that this change was necessary as leaf elements are shared across all hierarchies and name conflict between leaves and consolidations in hierarchies is prohibited. Therefore as the element exists in the Leaves hierarchy it isn't possible to create a consolidation with the same name in another hierarchy. (BTW this also means that we need to be much more careful than before about specifying the correct element type on creation than the old days.)
Please place all requests for help in a public thread. I will not answer PMs requesting assistance.

Wim Gielis
MVP
Posts: 2089
Joined: Mon Dec 29, 2008 6:26 pm
OLAP Product: TM1
Version: PAL 2.0.7
Excel Version: Office 365 - latest
Location: Brussels, Belgium
Contact:

Re: Change to DimensionElementComponentAdd in PA vs 10.2?

Post by Wim Gielis » Tue Jun 25, 2019 6:05 am

Thanks Lotsaram. The problem is for example a classic parent-child hierarchy structure. A is the parent of B (line 1) while B is the parent of C (line 2). But line 1 in the data source does not know about line 2. What should we do then ? Pass over the data source twice and use a temporary dimension to store the information of all records ?
Best regards,

Wim Gielis

Excel Most Valuable Professional, 2011-2014
http://www.wimgielis.com ==> 109 TM1 articles and a lot of custom code
Newest blog article: Set up a TM1 service with AutoHotKey

User avatar
Steve Rowe
Site Admin
Posts: 1922
Joined: Wed May 14, 2008 4:25 pm
OLAP Product: TM1
Version: 10.2.2., PAW
Excel Version: Nearly all of them

Re: Change to DimensionElementComponentAdd in PA vs 10.2?

Post by Steve Rowe » Wed Jun 26, 2019 2:19 pm

The second new behaviour, being unable to add an a child to an N level and so protecting the application from data loss is an "interesting" change. Whilst for many use cases it is sensible it makes other use cases much more challenging.

In our code base where we want to protect the environment from this we perform a test before the ComponentAdd and in our dimension maintenance tool an elements type is declared at the point of creation.

IBM really shouldn't be making this changes in such a hard way though, as Wim points out there is a very common use case where this change will break peoples existing code base. This is where you don't know if an element is a C or an N until you come to build the relationship that is defined in the data source. Typically this would be from a relational source. There is probably a lot of legacy code that is now going to start failing.

Rather than just changing the behaviour (and, I assume, not announcing it) it would be far better to add an optional 1/0 parameter to the end of the function. 0 being the default protective behaviour and 1 being the old behaviour. This would allow people to easily update their legacy code as required. If this come about as the results of a complaint / RFE they should have just been told to write their code properly and check for an elements type before performing the operation.

Now we need to know an elements type the first time we encounter it, this implies that we need to analyse the whole structure before creating it and so, (as Wim states) we need to process it twice.

Lotsaram, Re the point about hierarchies, this probably makes sense but there is a specific function for adding relationships to hierarchies, this is the one that needs this behaviour. Not the one that is for non-hierarchy legacy builds. It has been stated by IBM many times that the hierarchy functionality required no code changes until you activate it against a specific dimension. This would no longer seem to be the case.

Maybe the new behaviour only arise if hierarchies are allowed in the cfg? (I don't have a testbed to hand to check, sorry)

lotsaram
MVP
Posts: 3251
Joined: Fri Mar 13, 2009 11:14 am
OLAP Product: TableManager1
Version: PA 2.0.x
Excel Version: Office 365
Location: Switzerland

Re: Change to DimensionElementComponentAdd in PA vs 10.2?

Post by lotsaram » Wed Jun 26, 2019 4:19 pm

The first one (automatic insert of element with implicit type N where component not known) appears to be a regression bug. It was reported in 2.0.1 and subsequently fixed, so my guess is this will also be fixed again in 2.0.7.

The 2nd case, as far as I know will only happen in the case of dimensions which contain hierarchies. For dimensions containing only the same names hierarchy and no Leaves hierarchy it should work just as before (but once a dimension has multiple hierarchies it's a no-go for the reasons explained).
Please place all requests for help in a public thread. I will not answer PMs requesting assistance.

Wim Gielis
MVP
Posts: 2089
Joined: Mon Dec 29, 2008 6:26 pm
OLAP Product: TM1
Version: PAL 2.0.7
Excel Version: Office 365 - latest
Location: Brussels, Belgium
Contact:

Re: Change to DimensionElementComponentAdd in PA vs 10.2?

Post by Wim Gielis » Wed Jun 26, 2019 10:55 pm

lotsaram wrote:
Wed Jun 26, 2019 4:19 pm
The 2nd case, as far as I know will only happen in the case of dimensions which contain hierarchies. For dimensions containing only the same names hierarchy and no Leaves hierarchy it should work just as before (but once a dimension has multiple hierarchies it's a no-go for the reasons explained).
I tested this in 2.0.6 IF3.

If the dimension only contains the same-named hierarchy and a Leaves hierarchy, it works fine.
Once the dimension contains any "custom" hierarchy, it throws an error.

It does not matter whether EnableNewHierarchyCreation=T or F.
Best regards,

Wim Gielis

Excel Most Valuable Professional, 2011-2014
http://www.wimgielis.com ==> 109 TM1 articles and a lot of custom code
Newest blog article: Set up a TM1 service with AutoHotKey

User avatar
Steve Rowe
Site Admin
Posts: 1922
Joined: Wed May 14, 2008 4:25 pm
OLAP Product: TM1
Version: 10.2.2., PAW
Excel Version: Nearly all of them

Re: Change to DimensionElementComponentAdd in PA vs 10.2?

Post by Steve Rowe » Thu Jun 27, 2019 7:19 am

Thanks for testing, so legacy code will continue to work until hierarchies exist versus the dimension.

Good

Wim Gielis
MVP
Posts: 2089
Joined: Mon Dec 29, 2008 6:26 pm
OLAP Product: TM1
Version: PAL 2.0.7
Excel Version: Office 365 - latest
Location: Brussels, Belgium
Contact:

Re: Change to DimensionElementComponentAdd in PA vs 10.2?

Post by Wim Gielis » Thu Jun 27, 2019 7:41 am

As Dimsiz of a non-existing dimension returns 0 rather than an error (*),
I'm thinking of testing whether the Dimsiz of 'Hierarchies_' | vDim exceeds 2 or not.
vDim is the dimension to be investigated.
If > 2, it will involve additional logic.

What do others think ?

(*) at least for the time being ;-)
Best regards,

Wim Gielis

Excel Most Valuable Professional, 2011-2014
http://www.wimgielis.com ==> 109 TM1 articles and a lot of custom code
Newest blog article: Set up a TM1 service with AutoHotKey

lotsaram
MVP
Posts: 3251
Joined: Fri Mar 13, 2009 11:14 am
OLAP Product: TableManager1
Version: PA 2.0.x
Excel Version: Office 365
Location: Switzerland

Re: Change to DimensionElementComponentAdd in PA vs 10.2?

Post by lotsaram » Thu Jun 27, 2019 9:26 am

Wim Gielis wrote:
Thu Jun 27, 2019 7:41 am
I'm thinking of testing whether the Dimsiz of 'Hierarchies_' | vDim exceeds 2 or not.
vDim is the dimension to be investigated.
If > 2, it will involve additional logic.

What do others think ?
I would create a temporary dynamic subset with MDX

Code: Select all

Expand('{TM1FILTERBYPATTERN( {TM1SUBSETALL( [}Dimensions] )}, "%vDim%:*")}')
and then test that the subset has >= 2 elements (not >=1 as Leaves will always exist if there are other hierarchies and even if all other hierarchies are deleted will remain and will be automatically removed on the next session start.)

I would do this rather than checking the }Hierarchies_ dimension as just say you have a custom "control dimension" (starting with "}") then although you can create hierarchies there will be no }Hierarchies_ dimension created.
Please place all requests for help in a public thread. I will not answer PMs requesting assistance.

User avatar
PavoGa
Community Contributor
Posts: 337
Joined: Thu Apr 18, 2013 6:59 pm
OLAP Product: TM1
Version: 10.2.2 FP7, PA2.0.05
Excel Version: 2013 PAW
Location: Cleveland, Tennessee

Re: Change to DimensionElementComponentAdd in PA vs 10.2?

Post by PavoGa » Thu Jun 27, 2019 2:48 pm

lotsaram wrote:
Mon Jun 24, 2019 2:45 pm

This change might have come in an earlier 2.0.x and not in 2.0.7. Is anyone able to confirm this behaviour and which 11.x version it appeared?

Many people may have exploited the more lenient behaviour of DimensionElementComponentAdd so although dead easy to patch this is definitely something to be aware of.
Ran headlong into this problem after moving to PA. Initially thought it was just the Hierarchy functions, but soon found out otherwise. Actually thought I'd posted about it, but could not find anything on it, so dropped the ball on that from months ago.
Wim Gielis wrote:
Mon Jun 24, 2019 8:04 pm
On a related note if I may. What I found out in the not so distant past is that if DimensionElementComponentAdd is used to add a child to a leaf element (which should change the leaf to consolidated), we get an error. I was almost sure this was not the case in the more distant past - but I’m not100% sure.
Have not run into this, or if I did, have managed to avoid it since. :|
Ty
Cleveland, TN

Post Reply