PMR68125 999 866: TM1 952 TI CubeView Incorrect Order

[Steve Vincent]

Easy one to replicate this. Was a problem way back in 9.0, seemed to be fixed around 9.1.x, am sure it was OK in 9.5.1 but is definately screwed in 9.5.2

When a TI uses a cube view as a datasource, if the cube has been reordered from its original build order the TI reads the variables in the REORDRED sequence not the build sequence. This goes against every single formula, function and code in TM1 that uses the build order at all times (TABDIM/DBRW/DBS etc). The issue only arises when a TI builds the view - manually created views remain in the correct build order.

Attached is a very simple example to illustrate it. 2 cubes built with 4 dims (A/B/C/D) in the same order. ORIGINAL is untouched, REORDER has, err, been reordered! (D/C/B/A). I created views called MANUAL in each cube by using the data export wizard. The TIs ORIGINAL - MANUAL VIEW & REORDER - MANUAL VIEW use these views as their datasource and you will see in both cases the order in the datasource tab is A B C D. The other TIs create the views called TI CREATED in each cube. When you check the datasource in these, one is A B C D and the other is D C B A.

This is bad because a lot of developers like to write models that create views & subsets, do great things then delete the views & subsets afterwards. It keeps models tidy, easy to manage and simplifies the interface for users. If a cube gets reordered after these TIs are written or even have the reorder changed, the TIs will fail. All of them will need to be recreated to deal with the reordered cube, and it just should not be this way.

[lotsaram]

This is a big issue with reordered/optimized cubes. I'm glad that someone else has loged this as I have already logged it (twice) in 9.4.1 FP3 and 9.5.1 only to face considerable problems convincing support that it was an issue as first they weren't able to replicate is without a huge effort on my part to spoon feed and then second they just didn't get that this was an issue, in fact I'm not sure it ever made it's way to engineering.

Support tech: "OK so I can see that the variables are not in the same order as the dimension list for the cube but the process I just wrote with the wizard to extract data out of the cube still works, so what is the issue?"

Me: "But if you have a generic process that works on any cube then the process will fail, maybe not on export of data but definately on import of data as the order that the cell address of each piece of cell data was exported out will not match the order of the dimensions in the cube and so any CellPutN statements dealing with that data will fail."

Support tech: "Sorry I don't get it. how is this an issue again?"

And so on and so on until I think I gave up in order to preserve my sanity.

[Steve Vincent]

not off to a good start - reply from support requested screenshots and error messages. Erm, i've already provided a model and instructions what more do you want?

[jim wood]

Oh dear. I can see a few calls and a bit a of anger coming your way. The stress ball is in the post!!

[Steve Vincent]

Confirmed its a defect and has been seen before (no, really?!!). Unable to provide me with an ETA, if it'll be a fix pack or bundled in a major release, or if it'll be just server side updates or client. Just keep an eye on the details of each FP that comes out and update this thread when you spot it's release.

[mattgoff]

I opened my own PMR on this (20932 019 866) and specifically asked for a fixpack.

This business with the wizards really highlights the direction IBM Cognos is taking the product. It's all well-and-good to make it easier for new users to get off the ground, but not if doing so comes at the expense of the people really using the product. I feel like we've been seeing more and more regression bugs, PMRs left open indefinitely (even when fully documented and repeatable with test cases), and embarrassingly incomplete or incorrect documentation while the devs work on wizards, drag-and-drop interfaces, and integration with other Cognos products. From what I've seen in previews of upcoming products, that's not going to change any time soon, either.

I will say that I'm still cautiously optimistic about Parallel Interaction. If I can ever get IT to finish building my server (seriously, how long can it take to install an OS?), I intended to take a very serious look at upgrading.

Matt

[mattgoff]

not off to a good start - reply from support requested screenshots and error messages. Erm, i've already provided a model and instructions what more do you want?

I think they've recently updated their support to a true L1/L2/L3 type system. L1 is now completely worthless ("Have you tried rebooting?"), and you just have to suffer through them until you get to L2. Which, as we saw in the past, wasn't exactly a barrel-of-monkeys experience either.

I really miss the days when you could open a ticket and have a TM1 hotshot like Mike Cirillo on the phone inside of a few days to either work you through the issue or get a dev on the phone to figure out how to resolve it. I remember literally getting experimental fixpacks the same week I opened tickets....

Matt

[jim wood]

We have hit this bug. I'll get the client to add their weight to it,

Jim.

[mattgoff]

The Defect that you quoted was actually evaluated by Development and deemed as an Enhancement request and not a defect. Would you like me to raise an Enhancement request on your behalf?

[jim wood]

I'm not being funny but that is piss poor.

[Alan Kirk]

The Defect that you quoted was actually evaluated by Development and deemed as an Enhancement request and not a defect. Would you like me to raise an Enhancement request on your behalf?

That is un-frickin'-believable. I haven't been affected by this since I've never found a dimension order that gives me an appreciable improvement over the existing one, but had it been my SR I'd definitely be demanding to talk to someone up the line and giving them multiple earfuls about support staff who can't understand the difference between "it-does-not-work-properly" defect and "it-could-work-better" enhancement. I suspect that this is just bone-a**ed laziness on the development team's part, given that most "enhancement" requests are probably filed at the back of a filing cabinet in a basement somewhere in a village in Djibouti and can therefore involve zero work.

[Steve Vincent]

tell them to f-off and quote my PMR instead. It's not what i was told and is being treated as a defect

[mattgoff]

tell them to f-off and quote my PMR instead. It's not what i was told and is being treated as a defect

Bad news-- I did quote your PMR! That's the one (not mine) they're talking about when they said "enhancement"....

BTW, weird that it was "for sure" working for you in 9.5.1 as that's the rev I'm on, and it's def not working.

Matt

[Steve Vincent]

Matt, whoever is telling you that either doesn't understand what they are looking at or IBM have changed their mind and not told me....

IBM Update 21/06/11 11:22
ECuRep Mail Gateway - Received Mail and stored in ECuRep
------------------------------------------------------------------------
Mail From: cognos_support@ecurep.ibm.com
/ecurep/pmr/6/8/68125,999,866/mail20110621-122224-cognos_support
File: mail.html 3263 bytes
---------------------------- EMAIL TEXT START --------------------------
This email was also sent to: steve.vincent@AAAAAAAAAAAAA.com
Hi Steve,
It looks like this is already a known issue and is logged as a defect:
.
https://www-304.ibm.com/support/entdocv ... 27740&wv=1

IBM Update 08/07/11 12:13
ECuRep Mail Gateway - Received Mail and stored in ECuRep
------------------------------------------------------------------------
Mail From: cognos_support@ecurep.ibm.com
/ecurep/pmr/6/8/68125,999,866/mail20110708-131332-cognos_support
File: mail.html 1846 bytes
---------------------------- EMAIL TEXT START --------------------------
This email was also sent to: steve.vincent@AAAAAAAAAAAAA.com
Hi Steve,
Is there anything else we can help with in regards to this PMR? The
issue is logged with Dev for next release.
Thanks,

at no point does my PMR state it's being treated as an enhancement. If they had i would have escalated it 'cos it blatently is not.

[mattgoff]

At long last, I was able to get my test case in front of a dev who solved it. Apparently in the pro file, there is a mapping of variable order to dimension order. I'm sure we've all opened a pro file and tried to decode the use of all of the portions not accessible in the GUI (and they wonder why I keep asking them to stop spending $s on the GUIs and start fixing bugs and writing documentation). It appears that two numbers separated by a comma are called a property. Property number 579 defines the mapping between dimensions and variables (the number after the comma likely refers to the number of items/mappings) and is named ProcessVariablesPositions. (Obviously top secret). In my case, the pro file contained the following:

579,15
1
2
3
5
6
9
10
11
8
4
7
12
0
0
0

At the dev's advice, I changed it directly in the pro file to the following:

579,15
1
2
3
4
5
6
7
8
9
10
11
12
0
0
0

And lo and behold, problem solved! Who knows, Steve, this might just work for you. Note that a server restart was required.

It's very frustrating that a month elapsed between when I submitted the test case and this trivial solution was found. In fact, support didn't even send me this solution-- they sent me an email telling me my test case was incomplete and they needed additional files (it wasn't), but I looked through the reply history and found the dev's comment.

My guess is that this was related to dimension reordering and that an old version of TM1 introduced the error somehow. I have no idea if this will pop up in future files, but I at least know how to fix it manually now....

Matt

[jim wood]

Are IBM still looking at a fix for the root cause of the issue?

[mattgoff]

Are IBM still looking at a fix for the root cause of the issue?

I would hope so. I updated the SR after the UK was done for the day (it's apparently impossible for me to get out of the UK support queue even though I've been back in the States for two years now), so they probably haven't seen it yet. If the past is any indication, they're going to want to close it unless I can reproduce from scratch (not my model) in 9.5.2....

Matt

[mattgoff]

No comment from them yet on continuing to root cause the issue vs closing the SR, but I did get the following to my query about getting documentation on the file formats:

Thanks for the feedback. Regarding documentation of the .pro file format - this is internal documentation accessible only to TM1 Development.

He was a little big curt/frosty too. I wonder if he's reading these forums. I'm sorry Keith, just venting!

Documentation is quickly becoming my #1 issue with TM1 (not true, replication reliability and WAN performance will forever be my windmills). If they want to have good documentation, they need to get rid of the notion of internal vs. external. Unless there is something proprietary in the file format (a la the Office file format kerfuffle), I can't see any reason why there can't just be Documentation. It's not like I'm asking to get a copy of the source....

Matt

[mattgoff]

They're still working on a root cause. And they read the forums (Hi Karin!).

Matt

[Steve Vincent]

oddly my report of this issue was closed while i was on holiday with no sign of a resolution or why it was closed. i have since re-opened it because the earlier comments from their PRA team didn't seem to show an understanding of the actual issue. Karin has been great at badgering them and does understand the issue tho, so there is still hope. The full reply is below;

all described in this PMR is "works as designed", means "Views", "TI" and "Ordering-mechanism" works as designed.

Only to clarify:
1) The order of dimensions in a cube will be determined once. This is the only relevant order and can never be changed for a cube until the cube will be deleted completely and will be created completely new --> no visible reordering of dimensions in a cube is possible.
2) A reordering is only "Server-internally" possible for RAM-consumption purposes (right click on cube and reorder dimensions) - but as said, this is only an internal reordering and has nothing to do with the only official and once determined cube-structure (dimension order)

The customer only changes the order of dimensions in a view (for view-purposes......) - this has nothing to do with reordering of dimensions in a cube.
Thats the flexibility within a cube for analyzing data, simulating data..........

To the customers statements:

"If a 10 dim cube is built (dims A to J) then reordered J to A, a TI would think the 1st dim of a view was J when creating the view itself. But TABDIM for the 1st dim would return A."

ANSWER: Reordered is only the view (not the cube) and a view could be a data source for the TI as in this example. The TI accepts this data source and the structure of this view-datasource of course. The TI takes over the offered dimensions-names in the order of the view and uses this as variable-names - this is what the customer means. This is a "good will" of the TI to take over the dimension-names as variables. For example if a dimension name could not be read from the TI (name has hyphen or umlauts in it.......) the TI will create a new varable name like V1, V2 etc . And it is correct, that TABDIM returns for example in this case A for the first dimension - as said above, the order of dimensions in a cube never changes.
Works as designed and expected.

"The biggest issue caused is when an original order cube is re-ordered. Any TI against it that created, used then destroyed it's own views has to be re-written. It's impracticle, confusing and unacceptable to have to write code this way"

ANSWER: I'm not sure if I understand correctly, but I will give an answer how it should work. Views for TI's must be always and only Public Views and Public Views can only be created and changed from Admin-Users. Public Views can never be changed from a "normal" User - they only can store private views. There should always be only a few Admin-User for a TM1-system to develop or administer. If a public view for a TI is created as data source it should never be changed in the order of dimensions - it is the same if a relational-database via ODBC would be the data source for TI and the relational-table would be changed in their order........ . Therefore in TM1, view names for TI's must be always public views which can not be changed from the users (except Admin-Users and they should know it). Additionally public views for TI-process could be named like "Viewname_TI" and it should be clear for Admin-Users too that this view should not be changed in the order of dimensions.

I have no issue with points 1 & 2, but the models i sent do NOT stand up to the answers posed. They have confused views from the cube browser with views used via the TI interface. I have no bones with the cube browser, but the TI interface should always use the order the cube was built in. It is plain to see in the test models as well as the example Matt posted that TI is not doing this, which is the bug i stated originally. Probably why it has taken a while for this probem to get anywhere.