View not loading in TI Process

[ramjam]

Hi,

I've done quite a bit of searching on this and haven't found a solution yet, so thought I'd try posting here. I've got a TI process that does an ASCIIOUTPUT from a view used as the data source. That process has been consistently failing for the last few weeks with an error popup of "Unexpected return in function: COrionTreView::OnProcessExecute" The log file error is:

2436 [4] WARN 2019-12-11 13:31:55.997 TM1.Server.Memory al_Alloc() outOfMemory Exception <<< MEMORY_TEMP_POOL_EXCEEDED >>> MaximumViewSize memory exceeded - apifunc# "155" - pool# "0" - poolsize "2516574976.000000"

It looks like the process is not able to open the view. We added 4GB of RAM (from 48 to 52GB) about a month ago, so I don't think it's really a memory issue. In addition, we have process that uses much larger views that are working OK. I tried exporting the view manually - that's worked intermittently. When it does work, it hangs on Record 1 for a couple of minutes, then starts exporting (slowly, imho), and exports about 150,000
records (zero's suppressed). Since it seems to take quite a while for that export to start working, my 1st thought is that the process is timing out - anyone know if that's possible, and if so, is there a config setting I'm missing?

The view uses a couple of dynamic subsets (pulling mainly Level 1 elements), and 4 static subsets. I've tried creating the view and subsets dynamically within the TI process, the making all of the subsets static, but no love there.

As a temporary workaround, instead of a view, I tried using one of the subsets as the datasource, the nesting some loops through the other dimensions to walk through all of the required coordinates. That works, but I'm getting the following non-fatal log error which I think may be a clue (or it could just be a red herring):

"xxxxxxxxxxxxx",Data Source line (2) Error: MetaData procedure line (22): Invalid key: Dimension Name: "dimname", Element Name (Key): ""
Error: MetaData procedure line (22): error repeats 998 times
"xxxxxxxxxxxxx",Data Source line (2) Error: MetaData procedure line (0):
Error: MetaData procedure line (10037): ............. Log file error limit has been reached.

Where xxxxxxxxxxx is an element that I'm actually getting for a subset and dimname is the name of the dimension - since I know that element actually does exist (it's coming from a subset, after all), that makes me suspicious. Also, we have another view that uses the same two dynamic subsets that is also failing, so it seems likely it's something in subset - is it possible for the dimension to be corrupted? If so, how do I identify/fix? The element that's identified in the error looks OK to me, but...

We migrated from 10.2.2 to 2.05 about 3 months ago - the views seemed to work OK until about 2-3 weeks ago.

Hope that's enough to go on w/o being too much detail - thanks for any ideas you guys might have.

[tomok]

The two issues you are seeing are completely unrelated. For the MEMORY_TEMP_POOL_EXCEEDED error it is indeed a memory issue. You have exceeded the amount of memory that can be allocated to a view so TM1 is wigging out. I'm guessing your view has a bunch of rule calculated and/or consolidations in it. TM1 has to calculate these before the view can be processed. If these are extensive it can take a heck of a lot longer to export than just raw data. This explains the delay. Look into the VMM setting for the cube in question and up it significantly. I know nothing about your environment so I can't offer any specific settings. You should also be aware that memory consumption in Planning Analytics is much higher than in TM1 10.2.2. I'm not going to get into specifics. You can Google it. However, this probably explains why it was never an issue before and it is now under PA.

Now to the metadata error. Why are you updating dimensions (the reason for code in the Metadata tab) in a process whose only purpose is to export data to a flat file? This makes no sense. You aren't telling us something here.

"xxxxxxxxxxxxx",Data Source line (2) Error: MetaData procedure line (22): Invalid key: Dimension Name: "dimname", Element Name (Key): ""

This tells me you are attempting some sort of dimension maintenance and you are supplying an empty string as the element. The answer is to look at the code where you are generating or assigning the element name to a variable and figure out why the variable is empty. Saying it came from a subset doesn't mean anything unless it is a subset of the exact same dimension you are trying to alter. Basically, it's not a TM1 problem. It's TM1 coding problem. Unless you post the actual code here no is going to be able to help you figure it out.

[gtonkin]

Have you tried increasing MaximumViewSize in your TM1s.CFG?
Looks like it has been tweaked but may not be high enough.

edit: had not seen Tom's post before I hit submit-my suggestion is a short-term attempt to see if it is something trivial but would recommend further investigation per Tom's post if it works.

[ramjam]

re. increasing max view size in TM1s - I was under the impression that was only for end users opening views and that would not impact TI processes - am I mis-informed? Could be, as I often see conflicting info on TM1 topics, sometimes even in IBM's own documentation:-).

[ramjam]

Thanks Tom,

I forgot to mention, I did play around with the VMM and VMT settings for the cube, but was pretty subtle with my changes, mainly b/c, as far as I can tell, those settings just control the circumstance under which TM1 will create Stargate views so I was not really understanding why this would affect a TI process - can you tell me if a process will actually timeout if the source view doesn't load in a timely manner? If not, it seems like committing system resources to an SG view that only gets used once a day would be inefficient - am I missing something here?

re. the consolidations/rules, there are a couple of simple rules in the source cube, but two of the dimensions may be consolidation heavy, how would that be determined - some ratio of base elements to consolidated elements? If so, at what point would you say yes, that dimensions has a lot of consolidations? I understand what you're saying about calculations consuming memory - do you know if all the consolidations are calculated when the view is built, or just the consolidations in the subsets used in the view?

[lotsaram]

re. increasing max view size in TM1s - I was under the impression that was only for end users opening views and that would not impact TI processes - am I mis-informed? Could be, as I often see conflicting info on TM1 topics, sometimes even in IBM's own documentation:-).

MaximumViewSize doesn't affect views used for TI processing except if you use ViewConstruct in which case it does.

[PavoGa]

Have you checked the dimension order? I have had performance issues with a cube that reordering the dimensions resolved.

[Andy Key]

MaximumViewSize doesn't affect views used for TI processing except if you use ViewConstruct in which case it does.

...and since 10.2.2 FP6, TM1 has had the ability to perform an implicit ViewConstruct on views used for TI processing. This was originally disabled by default - but this changed in FP7 to being enabled by default. Not sure in which versions the default changed in PA, and if I remember correctly the effect of MaximumViewSize on the threads constructing the view in the implicit ViewConstruct has changed between versions as well.

As the existence, default values and lifespan of the effects of these commands have changed over time, look at the documentation of DisableMTQViewConstruct, EnableMTQViewConstruct and MTQQuery for your specific version.

If you're simply running this single TI from Architect, it's probably the case that you can just put a DisableMTQViewConstruct(); in the prolog and this will sort it out.

[ramjam]

Solved? - I tested out the DisableMTQViewConstruct() in my QA environment and that seems to have corrected the problem. Thanks for the tip Andy - I'm not sure I would have ever gotten to that one on my own, and thank you to everyone else who provided suggestions.