Hi,
I have the following situation:
I have a TI process that works fine when launched by itself but when launched as part of another process (so by using executeprocess) it will fail with the message that the server was unable to allocate sufficient memory. Since the server itself is still under 1GB at this point I assume this to mean memory stack rather than pure RAM.
I have the same issue with this process on versions 9.5.1 and 9.5.2 32 Bit. On a 64 Bit server it works fine regardless of how you execute it.
Is there anyone who can clarify how exactly TM-1 is managing memory when TI processes are running. I.e. I always assumed it was caching in RAM untill it gets to the end of the tab (as it then commits) and then starts another cache.
Now since my experience above, this clearly is not the case. It seems that some sort of cache is created when starting a TI process and that this cache is 1) limited and 2) only released / recreated at the very end of the process. (As I see no other possibility to explain why the process works by itself but not when called over ExecuteProcess on 32 Bit).
Thanks for any info
Jeroen
About memory stacks and executeprocess
-
- Community Contributor
- Posts: 139
- Joined: Mon Sep 15, 2008 1:45 pm
-
- MVP
- Posts: 3654
- Joined: Fri Mar 13, 2009 11:14 am
- OLAP Product: TableManager1
- Version: PA 2.0.x
- Excel Version: Office 365
- Location: Switzerland
Re: About memory stacks and executeprocess
I think this belongs in a separate thread as it is a totally different subject and question.Jeroen Eynikel wrote:Hi,
I have the following situation:
I have a TI process that works fine when launched by itself but when launched as part of another process (so by using executeprocess) it will fail with the message that the server was unable to allocate sufficient memory. Since the server itself is still under 1GB at this point I assume this to mean memory stack rather than pure RAM.
I have the same issue with this process on versions 9.5.1 and 9.5.2 32 Bit. On a 64 Bit server it works fine regardless of how you execute it.
Is there anyone who can clarify how exactly TM-1 is managing memory when TI processes are running. I.e. I always assumed it was caching in RAM untill it gets to the end of the tab (as it then commits) and then starts another cache.
Now since my experience above, this clearly is not the case. It seems that some sort of cache is created when starting a TI process and that this cache is 1) limited and 2) only released / recreated at the very end of the process. (As I see no other possibility to explain why the process works by itself but not when called over ExecuteProcess on 32 Bit).
Thanks for any info
Jeroen
Sounds a bit weird. I have seen memory issues with TIs that recursively call themselves, TM1 doesn't seem to handle this very well. But splitting this into a "master" process to do the "calling" and a "slave" process to do the "doing" has always fixed this for me.
-
- Community Contributor
- Posts: 139
- Joined: Mon Sep 15, 2008 1:45 pm
Re: About memory stacks and executeprocess
Well I certainly agree with you that it is weird behaviour. I haven't ever had this problem before in 8 years of TM1 experience right now.
Maybe it will help a bit if I describe what the process does.
Basically it processes a source view and for every record in it it will cycle over several other dimensions (in while loops) to determine whether the combination has a value in a second cube.
So say my source view contains positions V, W, X, Y and Z. The process is then going to do multiple nested loops over dimensions A, B and C checking whether the combination of V, W, X, A, B, C has a data value in another cube. If so it writes a value to V,W,X,A,B,C in a third cube.
I have seen very weird behaviour with this so far. I.e. launching it directly: finishes succesfully and does what it is supposed to do.
Calling it from within another process over executeprocess => BigAlloc() error on one particular element in the 'V' range.
No problem on 64 Bit however.
My assumption right now is that when launching a TI process some sort of memory stack is reserved for it. And that launching another process from within this process does not create a new memory stack but rather keeps on using the old one. That would explain - to me at least - how what I am seeing is possible. It would also explain why I only have the problem on a 32 Bit server if one assumes the stack to be considerably bigger on a 64 Bit system which I think is a fair assumption.
Then again maybe it's a bug that manifests itself in both version 9.5.1 and 9.5.2 32 Bit, which is a possibility I cannot exclude either I guess.
If anyone knows how the memory allocation really works I'd be most gratefull for the info. If not I guess I will log it as a bug and see what comes of it.
Maybe it will help a bit if I describe what the process does.
Basically it processes a source view and for every record in it it will cycle over several other dimensions (in while loops) to determine whether the combination has a value in a second cube.
So say my source view contains positions V, W, X, Y and Z. The process is then going to do multiple nested loops over dimensions A, B and C checking whether the combination of V, W, X, A, B, C has a data value in another cube. If so it writes a value to V,W,X,A,B,C in a third cube.
I have seen very weird behaviour with this so far. I.e. launching it directly: finishes succesfully and does what it is supposed to do.
Calling it from within another process over executeprocess => BigAlloc() error on one particular element in the 'V' range.
No problem on 64 Bit however.
My assumption right now is that when launching a TI process some sort of memory stack is reserved for it. And that launching another process from within this process does not create a new memory stack but rather keeps on using the old one. That would explain - to me at least - how what I am seeing is possible. It would also explain why I only have the problem on a 32 Bit server if one assumes the stack to be considerably bigger on a 64 Bit system which I think is a fair assumption.
Then again maybe it's a bug that manifests itself in both version 9.5.1 and 9.5.2 32 Bit, which is a possibility I cannot exclude either I guess.
If anyone knows how the memory allocation really works I'd be most gratefull for the info. If not I guess I will log it as a bug and see what comes of it.
-
- Posts: 42
- Joined: Mon May 19, 2008 10:25 am
- OLAP Product: TM1, EV
- Version: 9.0 9.1 9.4 9.5 10.1 10.2
- Excel Version: 2003 2007 2010 2015
- Location: London, UK
Re: About memory stacks and executeprocess
Not sure if I am off beam here, but I had a problem with memory when I ran a process that was running a loop that created a new view with each pass
Each view was destroyed after use, but the memory consumption went up
I added the following to the cfg file and all has been well since
Hope this helps
Each view was destroyed after use, but the memory consumption went up
I added the following to the cfg file and all has been well since
Code: Select all
MaximumViewSize=2048
Anthony
That's no moon - that's a space station
That's no moon - that's a space station