Hi all.
I'm after some help regarding memory use. A cube I have has the below values from stats by cube after restarting the instance. What I'm struggling to work out is how the memory used for input data is so huge in relation to the number of populated numeric cells.
The cube has 8 dimensions and none of them have a silly number of elements in either.
Is there something obvious I am missing here or is it just that the numbers displayed to me in this cube could be inaccurate?
Any help appreciated
Memory Used for Views 0
Number of Stored Views 0
Number of Stored Calculated Cells 0
Number of Populated String Cells 0
Number of Populated Numeric Cells 236167
Number of Fed Cells 54290624
Memory Used for Calculations 0
Memory Used for Feeders 16777216
Memory Used for Input Data 1811936256
Total Memory Used 1828713472
Thanks in advance
Huge amount of memory used for input data
-
- Community Contributor
- Posts: 248
- Joined: Tue Nov 01, 2011 10:31 am
- OLAP Product: TM1
- Version: All
- Excel Version: All
- Location: Manchester
- Contact:
Re: Huge amount of memory used for input data
A very recent post looked at a very similar issue - http://www.tm1forum.com/viewtopic.php?f=3&t=12279
-
- Posts: 5
- Joined: Tue Apr 21, 2015 10:26 am
- OLAP Product: TM1
- Version: 10.1
- Excel Version: 2010
Re: Huge amount of memory used for input data
Thanks Edward. I think that thread was slightly different though, If I'm reading it correctly that post was more to do with memory used by input data increasing even if the same data was reloaded to the cube. My issue is just around the amount of memory used per input data value, straight after a restart.
-
- MVP
- Posts: 264
- Joined: Mon Nov 03, 2014 8:23 pm
- OLAP Product: TM1
- Version: 9.5.2 10.1 10.2 PA2
- Excel Version: 2016
Re: Huge amount of memory used for input data
I was going to suggest you look at reordering the dimensions in the cube since that can sometimes result in a decrease in memory consumption for the cube. However, If a sub-optimal dimension ordering is used I'd expect to see the populated numeric cells to memory used for input data ratio to be very close to the ratio between number of fed cells and memory used for feeders. Might still be worth looking into...
- qml
- MVP
- Posts: 1094
- Joined: Mon Feb 01, 2010 1:01 pm
- OLAP Product: TM1 / Planning Analytics
- Version: 2.0.9 and all previous
- Excel Version: 2007 - 2016
- Location: London, UK, Europe
Re: Huge amount of memory used for input data
There have been issues with the reliability of these stats with some versions of TM1. Which version are you on?
To me it looks like in your case 'Memory Used for Input Data' and 'Memory Used for Feeders' are both wrong. Your feeder-to-populated-cell ratio is 230. This makes me think that your 'Memory Used for Input Data' includes some wrongly attributed memory that's really used by feeders. In a well-ordered 8-dimensional cube I would expect the average feeder flag to take up about 30 bytes and the average populated cell to take up about 50 bytes (both are ballpark figures of course!). The numbers calculated on the basis of the stats you provided are 3 and 7672, respectively - both orders of magnitude outside the believable range. So in your case I would trust the 'Total Memory Used', but not the split reported.
The true issue I see a strong potential for is the feeder-to-populated-cell ratio. While it's not impossible to have a cube where each populated cell legitimately feeds 230 other calculated cells, it is a very rare occurrence. You likely have some massive overfeeding and if you solve that then your total memory use will drop drastically (right now it's very high for the amount of data you are storing).
To me it looks like in your case 'Memory Used for Input Data' and 'Memory Used for Feeders' are both wrong. Your feeder-to-populated-cell ratio is 230. This makes me think that your 'Memory Used for Input Data' includes some wrongly attributed memory that's really used by feeders. In a well-ordered 8-dimensional cube I would expect the average feeder flag to take up about 30 bytes and the average populated cell to take up about 50 bytes (both are ballpark figures of course!). The numbers calculated on the basis of the stats you provided are 3 and 7672, respectively - both orders of magnitude outside the believable range. So in your case I would trust the 'Total Memory Used', but not the split reported.
The true issue I see a strong potential for is the feeder-to-populated-cell ratio. While it's not impossible to have a cube where each populated cell legitimately feeds 230 other calculated cells, it is a very rare occurrence. You likely have some massive overfeeding and if you solve that then your total memory use will drop drastically (right now it's very high for the amount of data you are storing).
Kamil Arendt
-
- MVP
- Posts: 600
- Joined: Wed Aug 17, 2011 1:19 pm
- OLAP Product: TM1
- Version: 9.5.2 10.1 10.2
- Excel Version: 2003 2007
- Location: York, UK
Re: Huge amount of memory used for input data
this is highly likely particularly given thatqml wrote:To me it looks like in your case 'Memory Used for Input Data' and 'Memory Used for Feeders' are both wrong
is an exact power of 2.its_me wrote:Memory Used for Feeders 16777216