AsciiOutput Taking Too Long

[ps_in_atl]

I have a chore that runs 4 processes that do the following:

1) Export data for 1 year from a forecasting cube (calculated from 10+ other cubes) to a csv file.
2) Export data for 1 year from a forecasting cube containing headcount (calculated from same sources as above) to a csv file.
3) Import data from csv file created by #1 to a cube that essentially holds our Income Statement and Balance Sheet.
4) Import data from csv file created by #2 to a cube that holds headcount (just FTE totals).

#2 - #4 are relatively quick (42s, 4s, and 1s respectively), but #1 takes a whopping 10 mins! I know the issue is that the rules/feeders that make up this data need to be recalculated, but it just should not take this long. I left this company a year ago and am now back, so it seems like this length of time has occurred in the year since I've been gone because I don't recall it taking this long. I also know the underlying issue behind these recalcs taking so long is the rules/feeders themselves, because since I left the time the server takes to come up has gone from ~20 mins to 48 mins (!!!).

These copies are done once an hour and then are synced the star server. The one thing that puzzles me is that the copies can vary in length from hour to hour. One hour it will perform as described above, while the next hour the entire chore will run in 15s. But it's all still the same data - export all non-zero leaf-level data, zero-out target, copy in source. I could understand if it was the sync chore and it varied because of the number of transactions needed to process. But a straight data copy? My thought on this is that possibly a lack of updates means the stargate view doesn't have to be recalculated? My knowledge of stargate views is less than expert.

So I see two possible solutions to this:

1) The first is obvious to me: improve the rules and feeders. I need to do this anyway. However, I have little time right now and so I need solution #2 to come through for me;
2) There HAS to be a way to export the data in #1 without recalculating the affected data, reprocessing the rules/feeders, reassigning cube dependencies, etc.

Am I overlooking a config setting or some TI function that will get me what I want?

Thanks in advance for the help!

[tomok]

There HAS to be a way to export the data in #1 without recalculating the affected data, reprocessing the rules/feeders, reassigning cube dependencies, etc.

There isn't, nor has there ever been, at least to my knowledge, a way to tell TM1 "hey don't bother recalculating anything, just give me what you have right now". I suspect you are spot on in your diagnosis of a feeder problem. Your time would be better spent correcting this. You might also look into the various config settings that contol how much memory is allocated to the views to see if tweaking those might help you.

[ps_in_atl]

There isn't, nor has there ever been, at least to my knowledge, a way to tell TM1 "hey don't bother recalculating anything, just give me what you have right now".

I thought so, but hoping for a miracle. I guess I'm off to fix some crap feeders.

Thanks

[ykud]

There isn't, nor has there ever been, at least to my knowledge, a way to tell TM1 "hey don't bother recalculating anything, just give me what you have right now".

I thought so, but hoping for a miracle. I guess I'm off to fix some crap feeders.

Thanks

Just an idea -- what if you just removed the rule file (substitute it with blank one) in Prologue, Export the cube and then get rule back in Epilogue? Might work exactly as 'disable calcs'.

[tomok]

Just an idea -- what if you just removed the rule file (substitute it with blank one) in Prologue, Export the cube and then get rule back in Epilogue? Might work exactly as 'disable calcs'.

I'm afraid all that would do is make all the cells that had a value calculated for them become zero so the export file would be empty except for any that actually contain data.

[ykud]

Just an idea -- what if you just removed the rule file (substitute it with blank one) in Prologue, Export the cube and then get rule back in Epilogue? Might work exactly as 'disable calcs'.

I'm afraid all that would do is make all the cells that had a value calculated for them become zero so the export file would be empty except for any that actually contain data.

Yep, calculated values will not export in this case, but just as long as you import data back into cubes with identical calc rules -- I see no problem.

[David Usherwood]

Unticking (or setting off in code) the rule calculation setting should produce identical results to taking the rule off and is a lot less work.
I wouldn't think there should be any difference in performance either.
Why not write directly from the view to the destination? I agree reading CSV files in is very quick, but you'd be saving the disc IO.