Feeders not re-firing on restart

Post Reply
BigG
Community Contributor
Posts: 211
Joined: Tue Sep 15, 2009 11:13 pm
OLAP Product: IBMPA
Version: PA 2.0 Cloud
Excel Version: 2010

Feeders not re-firing on restart

Post by BigG »

Hi,

I have feeders from a data entry 4 dimension cube (with string picklists from dims) targeting a 7 dimension cube that has used rules to 'pivot' the picklist string items from source cube.

I feed from strings, and use feedstrings; in 1st line of rules of cube with feeders (sourcecube). Everything works when I make a change to the rules and save down, OR or CubeProcessFeeders in TI , OR i change a string selection referenced in the rule...BUT when I restart the TM1 server , the feeders will not re-fire. Anyone experienced this before? Seems the only way I can 'refire' rules after a restart is to save the rules file for the cube with feeders.

Note:
PersistentFeeders=F
ForceReevaluationOfFeedersForFedCellsOnDataChange is not turned on

eg.
Rule in target cube:

Code: Select all

[ {'Actual','Forecast','Budget'} ]     =N:
	IF(
		(DB('SourceCube', !Version, !Project, 'ScheduleType') @= !ScheduleType)
		&
		DB('SourceCube', !Version, !Project, 'StatusOverall') @= !ProjectStatus)
		&
		(DB('SourceCube', !Version, !Project, 'StageISC') @= !ProjectStages),
		DB(''SourceCube', !Version,  !Project, !Measure),
		# Zero
		0
	);
Feeder in source cube (feed from strings with Feedstrings; in rules 1st line):

Code: Select all

[{'StageISC','StatusOverall' ,'ScheduleType''}  ] =>
	DB('Targetcube' !Version,  !Project,
	DB('SourceCube', !Version, !Project, 'StageISC') ,
	DB('SourceCube', !Version, !Project, 'StatusOverall'),
	DB('SourceCube', !Version, !Project, 'ScheduleType'),
	!Measure);
Last edited by BigG on Tue Sep 02, 2014 10:28 pm, edited 1 time in total.
GG
BigG
Community Contributor
Posts: 211
Joined: Tue Sep 15, 2009 11:13 pm
OLAP Product: IBMPA
Version: PA 2.0 Cloud
Excel Version: 2010

Re: Feeders not re-firing on restart

Post by BigG »

Sorry I have corrected the 1st post as CubeProcessFeeders in TI does work but I was referencing the targetcube not the sourcecube with feeders. So the only thing that does not refire feeders is a restart...

I am on TM1 10.1.1
Last edited by BigG on Wed Sep 10, 2014 1:15 am, edited 1 time in total.
GG
pandinus
Posts: 78
Joined: Tue Mar 18, 2014 8:02 am
OLAP Product: TM1, Cognos Express
Version: 10.2.2
Excel Version: 2013

Re: Feeders not re-firing on restart or CubeProcessFeeders(

Post by pandinus »

Do you have PersistentFeeders=T in your tm1s.cfg?
BigG
Community Contributor
Posts: 211
Joined: Tue Sep 15, 2009 11:13 pm
OLAP Product: IBMPA
Version: PA 2.0 Cloud
Excel Version: 2010

Re: Feeders not re-firing on restart

Post by BigG »

PersistentFeeders=F
ForceReevaluationOfFeedersForFedCellsOnDataChange is not turned on
GG
BigG
Community Contributor
Posts: 211
Joined: Tue Sep 15, 2009 11:13 pm
OLAP Product: IBMPA
Version: PA 2.0 Cloud
Excel Version: 2010

Re: Feeders not re-firing on restart

Post by BigG »

Found the issue to be with the string feeders source, in example above 'StatusOverall' string measure was a calculated item, and being feed from 3 other string measures (StatusBudget, StatusQuality, StatusSchedule). The issue is with flow on feeders, a change happens to one of the 3 measures, then the feeder does not fire with StatusOverall as source.

best to change the above feeder to feed from the 3 measures eg:

Code: Select all

[{'StageISC',StatusBudget, StatusQuality, StatusSchedule ,'ScheduleType''}  ] =>
   DB('Targetcube' !Version,  !Project,
   DB('SourceCube', !Version, !Project, 'StageISC') ,
   DB('SourceCube', !Version, !Project, 'StatusOverall'),
   DB('SourceCube', !Version, !Project, 'ScheduleType'),
   !Measure);
This resolved the refiring on restart issue.
GG
TrevorGoss
Community Contributor
Posts: 217
Joined: Thu Aug 15, 2013 9:05 am
OLAP Product: TM1
Version: 10.2.1.1
Excel Version: 14.0.6129.5000

Re: Feeders not re-firing on restart

Post by TrevorGoss »

BigG ,

We have had similar issues with TM1 and feeders not re-firing upon start up.

The only difference with our issue is that we have PersistantFeeders on. We run a nightly restart on all of our services but each Sunday we delete all of our .FEEDER files. When we come in on Monday we notice that some feeders are not feeding. They are however working when we just restart the services without deleting the .FEEDER files.

When we first got this issue, we had to go into the rules file of the source of a cube, and press enter inside the file to "kick it" and then save the file. This forced the feeders to work. This issue is a manual way of using the CubeProcessFeeders function for TIs.

We believe it is somthing to do with our.FEEDERS file, because if I run a chore to delete the .FEEDER files, but leave out the files that give us problems, they remain fed upon restart.

At the moment we run a chore which uses the CubeProcessFeeders function for all the cubes which give us issue, at the moment it is 5 source cubes, that either need kicking, or the .FEEDER files to be left alone during deletion. This is a sticky plaster solution and not best for the long term.

Any ideas on why this might be the case?

FYI: ForceReevaluationOfFeedersForFedCellsOnDataChange is left out of our cfg files.
rmackenzie
MVP
Posts: 733
Joined: Wed May 14, 2008 11:06 pm

Re: Feeders not re-firing on restart

Post by rmackenzie »

TrevorGoss wrote:The only difference with our issue is that we have PersistantFeeders on. We run a nightly restart on all of our services but each Sunday we delete all of our .FEEDER files.
This raises some questions:
1. Why do you run a nightly restart on the services?
2. Why do you delete the .feeder files on a Sunday?
3. Which order do you do the tasks on the Sunday?
4. When do you do SaveDataAll in the model?

There's some tips here, by the way.
Robin Mackenzie
TrevorGoss
Community Contributor
Posts: 217
Joined: Thu Aug 15, 2013 9:05 am
OLAP Product: TM1
Version: 10.2.1.1
Excel Version: 14.0.6129.5000

Re: Feeders not re-firing on restart

Post by TrevorGoss »

rmackenzie, thanks for the reponse.
This raises some questions:
1. Why do you run a nightly restart on the services?
2. Why do you delete the .feeder files on a Sunday?
3. Which order do you do the tasks on the Sunday?
4. When do you do SaveDataAll in the model?
1. We found that overtime, the longer the services ran, the less competent they were. We found that the services became slower, less efficent and we noticed that restarting them helped these issues go away. There is also no harm in restarting them nightly, we restart them during the early hours of the morning, via a batch.

2. We delete the .feeder files each Sunday to allow newer, more reliable .feeder files to be created. With new feeders being added and old ones being removed, we want the .Feeder files to be up to date. We delete these with the TI function DeleteAllPersistanFeeders

3. We run a TI every night that calls the SaveDataAll TI and then fires a batch to restart the service. The script also works out if it is on a Sunday, and if it is, it calls the DeleteAllPersistantFeeders file.

4. We call the SaveDataAll function every night.

I prefre this method than not using Persistant Feeder files at all, on the whole I find the Persistant Feeders option a great advantage, if you change a static parameter in the .cfg file, you can restart the service and in a minute or so, the change has been made. Without them, we have to wait 30 minutes to over 1 hour for a service to start up, the CPU reaches the limit and the server is more or less useless whilst waiting for the service, so I really like them.

Please ask if you want more details.

Thanks
BrianL
MVP
Posts: 264
Joined: Mon Nov 03, 2014 8:23 pm
OLAP Product: TM1
Version: 9.5.2 10.1 10.2 PA2
Excel Version: 2016

Re: Feeders not re-firing on restart

Post by BrianL »

SaveDataAll will write out both the .cub and the .feeder files for any cubes that have changed data. If this is run after DeleteAllPersistantFeeders it could very well just recreate the .feeder files you were trying to remove.
BariAbdul
Regular Participant
Posts: 424
Joined: Sat Mar 10, 2012 1:03 pm
OLAP Product: IBM TM1, Planning Analytics, P
Version: PAW 2.0.8
Excel Version: 2019

Re: Feeders not re-firing on restart

Post by BariAbdul »

[quote SaveDataAll
This is a TM1 TurboIntegrator function, valid only in TurboIntegrator processes.
This function saves all TM1 data from server memory to disk and restarts the log
file.[/quote]

Thanks BrianL ,Unfortunately TM1 reference guide(10.1) doesn't say anything about it.
"You Never Fail Until You Stop Trying......"
TrevorGoss
Community Contributor
Posts: 217
Joined: Thu Aug 15, 2013 9:05 am
OLAP Product: TM1
Version: 10.2.1.1
Excel Version: 14.0.6129.5000

Re: Feeders not re-firing on restart

Post by TrevorGoss »

SaveDataAll will write out both the .cub and the .feeder files for any cubes that have changed data. If this is run after DeleteAllPersistantFeeders it could very well just recreate the .feeder files you were trying to remove.
Thanks BrianL, that sounds reasonable, will investigate further and test to see if it makes a difference, I will post my response.

Do you think it is also possible that conditional feeders will have a hand in this sort of problem/issue?

Thanks.
rmackenzie
MVP
Posts: 733
Joined: Wed May 14, 2008 11:06 pm

Re: Feeders not re-firing on restart

Post by rmackenzie »

TrevorGoss wrote:1. We found that overtime, the longer the services ran, the less competent they were. We found that the services became slower, less efficent and we noticed that restarting them helped these issues go away. There is also no harm in restarting them nightly, we restart them during the early hours of the morning, via a batch.
Hi Trevor - perhaps there is a bit of compromise... when you load the .cub file on start-up TM1 is loading disk-based data into memory. As queries are performed on the in-memory data they are cached and subsequent queries are available from these caches meaning speedier results. This is a basic principle and is subject to a bunch of configuration parameters (VMM, VMT plus tm1s.cfg parameters) but the idea is basically to build performance gains by caching results. Where you have a lot of cubes being constantly updated from source systems or user input then these gains will maybe lost through natural invalidation of the caches, but for straightforward reporting cubes doing the server restarts is destroying the caches perhaps unnecessarily.

I guess all the other steps you take are the result of working out what works best for you, but it sounds like a root-and-branch investigation of what is making your rules and feeders perform badly would suit you better in the long run.
Robin Mackenzie
TrevorGoss
Community Contributor
Posts: 217
Joined: Thu Aug 15, 2013 9:05 am
OLAP Product: TM1
Version: 10.2.1.1
Excel Version: 14.0.6129.5000

Re: Feeders not re-firing on restart

Post by TrevorGoss »

. As queries are performed on the in-memory data they are cached and subsequent queries are available from these caches meaning speedier results. This is a basic principle and is subject to a bunch of configuration parameters (VMM, VMT plus tm1s.cfg parameters) but the idea is basically to build performance gains by caching results
Hi rmackenzie,

if I am not mistaken, TM1 creates stargate views in memory once a query has been raised on a cube? Also, what parameters are you specifically talking about?

We are going to avoid using the SaveDataAll function after our DeleteAllPersistantFeeders function is called, see if makes any difference. We will find out on Monday.
I guess all the other steps you take are the result of working out what works best for you, but it sounds like a root-and-branch investigation of what is making your rules and feeders perform badly would suit you better in the long run.
Indeed, we will keep trying to find out what causes what and why.
TrevorGoss
Community Contributor
Posts: 217
Joined: Thu Aug 15, 2013 9:05 am
OLAP Product: TM1
Version: 10.2.1.1
Excel Version: 14.0.6129.5000

Re: Feeders not re-firing on restart

Post by TrevorGoss »

We are going to avoid using the SaveDataAll function after our DeleteAllPersistantFeeders function is called, see if makes any difference.
I applied this change a couple of weeks ago and it seems to of worked. Each Monday, after the .FEEDER files have been deleted, the cubes that containted feeders that did not re calculate upon start up are now being fed when the model starts up.
SaveDataAll will write out both the .cub and the .feeder files for any cubes that have changed data. If this is run after DeleteAllPersistantFeeders it could very well just recreate the .feeder files you were trying to remove.
This also seems to be true, we run the SaveDataAll and every sunday delete the .FEEDER files, but we no longer call the SaveDataAll function a second time.

The problem does look as if it has been resolved, but more testing to make sure!
Post Reply