Dimension update result in element loss

Post Reply
kenship
Posts: 58
Joined: Thu May 30, 2013 1:41 pm
OLAP Product: Cognos
Version: TM1 10.2
Excel Version: 2010

Dimension update result in element loss

Post by kenship » Thu Oct 19, 2017 1:41 pm

We had a data loss issue due to element loss during dimension update.

The TI process created to update this GL account dimension has been running for a few years without problem. One process will call on another at epilog.

I checked the log and no other process is running at the same time.

Below is the log. The process can be identified by "4148". As you can see, the process resulted in error due to a rule in one cube. However we never had this problem before and I wonder why we had the problem this time.

Any help is appreciated.

Kenneth


3640 [] INFO 2017-10-18 16:32:30.140 TM1.Chore Chore "6. Transfer SAL RPT to SAL OBM MAPPING" executed by scheduler
3640 [] INFO 2017-10-18 16:32:30.140 TM1.Process Process 06AM_Chorequit executed by chore 6. Transfer SAL RPT to SAL OBM MAPPING
3640 [] INFO 2017-10-18 16:32:30.140 TM1.Process Process "06AM_Chorequit": finished executing normally, elapsed time 0.00 seconds
3640 [] INFO 2017-10-18 16:32:30.140 TM1.Process Process corp_obm_sal_int_data_transfer_corp_SAL_RPT_to_corp_SAL_OBM_MAPPING executed by chore 6. Transfer SAL RPT to SAL OBM MAPPING
3640 [] INFO 2017-10-18 16:33:27.346 TM1.Process Process "corp_obm_sal_int_data_transfer_corp_SAL_RPT_to_corp_SAL_OBM_MAPPING": finished executing normally, elapsed time 57.21 seconds
3640 [] INFO 2017-10-18 16:33:27.346 TM1.Chore Chore "6. Transfer SAL RPT to SAL OBM MAPPING" time = 57.21 seconds
3640 [] INFO 2017-10-18 16:33:28.890 TM1.Chore Chore "6. Transfer SAL RPT to SAL OBM MAPPING" finished executing
4148 [eaa] INFO 2017-10-18 16:33:57.657 TM1.Process Process "meta_dim_update_corp_obm_account_FROM_CSV" executed by user "ABC"
4148 [eaa] ERROR 2017-10-18 16:33:58.733 TM1.Dimension corp_OBM_CALENDARIZATION Dimension Update Fail. Rule Is Invalid: Error compiling rule, line number 49: Element not found "01000"
[{'01000','01005','02500','29009','26110','26115','26125','57900','57910','70190'},'Method'] = S: 'N/A';
4148 [eaa] ERROR 2017-10-18 16:34:00.636 TM1.Process Process "meta_dim_update_corp_obm_account_FROM_CSV": : Execution was aborted. Error file: <TM1ProcessError_20171018163400_meta_dim_update_corp_obm_account_FROM_CSV.log>
5740 [] INFO 2017-10-18 16:34:30.105 TM1.Chore Chore "7. Transfer SAL OBM MAPPING to OBM BUDGET INPUT" executed by scheduler
5740 [] INFO 2017-10-18 16:34:30.105 TM1.Process Process corp_obm_data_transfer_corp_SAL_OBM_MAPPING_to_corp_OBM_BUDGET_INPUT executed by chore 7. Transfer SAL OBM MAPPING to OBM BUDGET INPUT
5740 [] ERROR 2017-10-18 16:34:30.324 TM1.Process Process "corp_obm_data_transfer_corp_SAL_OBM_MAPPING_to_corp_OBM_BUDGET_INPUT": : Execution was aborted. Error file: <TM1ProcessError_20171018163430_corp_obm_data_transfer_corp_SAL_OBM_MAPPING_to_corp_OBM_BUDGET_INPUT.log> : Dimension element not found
5740 [] INFO 2017-10-18 16:34:30.324 TM1.Process Process corp_obm_data_update_corp_OBM_BUDGET_INPUT_zero_FTE_n_CF_SnB_Amount executed by chore 7. Transfer SAL OBM MAPPING to OBM BUDGET INPUT
5740 [] ERROR 2017-10-18 16:34:31.291 TM1.Process Process "corp_obm_data_update_corp_OBM_BUDGET_INPUT_zero_FTE_n_CF_SnB_Amount": finished executing with errors. Error file: <TM1ProcessError_20171018163430_corp_obm_data_update_corp_OBM_BUDGET_INPUT_zero_FTE_n_CF_SnB_Amount.log> : Invalid key: Dimension Name: "corp_obm_account", Element Name (Key): "02500"
5740 [] INFO 2017-10-18 16:34:31.291 TM1.Process Process corp_SAL_OBM_MAPPING_to_corp_SPM_REORG_MAPPING executed by chore 7. Transfer SAL OBM MAPPING to OBM BUDGET INPUT
4148 [eaa] INFO 2017-10-18 16:34:34.130 TM1.Process Process "meta_dim_update_corp_obm_account_FROM_CSV" executed by user "ABC"
5740 [] INFO 2017-10-18 16:34:42.195 TM1.Process Process "corp_SAL_OBM_MAPPING_to_corp_SPM_REORG_MAPPING": finished executing normally, elapsed time 10.90 seconds
5740 [] INFO 2017-10-18 16:34:42.195 TM1.Process Process corp_obm_data_transfer_corp_SPM_REORG_MAPPING_to_corp_OBM_BUDGET_INPUT executed by chore 7. Transfer SAL OBM MAPPING to OBM BUDGET INPUT
5740 [] ERROR 2017-10-18 16:34:42.445 TM1.Process Process "corp_obm_data_transfer_corp_SPM_REORG_MAPPING_to_corp_OBM_BUDGET_INPUT": : Execution was aborted. Error file: <TM1ProcessError_20171018163442_corp_obm_data_transfer_corp_SPM_REORG_MAPPING_to_corp_OBM_BUDGET_INPUT.log> : Dimension element not found
5740 [] INFO 2017-10-18 16:34:42.445 TM1.Process Process meta_dim_update_corp_obm_BVE_alias executed by chore 7. Transfer SAL OBM MAPPING to OBM BUDGET INPUT
5740 [] INFO 2017-10-18 16:34:43.069 TM1.Process Process "meta_dim_update_corp_obm_BVE_alias": finished executing normally, elapsed time 0.62 seconds
5740 [] INFO 2017-10-18 16:34:43.069 TM1.Chore Chore "7. Transfer SAL OBM MAPPING to OBM BUDGET INPUT" time = 12.96 seconds
5740 [] INFO 2017-10-18 16:34:43.209 TM1.Chore Chore "7. Transfer SAL OBM MAPPING to OBM BUDGET INPUT" finished executing
4148 [eaa] ERROR 2017-10-18 16:34:44.317 TM1.Dimension corp_OBM_CALENDARIZATION Dimension Update Fail. Rule Is Invalid: Error compiling rule, line number 49: Element not found "01000"
[{'01000','01005','02500','29009','26110','26115','26125','57900','57910','70190'},'Method'] = S: 'N/A';
4148 [eaa] ERROR 2017-10-18 16:34:44.566 TM1.Process Process "meta_dim_update_corp_obm_account_FROM_CSV": : Execution was aborted. Error file: <TM1ProcessError_20171018163444_meta_dim_update_corp_obm_account_FROM_CSV.log>
5204 [] INFO 2017-10-18 16:37:50.053 TM1.Chore Chore "8. Transfer SAL OBM MAPPING and OBM BUDGET INPUT to OBM REPORT" executed by scheduler
5204 [] INFO 2017-10-18 16:37:50.053 TM1.Process Process corp_obm_data_transfer_corp_SAL_OBM_MAPPING_to_corp_OBM_BUDGET_REPORT executed by chore 8. Transfer SAL OBM MAPPING and OBM BUDGET INPUT to OBM REPORT
5204 [] ERROR 2017-10-18 16:37:50.256 TM1.Process Process "corp_obm_data_transfer_corp_SAL_OBM_MAPPING_to_corp_OBM_BUDGET_REPORT": : Execution was aborted. Error file: <TM1ProcessError_20171018163750_corp_obm_data_transfer_corp_SAL_OBM_MAPPING_to_corp_OBM_BUDGET_REPORT.log> : Dimension element not found
5204 [] INFO 2017-10-18 16:37:50.256 TM1.Process Process corp_obm_data_transfer_corp_OBM_BUDGET_INPUT_to_corp_OBM_REPORT_all executed by chore 8. Transfer SAL OBM MAPPING and OBM BUDGET INPUT to OBM REPORT
5204 [] INFO 2017-10-18 16:37:50.256 TM1.Process Process "corp_obm_data_transfer_corp_OBM_BUDGET_INPUT_to_corp_OBM_BUDGET_REPORT" run from process "corp_obm_data_transfer_corp_OBM_BUDGET_INPUT_to_corp_OBM_REPORT_all" by user "R*8. Transfer SAL OBM MAPPING and OBM BUDGET INPUT to OBM REPORT"
5204 [] INFO 2017-10-18 16:38:01.691 TM1.Process Process "corp_obm_data_transfer_corp_OBM_BUDGET_INPUT_to_corp_OBM_BUDGET_REPORT": finished executing normally, elapsed time 11.43 seconds
5204 [] INFO 2017-10-18 16:38:01.691 TM1.Process Process "corp_obm_data_transfer_corp_OBM_BUDGET_INPUT_to_corp_OBM_BUDGET_REPORT_sal_from_obm" run from process "corp_obm_data_transfer_corp_OBM_BUDGET_INPUT_to_corp_OBM_REPORT_all" by user "R*8. Transfer SAL OBM MAPPING and OBM BUDGET INPUT to OBM REPORT"
5204 [] ERROR 2017-10-18 16:38:01.784 TM1.Process Process "corp_obm_data_transfer_corp_OBM_BUDGET_INPUT_to_corp_OBM_BUDGET_REPORT_sal_from_obm": : Execution was aborted. Error file: <TM1ProcessError_20171018163801_corp_obm_data_transfer_corp_OBM_BUDGET_INPUT_to_corp_OBM_BUDGET_REPORT_sal_from_obm.log> : Dimension element not found
5204 [] INFO 2017-10-18 16:38:01.784 TM1.Process Process "corp_obm_data_transfer_corp_OBM_BUDGET_INPUT_to_corp_OBM_BUDGET_REPORT_WWW_Fcst_Adj" run from process "corp_obm_data_transfer_corp_OBM_BUDGET_INPUT_to_corp_OBM_REPORT_all" by user "R*8. Transfer SAL OBM MAPPING and OBM BUDGET INPUT to OBM REPORT"
5204 [] INFO 2017-10-18 16:38:01.940 TM1.Process Process "corp_obm_data_transfer_corp_OBM_BUDGET_INPUT_to_corp_OBM_BUDGET_REPORT_WWW_Fcst_Adj": finished executing normally, elapsed time 0.16 seconds
5204 [] ERROR 2017-10-18 16:38:01.956 TM1.Process Process "corp_obm_data_transfer_corp_OBM_BUDGET_INPUT_to_corp_OBM_REPORT_all": finished executing with errors. Error file: <TM1ProcessError_20171018163801_corp_obm_data_transfer_corp_OBM_BUDGET_INPUT_to_corp_OBM_REPORT_all.log>
5204 [] INFO 2017-10-18 16:38:01.956 TM1.Process Process corp_obm_data_transfer_corp_SPM_REORG_MAPPING_to_corp_OBM_BUDGET_REPORT executed by chore 8. Transfer SAL OBM MAPPING and OBM BUDGET INPUT to OBM REPORT
5204 [] ERROR 2017-10-18 16:38:02.205 TM1.Process Process "corp_obm_data_transfer_corp_SPM_REORG_MAPPING_to_corp_OBM_BUDGET_REPORT": : Execution was aborted. Error file: <TM1ProcessError_20171018163802_corp_obm_data_transfer_corp_SPM_REORG_MAPPING_to_corp_OBM_BUDGET_REPORT.log> : Dimension element not found
5204 [] INFO 2017-10-18 16:38:02.205 TM1.Chore Chore "8. Transfer SAL OBM MAPPING and OBM BUDGET INPUT to OBM REPORT" time = 12.15 seconds
5204 [] INFO 2017-10-18 16:38:03.266 TM1.Chore Chore "8. Transfer SAL OBM MAPPING and OBM BUDGET INPUT to OBM REPORT" finished executing

User avatar
jim wood
Site Admin
Posts: 3500
Joined: Wed May 14, 2008 1:51 pm
OLAP Product: TM1
Version: TM1 10.2.2
Excel Version: 2007
Location: 1639 Route 10, Suite 107, Parsippany, NJ, USA
Contact:

Re: Dimension update result in element loss

Post by jim wood » Thu Oct 19, 2017 1:59 pm

Does the process do a complete rebuild? If it does my guess would be that an account hard coded in to a rule is being removed. But without knowing the rule / process detail it's very difficult to say for certain.
Struggling through the quagmire of life to reach the other side of who knows where.
Application Consulting Group (ACG) TM1 Consulting
OS: Windows 7 64-bit. TM1 Version: 10.1.1

kenship
Posts: 58
Joined: Thu May 30, 2013 1:41 pm
OLAP Product: Cognos
Version: TM1 10.2
Excel Version: 2010

Re: Dimension update result in element loss

Post by kenship » Thu Oct 19, 2017 2:23 pm

Thanks for the reply.

The rule that triggered the error is here:

[{'01000','01005','02500','29009','26110','26115','26125','57900','57910','70190'},'Method'] = S: 'N/A';

the numbers are GL accounts which the process is trying to update.

What's strange is that the rule has been there for years and so the TI process and there are similar rules (meaning GL account hardcoded) in other cubes and have been working for years.

Why then suddenly we have a conflict? We later ran the dimension update process minutes later and have no issue, although it's already too late as data was lost.

Kenneth

orlando
Posts: 5
Joined: Fri Aug 04, 2017 8:27 am
OLAP Product: TM1
Version: 10.2.2
Excel Version: 2010

Re: Dimension update result in element loss

Post by orlando » Fri Oct 20, 2017 11:34 am

Hi Kenneth,

i guess there is a "DimensionDeleteAllElements" used in the process, that caused the problem, while the account wasn't in your datascource.
Maybe this will help in the future:
I try to avoid "DimensionDeleteAllElements". In most times when rebuilding a structure, i let a prozess run through the structure and destroy it (so all elements are on root level) These elements are the moved to a new parent "old elements".
The rebuild prozess checks for each element, if it is an child of "old elements" and remove it from there.
At the end you have your new structure - all elements that were no delivered by the datasource are find under "old elements", so no figures are lost.

Best Regards
orlando

User avatar
jim wood
Site Admin
Posts: 3500
Joined: Wed May 14, 2008 1:51 pm
OLAP Product: TM1
Version: TM1 10.2.2
Excel Version: 2007
Location: 1639 Route 10, Suite 107, Parsippany, NJ, USA
Contact:

Re: Dimension update result in element loss

Post by jim wood » Fri Oct 20, 2017 1:59 pm

Orlando,

That just can't be it as the process issue would have arisen earlier. You need to keep in mind when the delete all is committed. My guess would be that one of the rule accounts has actually been removed from the chart accounts. May be it's being recycled?

Jim.
Struggling through the quagmire of life to reach the other side of who knows where.
Application Consulting Group (ACG) TM1 Consulting
OS: Windows 7 64-bit. TM1 Version: 10.1.1

kenship
Posts: 58
Joined: Thu May 30, 2013 1:41 pm
OLAP Product: Cognos
Version: TM1 10.2
Excel Version: 2010

Re: Dimension update result in element loss

Post by kenship » Fri Oct 20, 2017 2:19 pm

I checked and it all the rule accounts are present in our load file. We don't remove chart account at all.

In the mean time I will disable that rule as it is not important.
jim wood wrote:
Fri Oct 20, 2017 1:59 pm
Orlando,

That just can't be it as the process issue would have arisen earlier. You need to keep in mind when the delete all is committed. My guess would be that one of the rule accounts has actually been removed from the chart accounts. May be it's being recycled?

Jim.

User avatar
jim wood
Site Admin
Posts: 3500
Joined: Wed May 14, 2008 1:51 pm
OLAP Product: TM1
Version: TM1 10.2.2
Excel Version: 2007
Location: 1639 Route 10, Suite 107, Parsippany, NJ, USA
Contact:

Re: Dimension update result in element loss

Post by jim wood » Fri Oct 20, 2017 2:43 pm

Instead of having hard coded rule like that, create a lookup cube or an alias and have the rule looking at it, so something like:

Code: Select all

['Method'] = S: 
   IF(CapIt(AttrS('Accounts',!Accounts,'Flag'))@='Y'
       ,'N/A'
       ,Continue
      );
That way the rule will not fail even if it is removed. It also takes away the problem Orlando mentioned,

Jim.
Struggling through the quagmire of life to reach the other side of who knows where.
Application Consulting Group (ACG) TM1 Consulting
OS: Windows 7 64-bit. TM1 Version: 10.1.1

Post Reply