Resolving Lock Contentions

Post Reply
User avatar
PavoGa
MVP
Posts: 617
Joined: Thu Apr 18, 2013 6:59 pm
OLAP Product: TM1
Version: 10.2.2 FP7, PA2.0.9.1
Excel Version: 2013 PAW
Location: Charleston, Tennessee

Resolving Lock Contentions

Post by PavoGa »

Have run into a wall.

Process A is a control process that runs various sub-processes, mostly through the ExecuteProcess function. However, one of the first things we need to do as part of the process is to insert a temporary element into one dimension, Versions, and add the members of a dynamic subset of the Versions dimension to that consolidation. We then use that consolidation element in a subsequent subset for datasource view. So, I decided to explore making the modification by running the process with TM1RunTI (ExecuteCommand) to make the dimension modifications in order to prevent contention locks for all users (Versions dim is HEAVILY used throughout the model).

And got slammed with a contention through a READ lock on a seemingly unrelated dimension. Here is the order in which things are happening:
  1. Process A reads the server instance from a }System_Environment cube to retrieve the TM1 instance to pass along to TM1RunTI (-connect parameter). This cube contains the Counter dim (dimension with elements 1...N)
  2. Process A then calls TM1RunTI, passing along the connection parameter read from the cube to execute Process B.
  3. Process B Prolog checks for the consolidation element and adds it (DimensionElementInsertDirect). If the C element exists, it removes its components (DimensionElementComponentDelete)
  4. Process B Prolog then creates a subset and adds the consolidation element to it
  5. Process B Prolog then creates dynamic subset of the elements designated to be components of the consolidation. Said subset is converted to a static subset.
  6. Process B Prolog then adds the members of the second subset to the C element referred to in step 3. (DimensionElementComponentAdd)
  7. Process B's Epilog is then supposed to execute a process that updates a process statistics cube we use and exits.
It never gets to Step 7. Using this code:

Code: Select all

                    strTimeStamp = TIMST(NOW, '\d:\h:\i:\s');
                    strOutput = 'Checkpoint 5';
                    strCommand = EXPAND('cmd /c echo %strOutput% >>"%strOutputFile%"');
                    ExecuteCommand(strCommand, 0);
...I can follow along what is completed in the process (the checkpoint number varies of course) and it never gets out of the PROLOG. Checkpoint 5 is the last code in the PROLOG, Checkpoint 6 is the first code in the EPILOG and the log contains:
Checkpoint 1
Checkpoint 2
Checkpoint 3
Checkpoint 4
Checkpoint 5
The lock seems to be preventing the PROLOG from completing. From the log:
1217636 [c] DEBUG 2017-06-01 17:01:32.823 TM1.Lock.Exception Contention encountered attempting to acquire lock (0x000000000243A6C8) on object [Dimension "Counter", addr=0x000000000243A610, index=R-232] in IX mode at ..\tm1_r7s\TM1DimensionImpl.cpp:8931 during function 'ProcessExecuteEx'.
Entering wait state 'IXCur'.
Blocked by the following 0 threads:
Which happens to be the last entry in the log until I have to shut it down to get out of the contention.

All that to ask this: why is a read on a cube containing the counter dimension preventing modifications to another dimension? And by the way, this runs fine IF instead of using TM1RunTI, ExecuteProcess is used OR the environment for the -connect parameter is hard-coded to avoid reading the }System_Environment cube. I will appreciate any insight.

Other questions: Is there a method to avoid the lock contention on the unrelated dimension without hard coding the environment? Am I chasing a rabbit? Why do I always grab precisely 10 M&Ms from the jar?

I am going to try changing the cube from }System_Environment to System_Environment to see if that is causing all this gnashing of teeth and pulling of hair.
Ty
Cleveland, TN
User avatar
PavoGa
MVP
Posts: 617
Joined: Thu Apr 18, 2013 6:59 pm
OLAP Product: TM1
Version: 10.2.2 FP7, PA2.0.9.1
Excel Version: 2013 PAW
Location: Charleston, Tennessee

Re: Resolving Lock Contentions

Post by PavoGa »

I am going to try changing the cube from }System_Environment to System_Environment to see if that is causing all this gnashing of teeth and pulling of hair.
Well that accomplished nothing.
Ty
Cleveland, TN
User avatar
PavoGa
MVP
Posts: 617
Joined: Thu Apr 18, 2013 6:59 pm
OLAP Product: TM1
Version: 10.2.2 FP7, PA2.0.9.1
Excel Version: 2013 PAW
Location: Charleston, Tennessee

Re: Resolving Lock Contentions

Post by PavoGa »

Update:

Tried using only the DIRECT dimension manipulation functions. Still locks up, although I have further refined where it is happening. It is locking up where the dynamic subset is created. Which is a little strange because it was getting past that previously. Here is the PROLOG for the second process:

Code: Select all

############################
# Process Name: SpendPlan - Setup Version Dimension.PRO

# Ty Cardin, May 31, 2017
#
# Doing this to try and reduce locking contention because the version dimension is actually modified during this.
############################

###################
# parameters
# pDebug = debug flag  DEFAULT to 0
# pSyncObject = the object to use the Synchronize function with.  Default = 'BooRadley' if needed
# pSubset1 = the subset name for populating consolidation element.
# pSubset2 = the subset to create for looping the in other extract processes.
####################

IF (pSyncObject @<> '');
	Synchronized(pSyncObject);
ENDIF;

NumericGlobalVariable('ProcessReturnCode');
NumericGlobalVariable('DataMinorErrorCount');



### Inititalize standard variables
strProcessName = GetProcessName();
tNow = NOW;
strNow = TIMST(NOW, '\d\h\i\s');
#strEnvironment = CELLGETS('}System - Environment', '0', 'Text');
strErrLogFolder = CELLGETS('Control Globals', 'Value', 'OutputPath');
strPrologErrLogFile = strErrLogFolder | strProcessName | '_PRO_err.log';
strMetaErrLogFile = strErrLogFolder | strProcessName | '_MET_err.log';
strDataErrLogFile = strErrLogFolder | strProcessName | '_DAT_err.log';
strEpilogErrLogFile = strErrLogFolder | strProcessName | '_EPI_err.log';
strDataFolder = CELLGETS('Control Globals', 'Value', 'DataFolderPath');
strSourcesFolder = CELLGETS('Control Globals', 'Value', 'ServerDataPath');
strRandom = SUBST(NumberToStringEx(RAND, '0.########0', '.', ''), 7, 5);
strTmp = 'tmp_' | strProcessName | '_' | strNow | '_' | strRandom;
strEmailProcess = 'eMail - Ctrl';
strUser = TM1User();
strScriptFolder = CELLGETS('Control Globals', 'Value', 'ScriptPath');
strTM1RunTI_ConfigFile = '"' | strScriptFolder | 'tm1RunTI_Config.cfg"';
cubEmail = 'eMail - Configuration';
nErrorFlag = 0;
nRecordsProcessed = 0;
nDataSourceCount = 0;
strScriptFolder = CELLGETS('Control Globals', 'Value', 'ScriptPath');
strTM1RunTI_ConfigFile = '"' | strScriptFolder | 'tm1RunTI_Config.cfg"';

### Initialize other variables

dimVersions = 'Versions';

strVersionTmpConsolidation = 'tmp_SpendPlan';

strOutputFile = strErrLogFolder | strProcessName | '_Test.txt';

                    strTimeStamp = TIMST(NOW, '\d:\h:\i:\s');
                    strOutput = EXPAND('"%strTimeStamp%  Checkpoint 1"');
                    strCommand = EXPAND('cmd /c echo %strOutput% >"%strOutputFile%"');
                    ExecuteCommand(strCommand, 0);
                    
### Main

nConsolidationExists = DIMIX(dimVersions, strVersionTmpConsolidation);
IF (nConsolidationExists = 0);
    DimensionElementInsertDirect(dimVersions, '', strVersionTmpConsolidation, 'C');
ELSE;
    nComponentCount = ELCOMPN(dimVersions, strVersionTmpConsolidation);
    i = nComponentCount;
    WHILE(i > 0);
        strComponent = ELCOMP(dimVersions, strVersionTmpConsolidation, i);
        DimensionElementComponentDeleteDirect(dimVersions, strVersionTmpConsolidation, strComponent);
        i = i - 1;
    END;
ENDIF;

                    strTimeStamp = TIMST(NOW, '\d:\h:\i:\s');
                    strOutput = EXPAND('"%strTimeStamp%  Checkpoint 2"');
                    strCommand = EXPAND('cmd /c echo %strOutput% >>"%strOutputFile%"');
                    ExecuteCommand(strCommand, 0);

# Create the first subset and insert the consolidation
IF (SubsetExists(dimVersions, pSubset1) = 1);
    SubsetDestroy(dimVersions, pSubset1);
ENDIF;
SubsetCreate(dimVersions, pSubset1);
SubsetElementInsert(dimVersions, pSubset1, strVersionTmpConsolidation, 1);

                    strTimeStamp = TIMST(NOW, '\d:\h:\i:\s');
                    strOutput = EXPAND('"%strTimeStamp%  Checkpoint 3a DataSource Subset created"');
                    strCommand = EXPAND('cmd /c echo %strOutput% >>"%strOutputFile%"');
                    ExecuteCommand(strCommand, 0);
                    
# Create the second subset of versions to populate the consolidation.
IF (SubsetExists(dimVersions, pSubset2) = 1);

                    strTimeStamp = TIMST(NOW, '\d:\h:\i:\s');
                    strOutput = EXPAND('"%strTimeStamp%  Checkpoint 3b Lookup Subset Exists"');
                    strCommand = EXPAND('cmd /c echo %strOutput% >>"%strOutputFile%"');
                    ExecuteCommand(strCommand, 0);
                    
    SubsetDestroy(dimVersions, pSubSet2);

                    strTimeStamp = TIMST(NOW, '\d:\h:\i:\s');
                    strOutput = EXPAND('"%strTimeStamp%  Checkpoint 3c Lookup Subset Destroyed"');
                    strCommand = EXPAND('cmd /c echo %strOutput% >>"%strOutputFile%"');
                    ExecuteCommand(strCommand, 0);
                    
ELSE; 
                    strTimeStamp = TIMST(NOW, '\d:\h:\i:\s');
                    strOutput = EXPAND('"%strTimeStamp%  Checkpoint 3d Subset did not exist"');
                    strCommand = EXPAND('cmd /c echo %strOutput% >>"%strOutputFile%"');
                    ExecuteCommand(strCommand, 0);
ENDIF;


strMDX = EXPAND('UNION( {[%dimVersions%].currentmember}
    , ORDER(FILTER( TM1SUBSETALL([%dimVersions%])
            , [%dimVersions%].[SpendPlan Use for Extracts] <> 0)
        , [%dimVersions%].[SpendPlan Use for Extracts]
        , BDESC)
    , ALL)');
ASCIIOUTPUT(strPrologErrLogFile, strMDX);
SubsetCreateByMDX(pSubset2, strMDX);

                    strTimeStamp = TIMST(NOW, '\d:\h:\i:\s');
                    strOutput = EXPAND('"%strTimeStamp%  Checkpoint 3e MDX Subset Created"');
                    strCommand = EXPAND('cmd /c echo %strOutput% >>"%strOutputFile%"');
                    ExecuteCommand(strCommand, 0);


SubsetElementDelete(dimVersions, pSubset2, 1);

                    strTimeStamp = TIMST(NOW, '\d:\h:\i:\s');
                    strOutput = EXPAND('"%strTimeStamp%  Checkpoint 3f MDX Subset Converted"');
                    strCommand = EXPAND('cmd /c echo %strOutput% >>"%strOutputFile%"');
                    ExecuteCommand(strCommand, 0);

nSubSize = SubsetGetSize(dimVersions, pSubset2);

                    strTimeStamp = TIMST(NOW, '\d:\h:\i:\s');
                    strOutput = EXPAND('"%strTimeStamp%  Checkpoint 4"');
                    strCommand = EXPAND('cmd /c echo %strOutput% >>"%strOutputFile%"');
                    ExecuteCommand(strCommand, 0);

# populate the Consolidation with components
i = 1;
WHILE(i <= nSubSize);
    strVersion = SubsetGetElementName(dimVersions, pSubset2, i);
    DimensionElementComponentAddDirect(dimVersions, strVersionTmpConsolidation, strVersion, 1);
    i = i + 1;
END;



                    strTimeStamp = TIMST(NOW, '\d:\h:\i:\s');
                    strOutput = EXPAND('"%strTimeStamp%  Checkpoint 5"');
                    strCommand = EXPAND('cmd /c echo %strOutput% >>"%strOutputFile%"');
                    ExecuteCommand(strCommand, 0);
and here are the log file results:
"02:12:32:07 Checkpoint 1"
"02:12:32:07 Checkpoint 2"
"02:12:32:07 Checkpoint 3a DataSource Subset created"
"02:12:32:07 Checkpoint 3d Subset did not exist"
Last edited by PavoGa on Tue Jun 06, 2017 2:13 pm, edited 1 time in total.
Ty
Cleveland, TN
User avatar
Steve Rowe
Site Admin
Posts: 2415
Joined: Wed May 14, 2008 4:25 pm
OLAP Product: TM1
Version: TM1 v6,v7,v8,v9,v10,v11+PAW
Excel Version: Nearly all of them

Re: Resolving Lock Contentions

Post by Steve Rowe »

Suggest you get your testbed onto a version that supports the TI debugger. This will tell you what lines of code are triggering the lock, ofc if you a version related issue that won't help too much....
Technical Director
www.infocat.co.uk
User avatar
PavoGa
MVP
Posts: 617
Joined: Thu Apr 18, 2013 6:59 pm
OLAP Product: TM1
Version: 10.2.2 FP7, PA2.0.9.1
Excel Version: 2013 PAW
Location: Charleston, Tennessee

Re: Resolving Lock Contentions

Post by PavoGa »

Thanks, Steve.
Ty
Cleveland, TN
Wim Gielis
MVP
Posts: 3113
Joined: Mon Dec 29, 2008 6:26 pm
OLAP Product: TM1, Jedox
Version: PAL 2.0.9.18
Excel Version: Microsoft 365
Location: Brussels, Belgium
Contact:

Re: Resolving Lock Contentions

Post by Wim Gielis »

Do you have a version of TM1 that supports temporary subsets and views ?
Best regards,

Wim Gielis

IBM Champion 2024
Excel Most Valuable Professional, 2011-2014
https://www.wimgielis.com ==> 121 TM1 articles and a lot of custom code
Newest blog article: Deleting elements quickly
BrianL
MVP
Posts: 264
Joined: Mon Nov 03, 2014 8:23 pm
OLAP Product: TM1
Version: 9.5.2 10.1 10.2 PA2
Excel Version: 2016

Re: Resolving Lock Contentions

Post by BrianL »

PavoGa wrote:All that to ask this: why is a read on a cube containing the counter dimension preventing modifications to another dimension? And by the way, this runs fine IF instead of using TM1RunTI, ExecuteProcess is used OR the environment for the -connect parameter is hard-coded to avoid reading the }System_Environment cube.

This part has an easy answer. Process B needs to IX lock (modify) the dimension. Process A has already started reading from the same dimension, so B needs to wait for A to get out of the way. Since you've used ExecuteCommand to call tm1runti.exe in a blocking manner (which is by the way counter to IBM documentation) this will never complete because there's a cycle in the wait graph. TM1 can't detect or resolve it because it doesn't know A is waiting for B to finish.

If you use ExecuteProcess there's no wait because it's all the same thread doing the work so there are no conflicts.
If you hardcode the values you're avoiding the R lock on process A so, again you've broken the cycle.


As for how to get around this, it really depends on which version of TM1 you are running. Creating a temporary subset has a lower lock footprint, but is relatively new. Even newer are PullInvalidationSubsets (10.3+ I believe) which also lower the lock footprint for subset actions.
User avatar
PavoGa
MVP
Posts: 617
Joined: Thu Apr 18, 2013 6:59 pm
OLAP Product: TM1
Version: 10.2.2 FP7, PA2.0.9.1
Excel Version: 2013 PAW
Location: Charleston, Tennessee

Re: Resolving Lock Contentions

Post by PavoGa »

Wim Gielis wrote:Do you have a version of TM1 that supports temporary subsets and views ?
10.2.2 fixpack 2.
Ty
Cleveland, TN
User avatar
PavoGa
MVP
Posts: 617
Joined: Thu Apr 18, 2013 6:59 pm
OLAP Product: TM1
Version: 10.2.2 FP7, PA2.0.9.1
Excel Version: 2013 PAW
Location: Charleston, Tennessee

Re: Resolving Lock Contentions

Post by PavoGa »

This part has an easy answer. Process B needs to IX lock (modify) the dimension. Process A has already started reading from the same dimension, so B needs to wait for A to get out of the way. Since you've used ExecuteCommand to call tm1runti.exe in a blocking manner (which is by the way counter to IBM documentation) this will never complete because there's a cycle in the wait graph. TM1 can't detect or resolve it because it doesn't know A is waiting for B to finish.
Brian, thank you for the response.

I'm not quite following you here because we are not modifying or creating a subset on the same object that appears to be causing the lock. That object is the dimension "Counter," while we are trying to modify/create a subset on "Versions." I've not had a chance, but I'll look at the URL a little later. It is completely understandable if the lock appeared to be on the same object, but that is not what is being observed.

Thanks again, to you and Wim for the responses.
Ty
Cleveland, TN
User avatar
PavoGa
MVP
Posts: 617
Joined: Thu Apr 18, 2013 6:59 pm
OLAP Product: TM1
Version: 10.2.2 FP7, PA2.0.9.1
Excel Version: 2013 PAW
Location: Charleston, Tennessee

Re: Resolving Lock Contentions

Post by PavoGa »

Suggest you get your testbed onto a version that supports the TI debugger. This will tell you what lines of code are triggering the lock, ofc if you a version related issue that won't help too much....
Steve, we downloaded this and it appears it does not work with 10.2.2 Fixpack 2 which we are on. Looking to go ahead and move to Fixpack 7.

Ty
Ty
Cleveland, TN
Wim Gielis
MVP
Posts: 3113
Joined: Mon Dec 29, 2008 6:26 pm
OLAP Product: TM1, Jedox
Version: PAL 2.0.9.18
Excel Version: Microsoft 365
Location: Brussels, Belgium
Contact:

Re: Resolving Lock Contentions

Post by Wim Gielis »

PavoGa wrote:
Wim Gielis wrote:Do you have a version of TM1 that supports temporary subsets and views ?
10.2.2 fixpack 2.
So, no, you would need TM1 10.2.2 Fix Pack 4 or higher.
Best regards,

Wim Gielis

IBM Champion 2024
Excel Most Valuable Professional, 2011-2014
https://www.wimgielis.com ==> 121 TM1 articles and a lot of custom code
Newest blog article: Deleting elements quickly
EvgenyT
Community Contributor
Posts: 324
Joined: Mon Jul 02, 2012 9:39 pm
OLAP Product: TM1
Version: PAL 2.0.8
Excel Version: 2016
Location: Sydney, Australia

Re: Resolving Lock Contentions

Post by EvgenyT »

PavoGa wrote:
This part has an easy answer. Process B needs to IX lock (modify) the dimension. Process A has already started reading from the same dimension, so B needs to wait for A to get out of the way. Since you've used ExecuteCommand to call tm1runti.exe in a blocking manner (which is by the way counter to IBM documentation) this will never complete because there's a cycle in the wait graph. TM1 can't detect or resolve it because it doesn't know A is waiting for B to finish.
Brian, thank you for the response.

I'm not quite following you here because we are not modifying or creating a subset on the same object that appears to be causing the lock. That object is the dimension "Counter," while we are trying to modify/create a subset on "Versions." I've not had a chance, but I'll look at the URL a little later. It is completely understandable if the lock appeared to be on the same object, but that is not what is being observed.

Thanks again, to you and Wim for the responses.
Because Version intersects every cell in that cube. B cant modify something that A is already reading from, else it would undermine data integrity.
EvgenyT
Community Contributor
Posts: 324
Joined: Mon Jul 02, 2012 9:39 pm
OLAP Product: TM1
Version: PAL 2.0.8
Excel Version: 2016
Location: Sydney, Australia

Re: Resolving Lock Contentions

Post by EvgenyT »

PavoGa wrote:Update:

Tried using only the DIRECT dimension manipulation functions. Still locks up, although I have further refined where it is happening. It is locking up where the dynamic subset is created. Which is a little strange because it was getting past that previously. Here is the PROLOG for the second process:

Code: Select all

############################
# Process Name: SpendPlan - Setup Version Dimension.PRO

# Ty Cardin, May 31, 2017
#
# Doing this to try and reduce locking contention because the version dimension is actually modified during this.
############################

###################
# parameters
# pDebug = debug flag  DEFAULT to 0
# pSyncObject = the object to use the Synchronize function with.  Default = 'BooRadley' if needed
# pSubset1 = the subset name for populating consolidation element.
# pSubset2 = the subset to create for looping the in other extract processes.
####################

IF (pSyncObject @<> '');
	Synchronized(pSyncObject);
ENDIF;

NumericGlobalVariable('ProcessReturnCode');
NumericGlobalVariable('DataMinorErrorCount');



### Inititalize standard variables
strProcessName = GetProcessName();
tNow = NOW;
strNow = TIMST(NOW, '\d\h\i\s');
#strEnvironment = CELLGETS('}System - Environment', '0', 'Text');
strErrLogFolder = CELLGETS('Control Globals', 'Value', 'OutputPath');
strPrologErrLogFile = strErrLogFolder | strProcessName | '_PRO_err.log';
strMetaErrLogFile = strErrLogFolder | strProcessName | '_MET_err.log';
strDataErrLogFile = strErrLogFolder | strProcessName | '_DAT_err.log';
strEpilogErrLogFile = strErrLogFolder | strProcessName | '_EPI_err.log';
strDataFolder = CELLGETS('Control Globals', 'Value', 'DataFolderPath');
strSourcesFolder = CELLGETS('Control Globals', 'Value', 'ServerDataPath');
strRandom = SUBST(NumberToStringEx(RAND, '0.########0', '.', ''), 7, 5);
strTmp = 'tmp_' | strProcessName | '_' | strNow | '_' | strRandom;
strEmailProcess = 'eMail - Ctrl';
strUser = TM1User();
strScriptFolder = CELLGETS('Control Globals', 'Value', 'ScriptPath');
strTM1RunTI_ConfigFile = '"' | strScriptFolder | 'tm1RunTI_Config.cfg"';
cubEmail = 'eMail - Configuration';
nErrorFlag = 0;
nRecordsProcessed = 0;
nDataSourceCount = 0;
strScriptFolder = CELLGETS('Control Globals', 'Value', 'ScriptPath');
strTM1RunTI_ConfigFile = '"' | strScriptFolder | 'tm1RunTI_Config.cfg"';

### Initialize other variables

dimVersions = 'Versions';

strVersionTmpConsolidation = 'tmp_SpendPlan';

strOutputFile = strErrLogFolder | strProcessName | '_Test.txt';

                    strTimeStamp = TIMST(NOW, '\d:\h:\i:\s');
                    strOutput = EXPAND('"%strTimeStamp%  Checkpoint 1"');
                    strCommand = EXPAND('cmd /c echo %strOutput% >"%strOutputFile%"');
                    ExecuteCommand(strCommand, 0);
                    
### Main

nConsolidationExists = DIMIX(dimVersions, strVersionTmpConsolidation);
IF (nConsolidationExists = 0);
    DimensionElementInsertDirect(dimVersions, '', strVersionTmpConsolidation, 'C');
ELSE;
    nComponentCount = ELCOMPN(dimVersions, strVersionTmpConsolidation);
    i = nComponentCount;
    WHILE(i > 0);
        strComponent = ELCOMP(dimVersions, strVersionTmpConsolidation, i);
        DimensionElementComponentDeleteDirect(dimVersions, strVersionTmpConsolidation, strComponent);
        i = i - 1;
    END;
ENDIF;

                    strTimeStamp = TIMST(NOW, '\d:\h:\i:\s');
                    strOutput = EXPAND('"%strTimeStamp%  Checkpoint 2"');
                    strCommand = EXPAND('cmd /c echo %strOutput% >>"%strOutputFile%"');
                    ExecuteCommand(strCommand, 0);

# Create the first subset and insert the consolidation
IF (SubsetExists(dimVersions, pSubset1) = 1);
    SubsetDestroy(dimVersions, pSubset1);
ENDIF;
SubsetCreate(dimVersions, pSubset1);
SubsetElementInsert(dimVersions, pSubset1, strVersionTmpConsolidation, 1);

                    strTimeStamp = TIMST(NOW, '\d:\h:\i:\s');
                    strOutput = EXPAND('"%strTimeStamp%  Checkpoint 3a DataSource Subset created"');
                    strCommand = EXPAND('cmd /c echo %strOutput% >>"%strOutputFile%"');
                    ExecuteCommand(strCommand, 0);
                    
# Create the second subset of versions to populate the consolidation.
IF (SubsetExists(dimVersions, pSubset2) = 1);

                    strTimeStamp = TIMST(NOW, '\d:\h:\i:\s');
                    strOutput = EXPAND('"%strTimeStamp%  Checkpoint 3b Lookup Subset Exists"');
                    strCommand = EXPAND('cmd /c echo %strOutput% >>"%strOutputFile%"');
                    ExecuteCommand(strCommand, 0);
                    
    SubsetDestroy(dimVersions, pSubSet2);

                    strTimeStamp = TIMST(NOW, '\d:\h:\i:\s');
                    strOutput = EXPAND('"%strTimeStamp%  Checkpoint 3c Lookup Subset Destroyed"');
                    strCommand = EXPAND('cmd /c echo %strOutput% >>"%strOutputFile%"');
                    ExecuteCommand(strCommand, 0);
                    
ELSE; 
                    strTimeStamp = TIMST(NOW, '\d:\h:\i:\s');
                    strOutput = EXPAND('"%strTimeStamp%  Checkpoint 3d Subset did not exist"');
                    strCommand = EXPAND('cmd /c echo %strOutput% >>"%strOutputFile%"');
                    ExecuteCommand(strCommand, 0);
ENDIF;


strMDX = EXPAND('UNION( {[%dimVersions%].currentmember}
    , ORDER(FILTER( TM1SUBSETALL([%dimVersions%])
            , [%dimVersions%].[SpendPlan Use for Extracts] <> 0)
        , [%dimVersions%].[SpendPlan Use for Extracts]
        , BDESC)
    , ALL)');
ASCIIOUTPUT(strPrologErrLogFile, strMDX);
SubsetCreateByMDX(pSubset2, strMDX);

                    strTimeStamp = TIMST(NOW, '\d:\h:\i:\s');
                    strOutput = EXPAND('"%strTimeStamp%  Checkpoint 3e MDX Subset Created"');
                    strCommand = EXPAND('cmd /c echo %strOutput% >>"%strOutputFile%"');
                    ExecuteCommand(strCommand, 0);


SubsetElementDelete(dimVersions, pSubset2, 1);

                    strTimeStamp = TIMST(NOW, '\d:\h:\i:\s');
                    strOutput = EXPAND('"%strTimeStamp%  Checkpoint 3f MDX Subset Converted"');
                    strCommand = EXPAND('cmd /c echo %strOutput% >>"%strOutputFile%"');
                    ExecuteCommand(strCommand, 0);

nSubSize = SubsetGetSize(dimVersions, pSubset2);

                    strTimeStamp = TIMST(NOW, '\d:\h:\i:\s');
                    strOutput = EXPAND('"%strTimeStamp%  Checkpoint 4"');
                    strCommand = EXPAND('cmd /c echo %strOutput% >>"%strOutputFile%"');
                    ExecuteCommand(strCommand, 0);

# populate the Consolidation with components
i = 1;
WHILE(i <= nSubSize);
    strVersion = SubsetGetElementName(dimVersions, pSubset2, i);
    DimensionElementComponentAddDirect(dimVersions, strVersionTmpConsolidation, strVersion, 1);
    i = i + 1;
END;



                    strTimeStamp = TIMST(NOW, '\d:\h:\i:\s');
                    strOutput = EXPAND('"%strTimeStamp%  Checkpoint 5"');
                    strCommand = EXPAND('cmd /c echo %strOutput% >>"%strOutputFile%"');
                    ExecuteCommand(strCommand, 0);
and here are the log file results:
"02:12:32:07 Checkpoint 1"
"02:12:32:07 Checkpoint 2"
"02:12:32:07 Checkpoint 3a DataSource Subset created"
"02:12:32:07 Checkpoint 3d Subset did not exist"
Direct functions do not remove lock contention. Only speed up the process by avoiding a full rewrite of the dimension.
User avatar
PavoGa
MVP
Posts: 617
Joined: Thu Apr 18, 2013 6:59 pm
OLAP Product: TM1
Version: 10.2.2 FP7, PA2.0.9.1
Excel Version: 2013 PAW
Location: Charleston, Tennessee

Re: Resolving Lock Contentions

Post by PavoGa »

Okay, we upgraded to FP7, got the debugger, were all smiles and happy until...

we tried to use the debugger. Apparently it does not like the statement SubsetCreateByMDX because bad things happen. Bad things, man. First it quits debugging as in the Continue, StepOver, StepIn and StepOut buttons all grey out and only the Execute button is live. It also drops all the variable and lock information in their respective windows as in goes blank. Pressing Execute starts over from the top of the current process or sub-process one may be in.

Is there, by remote chance, any documentation available for the debugger or...are we on our own?

:(
Ty
Cleveland, TN
User avatar
PavoGa
MVP
Posts: 617
Joined: Thu Apr 18, 2013 6:59 pm
OLAP Product: TM1
Version: 10.2.2 FP7, PA2.0.9.1
Excel Version: 2013 PAW
Location: Charleston, Tennessee

Re: Resolving Lock Contentions

Post by PavoGa »

Note to self: Do not start the debugger from the JAR file. Start it from the BAT file. That is all.

All is good in east Tennessee.
Last edited by PavoGa on Fri Jun 16, 2017 12:00 pm, edited 1 time in total.
Ty
Cleveland, TN
User avatar
PavoGa
MVP
Posts: 617
Joined: Thu Apr 18, 2013 6:59 pm
OLAP Product: TM1
Version: 10.2.2 FP7, PA2.0.9.1
Excel Version: 2013 PAW
Location: Charleston, Tennessee

Re: Resolving Lock Contentions

Post by PavoGa »

Because Version intersects every cell in that cube. B cant modify something that A is already reading from, else it would undermine data integrity.
That would be correct if it were true in this case. The Versions is not a dimension in either of the two cubes being read before the modification. We are not attempting a modification on the Counter dimension which is what is being reported as the blocking object by the debugger.

I suspect the problem is most likely a rule that uses the version dimension for one of those cubes. Have not had a chance to test. When I do, I'll respond if that resolves it.
Ty
Cleveland, TN
Post Reply