how tm1 locks work at the time the view is read

Post Reply
Drg
Posts: 84
Joined: Fri Aug 12, 2016 10:02 am
OLAP Product: tm1
Version: 10.2.0 - 10.3.0
Excel Version: 2010

how tm1 locks work at the time the view is read

Post by Drg » Wed Feb 07, 2018 8:45 am

colleagues today faced with the most interesting fact of mutual locks, a collective help is necessary in the correct understanding of the logic of TM1.

So to the point: I have a parent process that starts parralel processes that in turn create temporary views and write them to the database.

Code: Select all

Prarent_process
    parse cube configuration and RunTi Child_Process (no wait execution)          
    Wait wheen all runti executed(use file flags created child_process) - to control the count of running processes in moment 

Child_Process
    Prolog
       create file flag (start)
       Create temp view createview(cube,viewname, 1 - temp)
    Data
       add data to javaextension array
       push javaextension array to SAP hana use jdbc.jar
    Epilog
       push less data   
       Delete file flag (end)               
almost always everything works as it should, but there are moments when deadlocks occur.
I have it appeared 1-2 times on a productive environment and I did not catch their log of locks.
Here's what I have:
TM1TOP snapshot
2801 Exp - Run:R POST /api/v1/Processes('SYS_EXPORT_DATA_TO_HANA_DB')/tm1.Execute - 1656
67192 Exp TM1RunTI Run:R ProcessExecuteEx Process SYS_HANA_BATCH_INSERT - 40
55940 Exp TM1RunTI Run:R ProcessExecuteEx Process SYS_HANA_BATCH_INSERT - 70
36208 Exp TM1RunTI Run:R - Client KHD_1454_FB1 - 0
37832 Exp TM1RunTI Run:R ProcessExecuteEx Process SYS_HANA_BATCH_INSERT Data (22) 76
40956 Exp TM1RunTI Run:R ProcessExecuteEx Process SYS_HANA_BATCH_INSERT Data (62) 744
27112 Exp TM1RunTI Wait:IXCur - Client KHD_1454_FB1 SYS_HANA_BATCH_INSERT:Unknown (0), WaitingOn (3788) 1400
59784 Exp TM1RunTI Run:R ProcessExecuteEx Process SYS_HANA_BATCH_INSERT - 62
52612 Exp TM1RunTI Run:R ProcessExecuteEx Process SYS_HANA_BATCH_INSERT - 300
then I see deadlock on thread 27112 type Wait:IXCur and I know who exactly we are waiting for WaitingOn (3788).
but there(3788) are no such threads in TM1TOP and not when there was (I'm logging user activity). this is the first thing that scared me, phantom thread O_o !

TM1Server.log
In the log file of the server, I found this thread, this system thread runs the Prarent_process
as I noticed the thread with the number I need is a system number, it starts the parent process
3788 INFO 17:56 TM1.Process Process "PARENT_PROCESS" executed by user "KHD"
27112 INFO 17:56 TM1.Process Process "SYS_LOG_EXECUTE_PROCESS" executed by user "KHD"
27112 INFO 17:56 TM1.Process Process "CHILD_PROCESS" run from process "SYS_LOG_EXECUTE_PROCESS" by user "KHD"
27112 INFO 17:56 TM1.TILogOutput START UNLOAD DATA BY SESSION:PCKG212208726_D425954
Here, the process has to read the view and start unloading, but when trying to read the presentation, the system decided to complete the dependencies
27112 INFO 17:57 TM1.Cube.Dependency Adding cube dependency: Cube 'КОНС_Журнал' depends on cube '}ElementAttributes_УПС'.
27112 INFO 18:00 TM1.Cube.Dependency Adding cube dependency: Cube 'КОНС_Журнал' depends on cube 'ФВ_Тр_Выплаты'.
27112 INFO 18:00 TM1.Cube.Dependency Adding cube dependency: Cube 'КОНС_Журнал' depends on cube 'ФВ_Выплаты'.
27112 INFO 18:01 TM1.Cube.Dependency Adding cube dependency: Cube 'КОНС_Баланс' depends on cube 'ФВ_ДБП'.
at some point the system tries to build a dependency in a cube(КОНС_Баланс) that has tried to read and unload it into this one and there is not an deadlock.
after a very long wait I kill the process by windows process explorer( RunTI.exe with thread number in tm1 27112)
3788 INFO 18:39 TM1.Process Process "PARENT_PROCESS": finished executing normally, elapsed time 2571.96 seconds
after which the system calmly completes the dependencies and completes the thread
27112 INFO 18:39 TM1.Cube.Dependency Adding cube dependency: Cube 'КОНС_Баланс' depends on cube 'ОТЧ_Баланс'.
27112 INFO 18:41 TM1.Cube.Dependency Adding cube dependency: Cube 'КОНС_Баланс' depends on cube 'АДМ_Курсы_валют'.
27112 INFO 18:41 TM1.Cube.Dependency Adding cube dependency: Cube 'КОНС_Баланс' depends on cube '}ElementAttributes_БП'.
27112 INFO 18:41 TM1.Cube.Dependency Adding cube dependency: Cube 'КОНС_Баланс' depends on cube '}ElementAttributes_Год'.
27112 INFO 18:42 TM1.Process Process "CHILD_PROCESS": finished executing normally, elapsed time 2759.95 seconds
27112 INFO 18:42 TM1.Process Process "SYS_LOG_EXECUTE_PROCESS": finished executing normally, elapsed time 2759.96 seconds
This raises several questions that I would like to clarify with the community:
Whether correctly I understand that if I try to open for the first time a view of a cube which uses a rule with IF() and DB() and ATTRS().
Does the system, if necessary, take out the dependency structure?
Are there any assumptions about how this can be done before reading the view process?

I tried to make the creation of the view into a separate process!
I use the ViewConstrict() function until the view is assigned to the process.
P.S>
I understand that it is unlikely to get an answer because I have very complicated logic. But all your ideas or theory will be useful to me, do not be shy.

User avatar
Steve Rowe
Site Admin
Posts: 1800
Joined: Wed May 14, 2008 4:25 pm
OLAP Product: TM1
Version: 10.2.2., PAW
Excel Version: Nearly all of them

Re: how tm1 locks work at the time the view is read

Post by Steve Rowe » Wed Feb 07, 2018 9:43 am

When working on this type of issue previously there were two types of locks that we had to manage in addition to getting the TI process correct.

One is cube dependency for rules, the engine builds most of these when the server starts however it seems to miss out those related to attribute cubes. When the engine encounters a dependency that has not been built a lock is put on the system.
=> Use the TI command AddCubeDependency to add the relationship, you can set this TI up to run when the server starts and then never worry about it again. Note that the help is confusing on this command, from memory the argument list is reversed. You might have to update this TI several times while the engine discovers new locks, there are pretty easy to pick up from the log file though, as you have found.

The other was users logging off the system, in some releases when a user logs off all the private objects that were used in the session are written back to the server (even if no explicit request is made), this creates a lock as well. There is a cfg to fix this behaviour but unfortunately I can't remember what it is or find it in the documentation, hopefully someone else can post it...
HTH

Post Reply