Parallel run vs. ICX

Post Reply
vladino
Posts: 110
Joined: Sat Nov 06, 2010 10:10 am
OLAP Product: Cognos TM1
Version: 10.2.2
Excel Version: Excel 2013

Parallel run vs. ICX

Post by vladino »

Hi guys,
I'm trying to run several TI processes in parallel - sometimes I'm successful but sometimes not. Here's the thing...

I have one source cube and one target cube. The structure is nearly the same and I need to get the data out of the source cube and transfer it into the target one. For some purposes I'm exporting the data into a text file and then the text file is imported into the target cube.

To make it faster I would like to leverage parallel run of TI processes. I have (nearly) the same source views (there's difference only for ORG dim - ie. for each ORG unit I have a separate view). Then I have created adequate TI processes and I would like to run all these TI processes in the same time.

As far as I know there shouldn't be any IXC lock on both source and target cubes because each TI process is reading/writing from/to a separate ORG unit, right?

Sometimes it works and I can see several TI processes running at the same time. But sometimes there's only one TI process running and others are waiting in IXC state for the previous one to be finished.

Can anyone explain this behavior? Why am I getting IXC locks if each TI process is reading/writing from/to a separate ORG unit.

Thanks in advance!

BR
Vladino

EDIT What about temp subsets? If there are several processes running in parallel and all are trying to create a dynamic temp subset with the same name? Could this be the issue???
lotsaram
MVP
Posts: 3651
Joined: Fri Mar 13, 2009 11:14 am
OLAP Product: TableManager1
Version: PA 2.0.x
Excel Version: Office 365
Location: Switzerland

Re: Parallel run vs. ICX

Post by lotsaram »

All object names need to be unique. For parallel processing from/to the same cube it is important that all views and all subsets be uniquely named.
Please place all requests for help in a public thread. I will not answer PMs requesting assistance.
vladino
Posts: 110
Joined: Sat Nov 06, 2010 10:10 am
OLAP Product: Cognos TM1
Version: 10.2.2
Excel Version: Excel 2013

Re: Parallel run vs. ICX

Post by vladino »

lotsaram wrote: Thu Mar 01, 2018 6:27 am All object names need to be unique. For parallel processing from/to the same cube it is important that all views and all subsets be uniquely named.
Even if all views are using the same subset for products, measures etc?
It's not the same cube, I'm just exporting the data from the cube to the text file using Bedrock View export to File.
lotsaram
MVP
Posts: 3651
Joined: Fri Mar 13, 2009 11:14 am
OLAP Product: TableManager1
Version: PA 2.0.x
Excel Version: Office 365
Location: Switzerland

Re: Parallel run vs. ICX

Post by lotsaram »

vladino wrote: Thu Mar 01, 2018 6:51 am
lotsaram wrote: Thu Mar 01, 2018 6:27 am All object names need to be unique. For parallel processing from/to the same cube it is important that all views and all subsets be uniquely named.
Even if all views are using the same subset for products, measures etc?
It's not the same cube, I'm just exporting the data from the cube to the text file using Bedrock View export to File.
If it's not the same cube then view names could be the same.
If dimensions are common even if cubes aren't then the subsets need to be uniquely named if there is any manipulation (editing) of the subsets. You could use the same subset name where all the TI processes don't touch or edit the subset in any way, just use it. But it is much safer th just make sure that any temp subsets for processing are done with unique names.
Please place all requests for help in a public thread. I will not answer PMs requesting assistance.
Drg
Regular Participant
Posts: 159
Joined: Fri Aug 12, 2016 10:02 am
OLAP Product: tm1
Version: 10.2.0 - 10.3.0
Excel Version: 2010

Re: Parallel run vs. ICX

Post by Drg »

vladino wrote: Wed Feb 28, 2018 10:40 pm
As far as I know there shouldn't be any IXC lock on both source and target cubes because each TI process is reading/writing from/to a separate ORG unit, right?
dear friend, let's imagine a situation where you open an architect and run one of your processes. after you run another architect and try to log in and execute the second process.
what result will you get?
vladino
Posts: 110
Joined: Sat Nov 06, 2010 10:10 am
OLAP Product: Cognos TM1
Version: 10.2.2
Excel Version: Excel 2013

Re: Parallel run vs. ICX

Post by vladino »

Drg wrote: Thu Mar 01, 2018 1:51 pm dear friend, let's imagine a situation where you open an architect and run one of your processes. after you run another architect and try to log in and execute the second process.
what result will you get?
I guess it depends on what is the purpose of both processes? :)
vladino
Posts: 110
Joined: Sat Nov 06, 2010 10:10 am
OLAP Product: Cognos TM1
Version: 10.2.2
Excel Version: Excel 2013

Re: Parallel run vs. ICX

Post by vladino »

lotsaram wrote: Thu Mar 01, 2018 1:18 pm If it's not the same cube then view names could be the same.
If dimensions are common even if cubes aren't then the subsets need to be uniquely named if there is any manipulation (editing) of the subsets. You could use the same subset name where all the TI processes don't touch or edit the subset in any way, just use it. But it is much safer th just make sure that any temp subsets for processing are done with unique names.
OK so in other words - if I'm using any already existing and non-temp subset I don't need to duplicate this subset with unique name? And all I need is to create a subset with unique name if this subset is changed within any TI process?

Or is there any other reason for IXC locks?
Drg
Regular Participant
Posts: 159
Joined: Fri Aug 12, 2016 10:02 am
OLAP Product: tm1
Version: 10.2.0 - 10.3.0
Excel Version: 2010

Re: Parallel run vs. ICX

Post by Drg »

vladino wrote: Thu Mar 01, 2018 2:31 pm
Drg wrote: Thu Mar 01, 2018 1:51 pm dear friend, let's imagine a situation where you open an architect and run one of your processes. after you run another architect and try to log in and execute the second process.
what result will you get?
I guess it depends on what is the purpose of both processes? :)
so check that this!
is not a long time to try to run waits or logn execution process(maybe empty while(i<100000)) from two different architects.
In fact, everything is not as fun as using the concept of user thread.

If to be absolutely simple to you for detouring of locks I would advise to launch processes under different users. :roll:
Drg
Regular Participant
Posts: 159
Joined: Fri Aug 12, 2016 10:02 am
OLAP Product: tm1
Version: 10.2.0 - 10.3.0
Excel Version: 2010

Re: Parallel run vs. ICX

Post by Drg »

I do not know the nature of such locks, I guess only that this is due to the presentation of data at a certain point in time
those.
in the moment you read of the cube, somebody put data into this cube but the reading is not blocked and the record is not blocked either, and the data you read is actually obsolete but not for your session. And perhaps the session time is tied to the user and not to a specific authorization. which leads to the occurrence of locks.

Below is a description of the post where I came across a similar problem
http://tm1forum.com/viewtopic.php?f=3&t=13857
PeteB
Posts: 23
Joined: Sun Sep 23, 2012 2:30 am
OLAP Product: Planning Analytics Local, PAW
Version: PAL 2.0.6, PAW 2.0.45
Excel Version: 2010

Re: Parallel run vs. ICX

Post by PeteB »

Hi Everyone reading this thread,

Firstly apologies if you have read a similar comment on other very similar treads. It is very likely we each have had a similar problem, which can be fixed by IBM in a future release. I am told by IBM this should be a trivial piece of work to deliver the copy function from a slice of rules cells to value cells and the performance should be significantly faster for TM1 to internalise the copy rather than the current approach of exporting the rule values and importing into the snapshot version including across cubes.

Please vote for an RFE for IBM to provide the function to copy rule based cells to values.
https://www.ibm.com/developerworks/rfe/ ... _ID=116669 - TM1 Data Snapshot functionality - Copy rule to values in a TM1 cube

IBM has commented they intend to decline the request as functionality already exists, but I am hoping with enough votes they will reconsider their position.
User avatar
Steve Rowe
Site Admin
Posts: 2410
Joined: Wed May 14, 2008 4:25 pm
OLAP Product: TM1
Version: TM1 v6,v7,v8,v9,v10,v11+PAW
Excel Version: Nearly all of them

Re: Parallel run vs. ICX

Post by Steve Rowe »

Hi PeteB,

Please don't use other posts to drum up support for your RFE, we have a specific forum to post details of RFE in (http://www.tm1forum.com/viewforum.php?f=19).

If we allow this behaviour posts will become a mess of the OPs topic and people boosting their RFEs.

Many thanks,
Technical Director
www.infocat.co.uk
BrianL
MVP
Posts: 264
Joined: Mon Nov 03, 2014 8:23 pm
OLAP Product: TM1
Version: 9.5.2 10.1 10.2 PA2
Excel Version: 2016

Re: Parallel run vs. ICX

Post by BrianL »

This sounds like a good use case for the TI debugger. If you're running 10.2.2 FP7 or newer you can set a breakpoint in the debugger on the IX lock acquisition. This should help you understand what TI code is causing the lock.
Post Reply