Manage Rights when a big chunk of elements are deleted

Post Reply
Paul-TM1
Posts: 64
Joined: Tue Jun 13, 2017 3:20 pm
OLAP Product: TM1
Version: 10.2.2
Excel Version: 2010

Manage Rights when a big chunk of elements are deleted

Post by Paul-TM1 » Mon Feb 25, 2019 7:47 pm

Hi all,
In 10.2.2, when ever I delete a chunk of elements from the "Approval hierarchy" dimension and apply manage Rights, it takes 4-5 hours to complete the process in multi threaded configuration.

Code: Select all

 <deployment maxThreads="4"/>
Normal manage rights with 4 threads takes about an hour. What happens in this 4-5 hours is a mystery. Can someone please explain the long process? What is different?

In PA 2.0.6, I deleted some elements and started manage rights and it's running since 3 days. I see the same processes running and it's not hung. Can any one suggest what could be done?
"}tp_get_top_node"
}tp_util_is_in_subset"
"}tp_get_parent_in_subset"


Thanks,
Paul.

User avatar
Elessar
Regular Participant
Posts: 162
Joined: Mon Nov 21, 2011 12:33 pm
OLAP Product: TM1
Version: 10.2.2
Excel Version: 2010
Location: Russia

Re: Manage Rights when a big chunk of elements are deleted

Post by Elessar » Wed Feb 27, 2019 8:21 am

Hi,

How long does it take to do a SecurityRefresh on your server?
Sorry for my English ;)

Paul-TM1
Posts: 64
Joined: Tue Jun 13, 2017 3:20 pm
OLAP Product: TM1
Version: 10.2.2
Excel Version: 2010

Re: Manage Rights when a big chunk of elements are deleted

Post by Paul-TM1 » Wed Feb 27, 2019 4:45 pm

Hi Elessar,
Thanks for the reply.
It takes about an hour for SecurityRefresh normally. On PA 2.0.4 and 2.0.6, it takes about 1.5 hours. Since it was taking longer, I asked IBM and they gave us an interim fix and that does not complete securityRefresh. It stops before completing with an error as below.

[Default Executor-thread-22303] ERROR (getApplicationSecurity) com.ibm.cognos.pmpsvc.PMPService - getApplicationSecurity connection lost exception raised.

Any help is appreciated.
Thanks.

lotsaram
MVP
Posts: 3239
Joined: Fri Mar 13, 2009 11:14 am
OLAP Product: TableManager1
Version: PA 2.0.x
Excel Version: Office 365
Location: Switzerland

Re: Manage Rights when a big chunk of elements are deleted

Post by lotsaram » Wed Feb 27, 2019 9:44 pm

Paul-TM1 wrote:
Wed Feb 27, 2019 4:45 pm
It takes about an hour for SecurityRefresh normally. On PA 2.0.4 and 2.0.6, it takes about 1.5 hours.
Oh my!
I have worked on some pretty huge models (and talking security not just data; thousands of users, thousands of groups) but I have never seen SecurityRefresh take longer than about 5 - 10 minutes. In most models, even very large ones, it is considerably less.

A very good argument for getting rid of the Contributor application.
Please place all requests for help in a public thread. I will not answer PMs requesting assistance.

User avatar
ykud
Posts: 52
Joined: Sat Jan 10, 2009 10:52 am
Contact:

Re: Manage Rights when a big chunk of elements are deleted

Post by ykud » Thu Feb 28, 2019 12:59 am

Paul-TM1 wrote:
Mon Feb 25, 2019 7:47 pm
Hi all,
In 10.2.2, when ever I delete a chunk of elements from the "Approval hierarchy" dimension and apply manage Rights, it takes 4-5 hours to complete the process in multi threaded configuration.

Code: Select all

 <deployment maxThreads="4"/>
Normal manage rights with 4 threads takes about an hour. What happens in this 4-5 hours is a mystery. Can someone please explain the long process? What is different?
In my experience, the longest part is usually the }tp_rights_update_RDCLS_static_rights process that does a really huge loop over the groups / approval nodes / control dimension intersections and copies the rule calculated rights value to a 'static' slice. Because it does it for every cube in your application and you maybe have a lot of groups / approval elements / elements in control dimension -- it will take ages to scan through it.
The parallel processing 'chunks' this process into parts, but doesn't really change the overall approach.

You can decrease the search space by pruning unnecessary groups / control elements.

You can also get hacky and optimise this process to look at only the applicable group / control dimension element subset for applications to reduce the time quite a bit. For example, looping through only the applicable 100 groups for an application instead of a 1000 in total would speed up the execution by a factor of 10 :) But this is a bit of an exercise in coding and you'd be changing the 'out of the box' process.

Cheers,
Y

User avatar
ykud
Posts: 52
Joined: Sat Jan 10, 2009 10:52 am
Contact:

Re: Manage Rights when a big chunk of elements are deleted

Post by ykud » Thu Feb 28, 2019 1:07 am

Paul-TM1 wrote:
Wed Feb 27, 2019 4:45 pm
Hi Elessar,
Thanks for the reply.
It takes about an hour for SecurityRefresh normally. On PA 2.0.4 and 2.0.6, it takes about 1.5 hours. Since it was taking longer, I asked IBM and they gave us an interim fix and that does not complete securityRefresh. It stops before completing with an error as below
Really try and see if you can remove some of the user groups you're having, it will make everything a lot faster :)
And it might be worth looking into setting PrivilegeGenerationOptimization to T and writing feeders for Cell Security rules, it improves cell security rule processing time.

User avatar
Elessar
Regular Participant
Posts: 162
Joined: Mon Nov 21, 2011 12:33 pm
OLAP Product: TM1
Version: 10.2.2
Excel Version: 2010
Location: Russia

Re: Manage Rights when a big chunk of elements are deleted

Post by Elessar » Thu Feb 28, 2019 7:16 am

Cellsecurity cubes are calculated each time when user reads data, they do not affect SecurityRefresh.

1,5h is really long for SecurityRefresh. Do you have rules in ElementSecurity cubes for large dimensions? There is a good practice to update ElementSecurity cubes for dimensions with >500 elements using TI, so that their rules will not be calculated during SecurityRefresh.

And another thing: how large is your }tp_jobs dimension? It can also affect application maintenance time:
https://www-01.ibm.com/support/docview. ... wg21987687
Last edited by Elessar on Sun Mar 03, 2019 4:37 pm, edited 1 time in total.
Sorry for my English ;)

Wim Gielis
MVP
Posts: 2052
Joined: Mon Dec 29, 2008 6:26 pm
OLAP Product: TM1
Version: PAL 2.0.7
Excel Version: Office 365 - latest
Location: Brussels, Belgium
Contact:

Re: Manage Rights when a big chunk of elements are deleted

Post by Wim Gielis » Thu Feb 28, 2019 10:25 am

Also you can have Cell Security cubes with less dimensions than the original cubes. Maybe this helps as well.
Best regards,

Wim Gielis

Excel Most Valuable Professional, 2011-2014
http://www.wimgielis.com ==> 109 TM1 articles and a lot of custom code
Newest blog article: Set up a TM1 service with AutoHotKey

Paul-TM1
Posts: 64
Joined: Tue Jun 13, 2017 3:20 pm
OLAP Product: TM1
Version: 10.2.2
Excel Version: 2010

Re: Manage Rights when a big chunk of elements are deleted

Post by Paul-TM1 » Fri Mar 01, 2019 9:58 pm

Thanks for all your ideas Ykud, Elessar, Lotsaram and Wim G.
I tried to run the ideas presented by Elessar and it did not get a positive result. May be a server restart might provide. I will be trying other ideas as I proceed.

Thanks again for your inputs.

Paul.

Catherine
Posts: 103
Joined: Wed May 20, 2009 7:30 am
OLAP Product: TM1
Version: 10.2.2 - PA
Excel Version: 2010
Location: Rennes, France

Re: Manage Rights when a big chunk of elements are deleted

Post by Catherine » Thu Jun 06, 2019 1:33 pm

Wim Gielis wrote:
Thu Feb 28, 2019 10:25 am
Also you can have Cell Security cubes with less dimensions than the original cubes. Maybe this helps as well.
Hi Wim,
Please, could you tell me more about this ?
From which version is it possible ? I saw it was possible through Performance modeler... In 10.2.2, using Perspectives, when I right click on a cube to implement cell security, it creates a control cube with all dimensions + }Groups dimensions. I have no choice regarding dimensions.
Thanks in advance

David Usherwood
Site Admin
Posts: 1357
Joined: Wed May 28, 2008 9:09 am

Re: Manage Rights when a big chunk of elements are deleted

Post by David Usherwood » Thu Jun 06, 2019 2:52 pm

It's the TI command CellSecurityCubeCreate, supported in 10.2.something onwards:

Code: Select all

https://www.tm1forum.com/viewtopic.php?f=3&t=11423&p=54152#p54152

Catherine
Posts: 103
Joined: Wed May 20, 2009 7:30 am
OLAP Product: TM1
Version: 10.2.2 - PA
Excel Version: 2010
Location: Rennes, France

Re: Manage Rights when a big chunk of elements are deleted

Post by Catherine » Thu Jun 06, 2019 4:00 pm

Thank you very much !
I will test it very soon.

Post Reply