Page 1 of 2

Slow Login in Architect

Posted: Fri Jun 07, 2019 12:14 pm
by macsir
Tm1 10.2.2 fp7
I noticed that for around two weeks, it will sometimes become very slow to login from Architect or Perspectives or Web. The Architect I was testing just sits on the TM1 server itself. Thus there shouldn't be any network issue. After a reboot, the duration of login went back to normal (less than 10 seconds). I am aware that the login duration is pretty much depend on the number of TM1 objects residing in the server.
However, after a period of time (days or weeks), it goes up to around 20 seconds, or 40 seconds or even 60 seconds randomly.
I checked the logs in every place and seems to be normal. No special findings. The only thing I can tell is when it happened, the server has many (~90-100) sessions concurrently although most of them are idle.
Could anyone share any idea on it? Thanks.

Re: Slow Login in Architect

Posted: Fri Jun 07, 2019 1:31 pm
by Edward Stuart
Which IntegratedSecurityMode are you using and are you using single sign on etc..?

Re: Slow Login in Architect

Posted: Fri Jun 07, 2019 4:18 pm
by gtonkin
Is there a difference when you use the server name vs the IP address? Have had numerous issues when the DNS is fritzed - it takes forever to connect the dots.

Re: Slow Login in Architect

Posted: Fri Jun 07, 2019 10:04 pm
by macsir
Edward Stuart wrote:
Fri Jun 07, 2019 1:31 pm
Which IntegratedSecurityMode are you using and are you using single sign on etc..?
Cam security mode 5 with sso. The cognos is installed on same server where TM1 is. Thanks.

Re: Slow Login in Architect

Posted: Fri Jun 07, 2019 10:05 pm
by macsir
gtonkin wrote:
Fri Jun 07, 2019 4:18 pm
Is there a difference when you use the server name vs the IP address? Have had numerous issues when the DNS is fritzed - it takes forever to connect the dots.
I tried using IP in architect. No difference and still slow. Thanks

Re: Slow Login in Architect

Posted: Mon Jun 10, 2019 12:11 pm
by Edward Stuart
Once you are logged in are there further issues or is it only login that sees the delays?

Do you have access to Cognos BI Security to see if anything odd is occuring at that end? My guess would be that the 'normal' identifiers for TM1 are clear so it could be a BI related issue?

Re: Slow Login in Architect

Posted: Mon Jun 10, 2019 8:55 pm
by macsir
Edward Stuart wrote:
Mon Jun 10, 2019 12:11 pm
Once you are logged in are there further issues or is it only login that sees the delays?

Do you have access to Cognos BI Security to see if anything odd is occuring at that end? My guess would be that the 'normal' identifiers for TM1 are clear so it could be a BI related issue?
Thanks, no further slowness issue after logged in as I observed.
I checked BI side and no special finding. What I can see from TM1 top is, after double clicking the server, it took around 5 seconds for showing "??? - SystemServerConnectWithCAM" and then after around 4 seconds, my logging name is shown correctly. After that, TM1 server took 1 or 2 seconds to show all objects. Therefore, the total duration is around 10 seconds now after a fresh reboot several days ago. Is it quite normal if someone can test it as well? Thanks. I don't think it is slow. When the issue really happened last time, the server took ages to show all objects after authenticating my user name with CAM.

Re: Slow Login in Architect

Posted: Mon Jun 10, 2019 8:56 pm
by macsir
By the way, TM1 web was also slow when I had the issue. Now it takes around 15 seconds to show applications and views after open the URL in Chrome.

Re: Slow Login in Architect

Posted: Tue Jun 11, 2019 11:34 am
by gtonkin
Anything suspicious in your tm1admsrv log files?

May be worthwhile adding some additional loggers in your tm1s-log.properties to see if there is anything untoward going on e.g.

Code: Select all

log4j.logger.TM1.Server.Network=DEBUG, STATS
log4j.appender.STATS=org.apache.log4j.SharedMemoryAppender
log4j.appender.STATS.MemorySize=5 MB
log4j.appender.STATS.File=tm1STATS.log
log4j.appender.STATS.MaxFileSize=5 MB
log4j.appender.STATS.MaxBackupIndex=5
log4j.appender.STATS.Timezone=Local
Have a look here for more loggers.

Re: Slow Login in Architect

Posted: Tue Jun 11, 2019 11:52 pm
by macsir
gtonkin wrote:
Tue Jun 11, 2019 11:34 am
Anything suspicious in your tm1admsrv log files?

May be worthwhile adding some additional loggers in your tm1s-log.properties to see if there is anything untoward going on e.g.

Code: Select all

log4j.logger.TM1.Server.Network=DEBUG, STATS
log4j.appender.STATS=org.apache.log4j.SharedMemoryAppender
log4j.appender.STATS.MemorySize=5 MB
log4j.appender.STATS.File=tm1STATS.log
log4j.appender.STATS.MaxFileSize=5 MB
log4j.appender.STATS.MaxBackupIndex=5
log4j.appender.STATS.Timezone=Local
Have a look here for more loggers.
Thanks, that's a good point. I will take action on them and report back.

Re: Slow Login in Architect

Posted: Wed Jun 19, 2019 3:54 am
by macsir
Just want to share,
The login duration was slowed down to around 20 seconds and I checked the DEBUG server log. There is no ERROR or WARNING at all.
Compared to a login from fresh reboot, the log is showing more

Code: Select all

105416   []   DEBUG   2019-06-19 05:59:55.385   TM1.Event   mt_WaitForMultipleObjects: Timed out.
85292   []   DEBUG   2019-06-19 05:59:55.385   TM1.Event   mt_WaitForMultipleObjects: Timed out.
28796   []   DEBUG   2019-06-19 05:59:55.385   TM1.Event   mt_WaitForSingleObject: Timed out on event 0x0000000000000144.
33076   []   DEBUG   2019-06-19 05:59:55.385   TM1.Event   mt_WaitForMultipleObjects: Timed out.
28796   []   DEBUG   2019-06-19 05:59:55.385   TM1.Event   mt_WaitForSingleObject: Waiting for event 0x0000000000000144.
71340   []   DEBUG   2019-06-19 05:59:55.385   TM1.Event   mt_WaitForMultipleObjects: Timed out.
28796   []   DEBUG   2019-06-19 05:59:55.400   TM1.Event   mt_WaitForSingleObject: Timed out on event 0x0000000000000144.
28796   []   DEBUG   2019-06-19 05:59:55.400   TM1.Event   mt_WaitForSingleObject: Waiting for event 0x0000000000000144.
28796   []   DEBUG   2019-06-19 05:59:55.416   TM1.Event   mt_WaitForSingleObject: Timed out on event 0x0000000000000144.
Not sure what that mean but seems those timeout events were consuming considerable time. Maybe I am wrong as it is out of my technical scope. But for now, the most practical workaround would be rebooting server weekly rather than monthly. :?

Re: Slow Login in Architect

Posted: Wed Jun 19, 2019 6:33 am
by Elessar
Hi,

When you log in to TM1 with ISM=5, TM1 checks CAM client*group mapping (and adds new client if any). This can be reason

How many groups do you have in TM1?
How many groups does the "slow" user belong to?
Do you have any rules in }Clientgroups cube? Any feeders from it?
Does the problem affect all users?

Re: Slow Login in Architect

Posted: Wed Jun 19, 2019 8:05 pm
by macsir
Elessar wrote:
Wed Jun 19, 2019 6:33 am
Hi,

When you log in to TM1 with ISM=5, TM1 checks CAM client*group mapping (and adds new client if any). This can be reason

How many groups do you have in TM1?
How many groups does the "slow" user belong to?
Do you have any rules in }Clientgroups cube? Any feeders from it?
Does the problem affect all users?
1. Around 70 groups but not all of them have members
2. when it slows, all groups I tested are slow
3. No
4. all users I know including myself, admin.

Thanks.

Re: Slow Login in Architect

Posted: Thu Jun 20, 2019 6:38 am
by Elessar
There should be "CAM_LPSvr.exe" processes. Do they consume CPU during slow user connections?
It can consume the whole core (for example, 25% in 4-core processor). If so, try to kill it in Task manager. Cognos will recreate new, if it needs.

And the most obvious (sorry if I didn't find this above ;) ): did you try to restart Cognos service?

Re: Slow Login in Architect

Posted: Thu Jun 20, 2019 8:00 pm
by macsir
Elessar wrote:
Thu Jun 20, 2019 6:38 am
There should be "CAM_LPSvr.exe" processes. Do they consume CPU during slow user connections?
It can consume the whole core (for example, 25% in 4-core processor). If so, try to kill it in Task manager. Cognos will recreate new, if it needs.

And the most obvious (sorry if I didn't find this above ;) ): did you try to restart Cognos service?
I did search "CAM_LPSvr.exe" and nothing special on the consumption.
I will try to restart Cognos service next time to see if it was improving.

Re: Slow Login in Architect

Posted: Mon Jun 24, 2019 5:31 pm
by Bakkone
Do you use SSL? If so, between which components?

Is BI and TM1 installed on the same server? Is TM1 installed on one or multiple servers?

Do you have any anti-virus on the servers?

Would it be possible for you to test with native login once to see if you still if its still slow?

Re: Slow Login in Architect

Posted: Mon Jun 24, 2019 10:49 pm
by macsir
Bakkone wrote:
Mon Jun 24, 2019 5:31 pm
Do you use SSL? If so, between which components?

Is BI and TM1 installed on the same server? Is TM1 installed on one or multiple servers?

Do you have any anti-virus on the servers?

Would it be possible for you to test with native login once to see if you still if its still slow?
1.Yes, UseSSL=T but CAM is not using SSL, means it is HTTP rather than HTTPS
2.Yes
3.Yes but seems not to be relevant
4.Good point, I will try next time.
Thanks.

Re: Slow Login in Architect

Posted: Tue Jun 25, 2019 2:22 pm
by Bakkone
macsir wrote:
Mon Jun 24, 2019 10:49 pm
1.Yes, UseSSL=T but CAM is not using SSL, means it is HTTP rather than HTTPS
2.Yes
3.Yes but seems not to be relevant
4.Good point, I will try next time.
Thanks.
1, Are you using custom certificates?

2a, Are you using IIS as a webserver for your CAM-installation?
2b, Is the database server for the BI-installation on the same machine?

3, AV can be the death of everything. Would it be possible to turn it off when things are slow to see if things are faster?

Whats the server OS?
What version of Cognos BI?
What Database are you using for your content store?

Re: Slow Login in Architect

Posted: Wed Jul 03, 2019 8:09 pm
by macsir
1, Are you using custom certificates?
Yes, tm1ca_v2.pem
2a, Are you using IIS as a webserver for your CAM-installation?
Yes
2b, Is the database server for the BI-installation on the same machine?
DB Server is another server

3, AV can be the death of everything. Would it be possible to turn it off when things are slow to see if things are faster?
Has tried disabled but no improvement

Whats the server OS?
Windows 2012
What version of Cognos BI?
10.2.2
What Database are you using for your content store?
SQL Server

Re: Slow Login in Architect

Posted: Wed Jul 17, 2019 9:41 pm
by macsir
Just want to update, looks like the login duration is stable and fast after cleaning lots of obsolete Performance Modeler additions out. Can't say it is 100% the reason but the login period is indeed improved. The cleaning is still in progress.