TM1 + SPSS

[MSidat]

Hi,

Just wanted to ask if anyone has any experience with working with SPSS and TM1 and how fluid is it in automating the sending of some data into SPSS and then pulling back into TM1 the output of some trend data.

[BariAbdul]

It might not be exactly what you have been looking for ,But just heads up ,Might give you some clues:
http://bit.ly/1R5bMov
http://bit.ly/1K00sbT Thanks

[MSidat]

Thanks Abdul.

Will have a look through the links although at first glance seems like quite a manual process in terms of the 2 system working together in a prod environment.

Would still be interested in hearing from anyone who have managed to used SPSS as part of a TM1 Process and automating the feeds between the systems.

[George Regateiro]

We have had a couple projects attempt to integrate with SPSS with varying results. The problems we have found are

1) The entire integration is based around text files. This just does not scale along with other issues
2) They rely completely on public objects
3) The writeback TIs do not work well since you have data at many levels of the hierarchy
4) The pieces do not communicate. TM1 kicking off SPSS and then TM1 knowing SPSS is done just does not happen well

Right now we are working on writing our own integration, the basic premise is

1) API app pulls from TM1
2) Results written to SQL DB
3) Stored Proc calls SPSS
4) API App waits for the results and does post processing


This allows us to be able to show more real time interaction between TM1 and SPSS while drastically increasing performance. Leveraging SPSS through sql has shown a 10x performance boost over the same data set through a text file. This is still in the early stages but it looks like this is the way we will go when it comes to SPSS TM1 integration.

[David Usherwood]

Interesting. Where's the bottleneck with text files? TI reads flat files very fast indeed.
What about using the new(ish) ExecuteJava(n/s) feature in 10.2+? Or (probably not so relevant) the REST/ODATA API?
Mind you, it would be preferable if the owner of the products built and supported the connection tools, wouldn't you agree?

[George Regateiro]

Interesting. Where's the bottleneck with text files? TI reads flat files very fast indeed.
What about using the new(ish) ExecuteJava(n/s) feature in 10.2+? Or (probably not so relevant) the REST/ODATA API?
Mind you, it would be preferable if the owner of the products built and supported the connection tools, wouldn't you agree?

TI is not the issue. The main issue with flat files is how they are handled within SPSS. In my current use case of 40k records the SPSS processing goes from 30 seconds to 3 seconds just by switching to a SQL DB. The other issue you have is with updates. In our case we are running analysis for individual plants while aggregating them, with flat files we have to update the entire file where with sql we can target the updates to just the plant that was resubmitting. The flat file solution for the integration works for small data sets but just does not scale very well.

Also at the total company level multi gig text files become a pain to manage and or read over the network.

A proper integration would be nice. There should be noting stopping the two systems from talking together better then it can talk to any other data source.

[BrianL]

We have had a couple projects attempt to integrate with SPSS with varying results. The problems we have found are

1) The entire integration is based around text files. This just does not scale along with other issues...

While I've never used SPSS, I thought IBM introduced direct support for connecting to TM1 in SPSS Modeller 16. From general noise I've heard at work, the support appears to be more mature in the newer v17 or even 17.1 releases of SPSS Modeller.

[George Regateiro]

We have had a couple projects attempt to integrate with SPSS with varying results. The problems we have found are

1) The entire integration is based around text files. This just does not scale along with other issues...

While I've never used SPSS, I thought IBM introduced direct support for connecting to TM1 in SPSS Modeller 16. From general noise I've heard at work, the support appears to be more mature in the newer v17 or even 17.1 releases of SPSS Modeller.

I am also not the SPSS person but from what I have seen so far is even their direct integration was really still just exporting the data under the hood and not leveraging the cubes directly. I will certainly ask my SPSS person to ensure they are on the latest of the releases. As much as I would like the challenge of what I am doing I would prefer to let IBM handle it.