Setting numerous cell values using TM1 Java API

[Ohgo]

Greetings!

We have developed a Java application to write cube data using the TM1 Java API. The application receives inputs in XML Format, parse them into TM1Val arrays, and repeatedly calls setCellValue() to write the values to a cube. This way the data in TM1 can be synchronized real time.

However as the input data grows larger, the array building and multiple calls to setCellValue() takes longer proportionally. I am looking for a way to optimize this process. Is there a way in the Java API to send values to a TM1 cube in batch? I may have overlooked a solution here so any suggestion is highly appreciated.

Thanks in advance.

[yyi]

Don't know how to batch send cell values. If it's slower than t.i. have you tried parsing to a table array and call tm1Chore.execute()

[ykud]

How many cells are talking about when data gets “larger”? SetCellValue is generally quite fast, I've run 100 parallel sessions banging data into planning sample without any “slowing” down.

”Slowing” usually starts when you get calculations / locking happening at the same time. Try running tm1top — does it show anything?

[Harvey]

If you are running a client application using the API, it has to go through a network layer to communicate with the server process.

If it is running on a different machine, the speed difference will be obvious, but even if your program is running on the server, it can still never be as fast as TI, as it communicates via a network socket, so each packet of data you send has a certain overhead.

Turbo integrator does not need to do this, as its code resides in the TM1 server process.

Your best bet to optimize the process is to create a CSV or other kind of text file and trigger Turbo Integrator to read it via the API. That is the only way you'll be able to get TI-like performance.

[Ohgo]

Hi yyi, lazarus,
Yes I am aware that TI has a much better perfomance. Unfortunately in my case creating flat files is not an option, so I am looking for another way to speed this thing up.

Hi ykud,
Thanks for sharing your experience. The data could get as large as tens thousands of cell values.. and we were trying to insert them into a cube of around 20 dimensions. I don't know if it is normal but last time it was not done in 15 minutes.

I end up cleaning up logging and doing other optimization to the Java app. While this isn't ideal and using TI, batch commands, and flat files is a lot faster, we'll try to live with it

Thanks for your suggestions.

[ykud]

Hi ykud,
Thanks for sharing your experience. The data could get as large as tens thousands of cell values.. and we were trying to insert them into a cube of around 20 dimensions. I don't know if it is normal but last time it was not done in 15 minutes.

Should be cube design issue \ rules \ calculations. I've simulated a lot larger loads, shouldn't be Java API problem.

Check with a simple csv file and a TI — if it'll be equally slow, then Java is not the bottleneck.