Greetings!
We have developed a Java application to write cube data using the TM1 Java API. The application receives inputs in XML Format, parse them into TM1Val arrays, and repeatedly calls setCellValue() to write the values to a cube. This way the data in TM1 can be synchronized real time.
However as the input data grows larger, the array building and multiple calls to setCellValue() takes longer proportionally. I am looking for a way to optimize this process. Is there a way in the Java API to send values to a TM1 cube in batch? I may have overlooked a solution here so any suggestion is highly appreciated.
Thanks in advance.
Setting numerous cell values using TM1 Java API
Re: Setting numerous cell values using TM1 Java API
Don't know how to batch send cell values. If it's slower than t.i. have you tried parsing to a table array and call tm1Chore.execute()
Yeon
Re: Setting numerous cell values using TM1 Java API
How many cells are talking about when data gets “larger”? SetCellValue is generally quite fast, I've run 100 parallel sessions banging data into planning sample without any “slowing” down.
”Slowing” usually starts when you get calculations / locking happening at the same time. Try running tm1top — does it show anything?
”Slowing” usually starts when you get calculations / locking happening at the same time. Try running tm1top — does it show anything?
- Harvey
- Community Contributor
- Posts: 236
- Joined: Mon Aug 04, 2008 4:43 am
- OLAP Product: PA, TM1, CX, Palo
- Version: TM1 8.3 onwards
- Excel Version: 2003 onwards
- Contact:
Re: Setting numerous cell values using TM1 Java API
If you are running a client application using the API, it has to go through a network layer to communicate with the server process.
If it is running on a different machine, the speed difference will be obvious, but even if your program is running on the server, it can still never be as fast as TI, as it communicates via a network socket, so each packet of data you send has a certain overhead.
Turbo integrator does not need to do this, as its code resides in the TM1 server process.
Your best bet to optimize the process is to create a CSV or other kind of text file and trigger Turbo Integrator to read it via the API. That is the only way you'll be able to get TI-like performance.
If it is running on a different machine, the speed difference will be obvious, but even if your program is running on the server, it can still never be as fast as TI, as it communicates via a network socket, so each packet of data you send has a certain overhead.
Turbo integrator does not need to do this, as its code resides in the TM1 server process.
Your best bet to optimize the process is to create a CSV or other kind of text file and trigger Turbo Integrator to read it via the API. That is the only way you'll be able to get TI-like performance.
Take your TM1 experience to the next level - TM1Innovators.net
-
- Posts: 2
- Joined: Mon Sep 03, 2012 11:27 am
- OLAP Product: TM1
- Version: 9.5.2 10.1
- Excel Version: 2003 2007 2010
Re: Setting numerous cell values using TM1 Java API
Hi yyi, lazarus,
Yes I am aware that TI has a much better perfomance. Unfortunately in my case creating flat files is not an option, so I am looking for another way to speed this thing up.
Hi ykud,
Thanks for sharing your experience. The data could get as large as tens thousands of cell values.. and we were trying to insert them into a cube of around 20 dimensions. I don't know if it is normal but last time it was not done in 15 minutes.
I end up cleaning up logging and doing other optimization to the Java app. While this isn't ideal and using TI, batch commands, and flat files is a lot faster, we'll try to live with it
Thanks for your suggestions.
Yes I am aware that TI has a much better perfomance. Unfortunately in my case creating flat files is not an option, so I am looking for another way to speed this thing up.
Hi ykud,
Thanks for sharing your experience. The data could get as large as tens thousands of cell values.. and we were trying to insert them into a cube of around 20 dimensions. I don't know if it is normal but last time it was not done in 15 minutes.
I end up cleaning up logging and doing other optimization to the Java app. While this isn't ideal and using TI, batch commands, and flat files is a lot faster, we'll try to live with it
Thanks for your suggestions.
Re: Setting numerous cell values using TM1 Java API
Should be cube design issue \ rules \ calculations. I've simulated a lot larger loads, shouldn't be Java API problem.Ohgo wrote: Hi ykud,
Thanks for sharing your experience. The data could get as large as tens thousands of cell values.. and we were trying to insert them into a cube of around 20 dimensions. I don't know if it is normal but last time it was not done in 15 minutes.
Check with a simple csv file and a TI — if it'll be equally slow, then Java is not the bottleneck.