ASCIIAPPEND

Suggest and discuss enhancements for TM1
Post Reply
User avatar
Steve Vincent
Site Admin
Posts: 1048
Joined: Mon May 12, 2008 8:33 am
OLAP Product: TM1
Version: 10.2.2 FP1
Excel Version: 2010
Location: UK

ASCIIAPPEND

Post by Steve Vincent » Wed Dec 10, 2008 4:18 pm

Something that i'd like to do is append data to an existing ascii file from a TI, but current options do not allow this. Even writing to the same file from the same TI from different tabs will overwrite the existing file.

My solution would be to add another command called ASCIIAPPEND, to have the same criteria as ASCIIOUTPUT but adds to the bottom of the file rather than overwriting it. If the file doesn't exist, it should create it.

If others believe that to be a worthwhile addition i'm quite happy to raise a request to Cognos. I can't see it being too difficult, so it'll probably take as long as undo spread has... :lol:
If this were a dictatorship, it would be a heck of a lot easier, just so long as I'm the dictator.
Production: TM1 64 bit 10.2.2, Windows 2008/2012 Server. Excel 2010, IE11 for t'internet

User avatar
Mike Cowie
Site Admin
Posts: 437
Joined: Sun May 11, 2008 7:07 pm
OLAP Product: TM1, MSAS
Version: Anything thru 11.x
Excel Version: 2003 - 2016
Location: Alabama, USA
Contact:

Re: ASCIIAPPEND

Post by Mike Cowie » Wed Dec 10, 2008 4:41 pm

Hi Steve,

It would be nice to see some better log file management - maybe even a constant/variable (like ASCIIOutputAppend=1;) that let you switch on append for ASCIIOUTPUT written files for the duration of a process rather than a separate function...

To tide you over in the interim, you can write out each tab's log to a separate file and combine them with a DOS command in a separate process or (if you don't care about adding in an Epilog log portion) in the Epilog. From an old Applix forum thread (http://applixforum.olapforums.com/viewP ... eadID=6929:

Code: Select all

ExecuteCommand('CMD /C Copy Prolog.cma+Metadata.cma+Data.cma CompleteLog.cma',1);


Hopefully you get the general idea... not perfect, but it can get you to a single custom log file in the end.

Regards,
Mike

User avatar
Steve Vincent
Site Admin
Posts: 1048
Joined: Mon May 12, 2008 8:33 am
OLAP Product: TM1
Version: 10.2.2 FP1
Excel Version: 2010
Location: UK

Re: ASCIIAPPEND

Post by Steve Vincent » Thu Dec 11, 2008 11:14 am

Thanks Mike, in my current case i need to combine files that will be created daily but i can deal with that using the same code in a different way. Its useful to have that option of combining output from tab in a single TI tho, thanks :)
If this were a dictatorship, it would be a heck of a lot easier, just so long as I'm the dictator.
Production: TM1 64 bit 10.2.2, Windows 2008/2012 Server. Excel 2010, IE11 for t'internet

User avatar
Alan Kirk
Site Admin
Posts: 5725
Joined: Sun May 11, 2008 2:30 am
OLAP Product: TM1
Version: 9.5.2 64 bit moving to 10.2.2
Excel Version: 2010
Location: Sydney, Australia
Contact:

Re: ASCIIAPPEND

Post by Alan Kirk » Wed Jun 29, 2011 10:14 pm

Steve Vincent wrote:Something that i'd like to do is append data to an existing ascii file from a TI, but current options do not allow this. Even writing to the same file from the same TI from different tabs will overwrite the existing file.

My solution would be to add another command called ASCIIAPPEND, to have the same criteria as ASCIIOUTPUT but adds to the bottom of the file rather than overwriting it. If the file doesn't exist, it should create it.

If others believe that to be a worthwhile addition i'm quite happy to raise a request to Cognos. I can't see it being too difficult, so it'll probably take as long as undo spread has... :lol:
Steve, I came across a situation where this would have been handy a day or two back. Have you raised it?

The only down side is what would happen if another process has a write lock on the file. My bet is that this is why we have the current situation where each tab creates its own file; the advantage is that since the tab creates the file itself, there's no possibility that the file could be locked.

My feeling is that the workaround for this chould be an argument in the AsciiAppend function which lets us tell the server what to do if it encounters such a write lock on the file that it's trying to append to:
0 = Ignore and continue processing (default)
1 = Fail the process with an error
2 = Create a new file with a timestamped name and write to that instead of the intended file.

Of course this would require a break away from the Applix mindset of "an error crashes the process even if we don't need or want it to". (See also, creating 256+ character strings in 9.0 which crash the process as opposed to just being ignored as they were in 8.2.12.)

The process should, of course, open the file with its own write lock.
"To them, equipment failure is terrifying. To me, it’s 'Tuesday.' "
-----------
Before posting, please check the documentation, the FAQ, the Search function and FOR THE LOVE OF GLUB the Request Guidelines.

lotsaram
MVP
Posts: 3006
Joined: Fri Mar 13, 2009 11:14 am
OLAP Product: TM1, CX
Version: TM1 10.2.2 PA 2.0x
Excel Version: 2010 2013 365
Location: Switzerland

Re: ASCIIAPPEND

Post by lotsaram » Thu Jun 30, 2011 7:16 am

I hadn't noticed this thread before but an ASCIIAppend function would certainly have my vote. The current workaround of declaring a separate test file for each tab of a process where using custom logging then having a batch file concatenate the files together is a long way from optimal.

User avatar
Alan Kirk
Site Admin
Posts: 5725
Joined: Sun May 11, 2008 2:30 am
OLAP Product: TM1
Version: 9.5.2 64 bit moving to 10.2.2
Excel Version: 2010
Location: Sydney, Australia
Contact:

Re: ASCIIAPPEND

Post by Alan Kirk » Wed Jul 20, 2011 10:50 pm

Alan Kirk wrote:
Steve Vincent wrote:Something that i'd like to do is append data to an existing ascii file from a TI, but current options do not allow this. Even writing to the same file from the same TI from different tabs will overwrite the existing file.

My solution would be to add another command called ASCIIAPPEND, to have the same criteria as ASCIIOUTPUT but adds to the bottom of the file rather than overwriting it. If the file doesn't exist, it should create it.

If others believe that to be a worthwhile addition i'm quite happy to raise a request to Cognos. I can't see it being too difficult, so it'll probably take as long as undo spread has... :lol:
Steve, I came across a situation where this would have been handy a day or two back. Have you raised it?
{Knock-knock} Steve? Anyone home? {Knock-knock, peers through gap in the curtains} Steve? Y'there?
"To them, equipment failure is terrifying. To me, it’s 'Tuesday.' "
-----------
Before posting, please check the documentation, the FAQ, the Search function and FOR THE LOVE OF GLUB the Request Guidelines.

User avatar
mattgoff
MVP
Posts: 505
Joined: Fri May 16, 2008 1:37 pm
OLAP Product: TM1
Version: 10.2.2.6
Excel Version: 2016
Location: Florida, USA
Contact:

Re: ASCIIAPPEND

Post by mattgoff » Thu Jul 21, 2011 5:29 pm

I like it. I jump through a lot of hoops w/o this capability. Technically I would prefer a flag added to the ASCIIOutput function a la Perl e.g.

read-only: open FILEHANDLE, "filename.txt"
read-write: open FILEHANDLE, ">filename.txt"
append: open FILEHANDLE, ">>filename.txt"
Please read and follow the Request for Assistance Guidelines. It helps us answer your question and saves everyone a lot of time.

mincharug.shulft
Posts: 125
Joined: Fri May 20, 2011 8:17 am
OLAP Product: Applix,Cognos TM!
Version: applix9.0Cognos tm1 9.5.1
Excel Version: Excel 2010 2007

Re: ASCIIAPPEND

Post by mincharug.shulft » Fri Jun 01, 2012 4:15 am

Steve ,
i have only one ASCIIOUTPUT under the prolog tab
and as soon as TI excutes asciioutout is overwriting the file.
BUT what i want it is Once TI excutes i have to add the new data into the existed txt file.is it possible?
and also i have seen is asciiappend=1 parameter so where i have to add this parameter either in cfg or..?
and otherhand
by walking through all gentle man posts and help guide....
tried using excutecommand but i cannot rechable.

actually my ascii as it was thereb in the prolog tab.
asciioutput('para.log',str_date | ' ' |user_cube);
asciioutput('para.log',str_date | ' '|cube1);
asciioutput('para.log',str_date |' '|cube2);
and soon TI runs its ganarating the para.log into database directory.
and when next time TI runs its overwriting the file and content of data as well.
but i want it is when next time TI runs -the new data and old data sould be there in the para.log.
please give your aid?

winsonlee
Regular Participant
Posts: 180
Joined: Thu Jul 01, 2010 3:06 am
OLAP Product: Cognos Express
Version: 9.5
Excel Version: 2007
Location: Melbourne, Australia

Re: ASCIIAPPEND

Post by winsonlee » Fri Jun 01, 2012 5:03 am

You can use dos to append the new logfile to an existing log file.

ExecuteCommand('cmd /c type para.log >> paraTemp.log',0);

User avatar
Harvey
Community Contributor
Posts: 210
Joined: Mon Aug 04, 2008 4:43 am
OLAP Product: TM1, CX, Palo
Version: 9.4+
Excel Version: 2003+
Contact:

Re: ASCIIAPPEND

Post by Harvey » Wed Nov 14, 2012 1:18 pm

Logging is an area that the new product I'm working on, Flow Responder, would fit nicely.

I think we've all had pain points in this area, but if you guys have any specific needs and requirements, please let me know and I'll create a logging plugin for the Flow Responder. This is a good opportunity to get in on the ground level and make sure I cover all the functionality on your wish list.

Feel free to get me privately if you prefer, at team@flowolap.com.
Flow OLAP - adding value to Cognos Partners worldwide.

User avatar
Alan Kirk
Site Admin
Posts: 5725
Joined: Sun May 11, 2008 2:30 am
OLAP Product: TM1
Version: 9.5.2 64 bit moving to 10.2.2
Excel Version: 2010
Location: Sydney, Australia
Contact:

Re: ASCIIAPPEND

Post by Alan Kirk » Wed Dec 17, 2014 10:38 pm

I've raised this in the DeveloperWorks RFE group as request 63589. You need to be logged into DeveloperWorks to see it but if you have an account and want this, please do log in and vote for it if you want it too.

Description
A problem occurs when you want to write an output file which is either:
- Sourced from multiple tabs of a TI process;
- Sourced from multiple TI processes; or
- Both.

Each TI tab (procedure) will create and write any file that it is told to when using AsciiOutput / TextOutput. That means that if you write a file in the Prolog of process 1, then want to write to the same file in the Metadata tab then the Metadata tab output will overwrite the Prolog tab's output.

The same is true if you try to output to the same file in the Data tab, the Epilog tab, or a subsequent process.

In many cases we want only a single file of output. Traditionally the only approach to this has been to output to separate files with different filenames and then use (for example) the command line Copy command to merge the files into one. This is awkward and unwieldy especially if there will be an unknown number of output files, depending on the conditions that exist at runtime.

This could be overcome if in addition to AsciiOutput and TextOutput we had AsciiAppend and TextAppend functions which would allow the new output to be appended to the end of an existing file.

As there would be the danger of the file being locked by another process I would suggest that there would need to be a function parameter to specify what to do in such a case:

0 = Ignore and continue processing (default)
1 = Fail the process with an error
2 = Create a new file with a timestamped name and write to that instead of the intended file.
User Case
There is a range of situations that users have come across where this is needed. One is where you want to create a single audit log for a range of processes. Another is where you want to create a single output file from multiple data sources (cubes in some cases, views in another).

Given the limitations on output sort order from views, for example, in some cases the only way to get the output into particular orders is to create multiple views and export them separately. As things stand this requires outputting to multiple files and combining them.

The specifics will vary from site to site but the need is wide ranging.
Business justification
Without this function we need to rely on unreliable and cumbersome workarounds to be able to unite output into a single file. It unnecessarily consumes disk space by having many separate files on the disk system, and introduces unnecessary complexity by having the combination process reliant on code which exists outside of the TM1 server. This could result in incomplete files which may have an impact on the recipients of those files.
"To them, equipment failure is terrifying. To me, it’s 'Tuesday.' "
-----------
Before posting, please check the documentation, the FAQ, the Search function and FOR THE LOVE OF GLUB the Request Guidelines.

User avatar
Alan Kirk
Site Admin
Posts: 5725
Joined: Sun May 11, 2008 2:30 am
OLAP Product: TM1
Version: 9.5.2 64 bit moving to 10.2.2
Excel Version: 2010
Location: Sydney, Australia
Contact:

Re: ASCIIAPPEND

Post by Alan Kirk » Fri Apr 10, 2015 12:17 pm

Joyous news, boys and girls, I just noticed that this request was last updated on the 7th of this month and now has a status of "Planned for Future Release".

How far in the future is unknown of course, but when it finally does get here (OK, I'll be Optimistic Ollie for a moment), I'll be the first to be patting Iboglix on the back for it.
"To them, equipment failure is terrifying. To me, it’s 'Tuesday.' "
-----------
Before posting, please check the documentation, the FAQ, the Search function and FOR THE LOVE OF GLUB the Request Guidelines.

Post Reply