Ascii Quote Fun and Games

Post Reply
User avatar
jim wood
Site Admin
Posts: 3678
Joined: Wed May 14, 2008 1:51 pm
OLAP Product: TM1
Version: TM1 10.2.2
Excel Version: 2007
Location: 1639 Route 10, Suite 107, Parsippany, NJ, USA
Contact:

Ascii Quote Fun and Games

Post by jim wood » Tue Feb 05, 2019 3:02 pm

Guys,

I'm trying to move awkward data between cubes and this includes free comment fields. The fields contain all kinds of crap so I can't use the usual delimiters as I'm exporting it to file first. I have therefore gone for the Ascii quote character set as Char(186) which then looks like this:
Quote.PNG
Quote.PNG (329 Bytes) Viewed 306 times
The process exports the data to file as expected without issue. The same process then loads the data in to the other cube. The problem is that the process doesn't seem to like the quote character. I get the following errors:
QuoteError.PNG
QuoteError.PNG (3 KiB) Viewed 306 times
I have made sure that the source ascii quote is set correctly yet the process still doesn't seem to like it.

Any idea why guys?

Jim.
Struggling through the quagmire of life to reach the other side of who knows where.
Application Consulting Group (ACG) TM1 Consulting
OS: Windows 10 64-bit. TM1 Version: 10.2.2

User avatar
jim wood
Site Admin
Posts: 3678
Joined: Wed May 14, 2008 1:51 pm
OLAP Product: TM1
Version: TM1 10.2.2
Excel Version: 2007
Location: 1639 Route 10, Suite 107, Parsippany, NJ, USA
Contact:

Re: Ascii Quote Fun and Games

Post by jim wood » Tue Feb 05, 2019 3:37 pm

Looks like it may have something to do with the extended Ascii character set. Looking at something else and will post back.
Struggling through the quagmire of life to reach the other side of who knows where.
Application Consulting Group (ACG) TM1 Consulting
OS: Windows 10 64-bit. TM1 Version: 10.2.2

User avatar
jim wood
Site Admin
Posts: 3678
Joined: Wed May 14, 2008 1:51 pm
OLAP Product: TM1
Version: TM1 10.2.2
Excel Version: 2007
Location: 1639 Route 10, Suite 107, Parsippany, NJ, USA
Contact:

Re: Ascii Quote Fun and Games

Post by jim wood » Tue Feb 05, 2019 3:58 pm

It was 100% because I was using an extended Ascii character 0186. Once I lowered it to character 94 everything worked as expected. It's interesting that the process itself recognized the extended character but the data tab part of the process did not,

Jim.
Struggling through the quagmire of life to reach the other side of who knows where.
Application Consulting Group (ACG) TM1 Consulting
OS: Windows 10 64-bit. TM1 Version: 10.2.2

Wim Gielis
MVP
Posts: 2011
Joined: Mon Dec 29, 2008 6:26 pm
OLAP Product: TM1
Version: PAL 2.0.6
Excel Version: 2016 - Office 365
Location: Brussels, Belgium
Contact:

Re: Ascii Quote Fun and Games

Post by Wim Gielis » Tue Feb 05, 2019 4:39 pm

I have seen strange cases in the past with extended Ascii characters and try to stay away from there.
Which includes French accents like é, è, ê too. Not sure if this is still an issue nowadays, I am talking about some 5 years ago.
Best regards,

Wim Gielis

Excel Most Valuable Professional, 2011-2014
http://www.wimgielis.com ==> 109 TM1 articles and a lot of custom code
Newest blog article: Active form claustrophobia

User avatar
jim wood
Site Admin
Posts: 3678
Joined: Wed May 14, 2008 1:51 pm
OLAP Product: TM1
Version: TM1 10.2.2
Excel Version: 2007
Location: 1639 Route 10, Suite 107, Parsippany, NJ, USA
Contact:

Re: Ascii Quote Fun and Games

Post by jim wood » Tue Feb 05, 2019 5:24 pm

Wim Gielis wrote:
Tue Feb 05, 2019 4:39 pm
I have seen strange cases in the past with extended Ascii characters and try to stay away from there.
Which includes French accents like é, è, ê too. Not sure if this is still an issue nowadays, I am talking about some 5 years ago.
We're still on 10.2.2 so it may have been fixed since.
Struggling through the quagmire of life to reach the other side of who knows where.
Application Consulting Group (ACG) TM1 Consulting
OS: Windows 10 64-bit. TM1 Version: 10.2.2

User avatar
paulsimon
MVP
Posts: 683
Joined: Sat Sep 03, 2011 11:10 pm
OLAP Product: TM1
Version: PA 2.0.5
Excel Version: 2016
Contact:

Re: Ascii Quote Fun and Games

Post by paulsimon » Tue Feb 05, 2019 8:29 pm

Hi Jim

You may already be doing this, but in the past when I had to write different languages (including chinese) to files and read them back, I did the following :

In the Prolog get the char set

Code: Select all

vCharSet =  CellGetS( vInfoCube, 'Any', 'Character Set' ) ;
When writing to a file in the Data Tab you need something that will track the very first record being written :

Code: Select all

IF( vDataRecCount = 1 ) ;

  # Before outputting anything to the main file, 
  # we need to set the character set immediately before
  # the first output statement.

  SetOutputCharacterSet( vPathFile , vCharSet ) ;
  textoutput( vPathFile , 'Element' , 'Attr'  , 'OldVal' , 'NewVal' ) ;

ENDIF;

When reading from a file in the Prolog you need something like

Code: Select all

SetInputCharacterSet( vCharSet ) ;
Regards

Paul Simon

Andy Key
MVP
Posts: 339
Joined: Wed May 14, 2008 1:37 pm
OLAP Product: TM1 CX
Version: 2-5 to 10-2-x
Excel Version: 2003 2007 2010 2013
Location: Sydney
Contact:

Re: Ascii Quote Fun and Games

Post by Andy Key » Wed Feb 06, 2019 3:07 am

If you're only writing the data out to a file so you can read it back in again, won't SetOutputEscapeDoubleQuote cover it? No need to use anything other than the standard double quote, although you can still choose something else if you want, it will double up whatever character you have as the ASCIIQuoteCharacter.

You're still stuck with the joys of things like embedded CR/LF though...
Andy Key

Post Reply