HelpSpot Help Desk Software | HelpSpot Blog | HelpSpot Support

Euro symbol and other characters


#1

Hi, we installed the demo version of help spot, we worked on it for some days to be able to interface it with our site with good results, thanks to your API. We have already decided to buy a license when the trial period is over, in the meantime we are working on some issues we are still facing. The most problematic one is the euro symbol: whether I send an email or I type it inside the response ticket, it gets garbled (replaced by a question mark) or disappears from the output, either inside your app or retrieving the response from the APIs. Interestingly, the character is present in the database table, so it gets saved, but when retrieved it is lost. Other characters happens to have similar problems, but the Euro is very important to us, since we do business (only) in this currency. So, help on this matter will be really appreciated!


#2

Hi Pier,

Unfortunately the symbol can’t be displayed due to problems with PHP and UTF8. We’re going to be upgrading HelpSpot later this year to work around these issues (these changes probably won’t be available until year end).

Until then you could use the word “euro” of course. In things like email templates you could try the HTML entity €


#3

Hi, this is very unfortunate for us, since when the users write to us and use such symbols, their email gets garbled, resulting both difficult to read for the operator that has to answer the request and give a bad impression about our care for the customer, if the ticket history shows question marks instead of what the user wrote. We are not only talking about the euro, but also various hypens, and other wide used european (mostly french) symbols, which cannot of course be replaced so easily.

I have to say that I don’t get what is this “php have problems with utf8” claim you make. I am a php developer, I write my application from the ground up to work with utf8, and I never get any such problems. I see your database doesn’t specify utf8 as his collation, instead is left with default swedish encoding, and that might be the beginning of the problems, as I read in other posts on this forum. What I really don’t understand, though, is why the euro symbol is correctly saved to the database when inserted through your app “edit ticket” panel, but it doesn’t get “retrieved”: your app and your apis simply don’t show it, it doesn’t even get converted in some other entity or in the question mark… so you can save it but cannot print it back, sounds really strange to me!

If, instead, I update a ticket via APIs with iso-8859-1 encoded data, the euro symbol simply gets replaced with a question mark.

If you can offer no help about this issue, I am considering to rewrite the email parser, creating my own version in order to be able to replace characters on the incoming emails that would end up garbled before converting them to tickets. The major issue here is that I don’t see a way to add attachments to the ticket updates through the apis, only notes seems to be supported. Is this correct? Is it possible, in this case, to interact directly with the database and can you give advice with the queries involved?

regards


#4

Hi Pier,

PHP has many problems with UTF8 data. A list of unsafe or potentially unsafe string functions can be found here: http://www.phpwact.org/php/i18n/utf-8

There’s lots more information available via google as well.

There are now some workarounds and some things in later PHP versions have made it easier to deal with or at least work around the PHP UTF8 issues. HelpSpot was originally created back when PHP 4.3 was the most recent build, also HelpSpot uses various libraries which are not UTF8 safe (primarily because they use the above string functions).

HelpSpot has the additional complexity of being run against different database system as well as in environments we don’t control so often mbstring or iconv may not be available for instance.

In a future release we’re going to use the modern work arounds and rewrite the libraries that are not UTF8 safe, but for now it cannot be supported.

If you have MBString installed the French characters should come through fine, we have many customers in France.

The API does support attachments via the parameters:
File#_sFilename
File#_sFileMimeType
File#_bFileBody

You can read more about those parameters at the bottom of this page:
http://www.helpspot.com/helpdesk/index.php?pg=kb.page&id=157


#5

When you say

If you have MBString installed the French characters should come through fine, we have many customers in France.

you mean with function overloading activated? because I have mbstring but not overload activated, and the behaviour is what I described. I will try now to activate overloading. Have you already got success cases with the euro simbol in incoming mails?


#6

No, not with overloading. You should be able to handle them in the regular iso-8859-1 charset I believe, though you could switch to another latin characters set if you think that will improve it.

The euro symbol isn’t part of the latin characters sets, just UTF8 so there’s no way to pull it in successfully at this time.


#7

That’s not entirely true, ie iso 8859-9 or 8859-15 do support the euro symbol http://www.cs.tut.fi/~jkorpela/latin9.html

So I am willing to try to change the encoding to one of the aforementioned and see if I can solve this issue this way. So, what I need to do in order to switch encoding?


#8

Sure, all you need to do to switch the encoding is in /helpspot/lang/english-us.php in the top is the HelpSpot character set. There’s also a few other items that may need changing at the top, but the most important is the HelpSpot character set.

You will need to keep this change between releases.

Your DB collation may need to be adjusted as well, but I wouldn’t do that until you’re sure you need to.


#9

Changing the charset to:
define(‘helpspot_charset’,‘iso-8859-15’);

didn’t resolved enything, but changing tidycharset to utf8:
define(‘lg_tidycharset’,‘utf8’);

seems to solve the issue with the euro character. This would explain why it disappeared instead of become garbled, it was ignored since unrecognised by the tidy library.
I will test more, have you already experience with changing this setting?


#10

That should be OK. Hard to say for sure, we’ve not tested it like that but just keep an eye on it and you can change it back if needed.


#11

Thanks for your help, I think I solved every major aspect of the integration with our systems. Regards