Something is wrong with forum text encoding

[polarbreeze]

I have tried all available text encoding options on the Mac in Safari, Firefox and Internet Explorer (via Parallels) - none of them render the forum text properly, for example the C-cedille in the head of this topic. What I see ends up being something like this:

Screen Shot 2012-05-05 at 7.06.16 AM.png (10.8 KiB) Viewed 2269 times

[Denkster]

Hi Polarbreeze,

Probably, an amateur moved the forum to the new platform.
While setting up the database and exporting-importing the database content, neglected to preserve character encoding of the database management system and data in the migration process.
Probably the DBMS renders UTF-8 coding as ISO-8xxx .

Sigh..

Eveline

[polarbreeze]

... neglected to preserve character encoding of the database management system and data in the migration process...

Oh, that will be hard to fix globally won't it. But at least they should correct the more visible errors, like the forum headings for example...

[q]

Looks like this has strewn ugly stuff throughout the forum. Here's but one example, where I think some curly quotes and apostrophes were pasted in.

http://www.gvox.com/forum/viewtopic.php?f=18&t=3040#p17720

q

Hi Polarbreeze,

Probably, an amateur moved the forum to the new platform.
While setting up the database and exporting-importing the database content, neglected to preserve character encoding of the database management system and data in the migration process.
Probably the DBMS renders UTF-8 coding as ISO-8xxx .

Sigh..

Eveline

[Andre_B]

I thought first something had to be fixed "globally" so that the forum would accept the non-US characters, but - having done an append in French - it looks like the site or the forum perfectly accepts these, but that the data were once corrupted.
Every subscriber could edit and correct his own appends - though some of mine were done with a userid which I lost.

[Hotch]

Hi,

We migrated to a new server and one of the problems seems to be the importing of the forum and text coding. I have our web developer and our hosting company looking into to a solution, but it may not be fixable without reverting back to the date we made the change. If that is the situation, we may "live with it".

[q]

... neglected to preserve character encoding of the database management system and data in the migration process...

... at least they should correct the more visible errors, like the forum headings for example...

Seems like a simple text search and replace throughout the php database would handle much of this blemish and annoyance.

Here's what I've detected so far:

================================
 = "hard" non breaking space
 <--- exists here invisibly
ASCII 202
option space on Mac

================================
— = em dash
—
ASCII 209
option-shift-dash on Mac

================================
“ = smart opening double quote
“
ASCII 210
opt [ on mac

================================
†= smart closing double quote
”
ASCII 211
opt shift [ on mac

================================
For the record here's a smart opening single quote:
‘
ASCII 212
option-] on Mac

================================
’ = smart closing single quote / smart apostrophe??
’
ASCII 213
option-shift-] on Mac

================================
• = bullet
•
ASCII 165
option 8 on Mac

Doug, you may want to weigh in on the accuracy of the ASCII numbers and character names.

Of course this doesn't account for the diacritical marks and foreign language symbols that first brought this issue to light.

q

[Denkster]

The fix for the correct rendering of the old content might be: the correct setting of one parameter..

That might however corrupt the new content.
The longer the forum administrator waits, the bigger the problem.

Search and replace?
The phpBB probably is based on MySQL, a really good open source DBMS!
One single query should be enough.

Of course the "somebody's little nephew" in charge of the forum lacks the required competence.

Eveline

[q]

The fix for the correct rendering of the old content might be: the correct setting of one parameter...

That might however corrupt the new content.
The longer the forum administrator waits, the bigger the problem.

Of course the "somebody's little nephew" in charge of the forum lacks the required competence.

Eveline

Eveline,

These very thoughts had crossed my mind. Thanks for voicing them.

q

[Hotch]

The text encoding issue has been solved to the point where any new posts will be encoded correctly.

[Andre_B]

The text encoding issue has been solved to the point where any new posts will be encoded correctly.

Yes, that's what I understood from the test I talked of on May 7th (here above).

In addition, I noticed that the category in the Board Index "Encore en Français" - which was presented in the first append of this thread - was corrected. I think it was the only "wrong phrase" in a header.

Other occurences could be retyped by their initial writers, it they want to.