Hi there,
i haven't a solution yet, but some curious news and temporary workaround..:
I've analyzed the code and set some more DEBUG markings and outputs.
What i found is, that the problem must be somewhere very early in the process.. I've generated a email with multiple umlauts in the subject and body. The behaving seems to be slightly different here.
When i print_r the var "$info" in function CreateTicket(), i get this line:
"[Subject] => =?ISO-8859-1?Q?Ein_n=E4chster_Test_f=FCr_Umlaute_4?="
You can see the mime (?) encoded chars..
Some line below you dumping the var "$vars['subject']" results in
"Ein n�chster Test f�r Umlaute 4"
As you can see, every umlaut is converted into some special utf8 char "EF BF BD".
Next, you have two inserts into the database:
INSERT INTO ost_form_entry_values SET field_id = 20, entry_id = 304, value = 'Ein n�chster Test f�r Umlaute 3'
INSERT INTO ost_ticket__cdata SET subject='Ein n�chster Test f�r Umlaute 3', ticket_id= 160 ON DUPLICATE KEY UPDATE subject='Ein n�chster Test f�r Umlaute 3'
Both have bad chars inside and the database cuts off the string.
The more curios thing is, that the body INSERT query seems to have some different content, as there is only ONE SINGLE char converted into the bad one "EF BF BD". All following chars are converted into 2 (?) byte UTF8 codes (ö = C3 B6, ä = C3 A4, ü = C3 BC and so on..), which do not harm the database:
INSERT INTO ost_thread_entry SET created = NOW(), type = 'M', thread_id = 173, title = 'Ein n�chster Test für Umlaute 3', format = 'html', user_id = 16, poster = 'Simon', flags = 64, body = ' <p><span style=\"font-family:sans-serif;font-size:small\">Sie ist ein Test f�r mehrere Umlaute.</span><br /> <span style=\"font-family:sans-serif;font-size:small\">Umlaute sind ö ä ü</span><br /> <span style=\"font-family:sans-serif;font-size:small\">Ausserdem das ß, ein sz.</span></p> '
See, there is only one bad char, in both, the subject and the body each. Very strange. Btw, the original body seems to be "Richtext/HTML" formated.
I'll add some textfile with the raw debug output i generated, so you can lookup the chars by yourself.
NOTE: Don't have permission to attach files.
I don't have the IMAP query in raw format yet, because i need to catch from the mailserver, i think. thats more complicated. maybe i find some way..
By now, i've hacked some "cleanup" code into mysqli.php:
<?PHP
unction db_query($query, $logError=true, $buffered=true) {
global $ost, $__db;
if ($__db->unbuffered_result) {
$__db->unbuffered_result->free();
$__db->unbuffered_result = false;
}
// WORKAROUND os: fehlerhafte Zeichen aus Query entfernen
if (substr($query, 0, 6) === 'INSERT') {
ini_set('mbstring.substitute_character', "?");
$query = (mb_convert_encoding($query, 'UTF-8', 'UTF-8'));
} // END Workaround
$tries = 3;
do {
$res = $__db->query($query,
$buffered ? MYSQLI_STORE_RESULT : MYSQLI_USE_RESULT);
// Retry the query due to deadlock error (#1213)
// TODO: Consider retry on #1205 (lock wait timeout exceeded)
// TODO: Log warning
?>
What it does, is converting the query string from UTF8 into UTF8. That seems to be a simple code, to filter out all "bad" chars, which can harm the database. I noticed some serious performance impact, so i've limited the filtering to all "INSERT" statements. Seems to do fine, by now, no problems.
Maybe someone can trace down the code, which does the strange things.
Simon
