Hi,

We are using osticket 1.16.2 with PHP 8.0.18. I also tried with an older version of osticket and newer PHP (eg. 8.1).

I try to find the reason but I'm unable. In our setup, we collect the ticket from a mail server and place it in a "ticket_created" folder.

For some emails, there is no ticket body just an (empty) text, but for some emails, there is half of the email text and the end of the mail is missing.

If I put back the original mail form ticket_created folder to INBOX I can reproduce the problem.
There are tickets that contain all the texts.

Regards

    bttd

    Probably contains errant < or > characters. We use HTML balancing and sanitization so it's likely the content is being stripped.

    Cheers.

      And is there any solution for this?
      Some filtering method?

        bttd

        You can customize the code to make the sanitization/balancing how you want it but you are most likely going to open yourself up to XSS, SQL injection, etc.

        Cheers.

        • bttd replied to this.
            6 days later

            KevinTheJedi

            I do not really understand this. There is a lots of email clients which able to show HTML-based emails without any risk of XSS or SQL injection.

              @bttd

              They use different sanitization/balancing methods than we do. With v2.0 we hope to have way better code sanitization.

              Cheers.

              • bttd replied to this.
                7 months later

                KevinTheJedi

                Sorry for the late response, but in the meantime I tried to find out how long does it take to 2.0 came out, and now for me it looks like it takes more than a year(s) form now. It is impossible in the versions 1.1x.xx a better filter or handling for this?

                    bttd

                    No, the legacy (current) codebase will not receive many more updates as we will shift our focus to v2.0 completely.

                    Cheers.

                      2 months later

                      And it is possible to turn off? This filter randomly cut the half of a plain text message.

                      I don’t know how long does the 2.0 first release takes, it almost 2 years when its anunced.

                        bttd

                        Certainly but you’ll have to modify the code and you’ll potentially open yourself up to different types of vulnerabilities but to each their own.

                        Cheers.

                        • bttd replied to this.

                            KevinTheJedi

                            Can you give me some instruction where to find this piece of code?

                            Btw here is the plain text of the message, what is cutted out by the filter:
                            `Tisztelt cím,

                            Some text here from the customer, and than a few dates:
                            12.17 - 123456.-
                            12.16 - 123456.-
                            12.15 - 123456.-
                            12.14 - 123456.-
                            12.13. - 123456.-
                            12.12. - 123456.-`

                            The message was cutted after: 12.17

                            I do some tests and a single "-" can cause to cut the whole message.

                            This is what I got in osticket:

                            But this is what I send:

                            I think It's not even close to HTML, SQL command or anything.

                            Regards

                                bttd

                                That content shouldn’t be stripped then unless the actual HTML is borked.

                                You can look at class Format.

                                Cheers.

                                • bttd replied to this.

                                    KevinTheJedi

                                    The second test message sent from Gmail. Also, the first mail, what the client write to Us is totally code free if I check the plain email content.

                                    If content shouldn’t be stripped, is this a bug? Can you check it too if possible, the same message?

                                    Regards

                                        bttd

                                        I can definitely try but I’ll need the full raw email to test.

                                        Cheers.

                                        • bttd replied to this.

                                            KevinTheJedi

                                            Hi!

                                            Here is the exported email:

                                            Return-Path: <sender@email.com>
Delivered-To: sentfrom@email.com
Received: from mail.domain.com
	by mail.domain.com with LMTP
	id mJGxKR3Mv2M9Wx0AQjjvkQ
	(envelope-from <sender@email.com>)
	for <sentfrom@email.com>; Thu, 12 Jan 2023 10:00:13 +0100
Return-path: <sender@email.com>
Envelope-to: sentfrom@email.com
Delivery-date: Thu, 12 Jan 2023 10:00:13 +0100
Received: from mail-qk1-f200.google.com ([209.85.222.200]:34714)
	by mail.domain.com with esmtps  (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
	(Exim 4.95)
	(envelope-from <sender@email.com>)
	id 1pFtRR-0085tp-9S
	for sentfrom@email.com;
	Thu, 12 Jan 2023 10:00:13 +0100
Received: by mail-qk1-f200.google.com with SMTP id r6-20020a05620a298600b007025c3760d4so12632827qkp.1
        for <sentfrom@email.com>; Thu, 12 Jan 2023 01:00:14 -0800 (PST)
X-Gm-Message-State: AFqh2kqf1RfQI7pzAnQdac9DG7CTSUgKNryu7laAstZWsi+4R0QKpNYj
	8kU+mlI7SZ0fNIYR7xZlgXJM6kJvl/rVVD6+9Ey5f5RwFj0xiLWBgrzgrSC791CdQvCHMq5Vr+u
	VQ5Dp9hDaMn5x0A==
X-Received: by 2002:a05:622a:1246:b0:3a6:91f2:62bf with SMTP id z6-20020a05622a124600b003a691f262bfmr114378439qtx.37.1673514012979;
        Thu, 12 Jan 2023 01:00:12 -0800 (PST)
X-Received: by 2002:a05:622a:1246:b0:3a6:91f2:62bf with SMTP id z6-20020a05622a124600b003a691f262bfmr114378366qtx.37.1673514012231;
        Thu, 12 Jan 2023 01:00:12 -0800 (PST)
ARC-Seal: i=1; a=rsa-sha256; t=1673514012; cv=none;
        d=google.com; s=arc-20160816;
        b=z7HAgDKGTM4RM01DuFzUKXLjkWAVFJQP536D1LUbmbpg4gMJUHizT6Bb3U9DKIMYYv
         Dz0FzhK8hOgihYAS+0YIWS92c3jYnvjFSJRfopUP3+ARTOREOqrRSH+fJXak/9FUks+k
         dnMS1bPcSZ2NITAxkVx4yd0qD3E/HC/DM580DhXdSYzRCsHclyZxW1YFgdG3DLOKUwTr
         Vjy+bEtQIbU9qodSnLskHThWaSERng8yzEStjyNFpFLiyKvZpqBOvb4sCkEjlVFo0I8R
         28XS29LIHJ/XUYGed8wmIY2q4DB8Smlzt6zHffZ6mTLqOUnnrgn6ToNokdH7srOkpZyF
         WfRA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816;
        h=to:subject:message-id:date:from:mime-version:dkim-signature;
        bh=sGXp/4VEuGH8w3jB8jdeuNHVXdS+Yj7oA1LlAK8dA6I=;
        b=BaJlgIcw3S4kIiTxOE+4/mxSJ6gJ0XinORf5AgokUTwGDsbzpNfGJVFdqfx6sSOyyd
         WsLjaW1wkwwwVUJZHPFg7XFhseusx6uzUHKYPR7ILkoX+e3fwcWYRiyNRh/8u/U1sK3R
         gFf2zUeUf4roAE7wUPjNFM3/V/j1SSP82OqMiAUMngo16oixTsC/YkYgZJJDbxj2/yWY
         hGJS/n5R/pE35zV8QoPNtlHnnEWzNk0Q65QOMcqSFhFup0BCRTqKrnIcnAAR6xBvOOtm
         UErmbnmQK7aaV47Zd6CZkt2pE0Uyd8H2mUvf2TAZkcBR5bDNITb7r/FAQhSumZmPCIxS
         AGRw==
ARC-Authentication-Results: i=1; mx.google.com;
       dkim=pass header.i=@company.domian header.s=google header.b=dYz5iwXD;
       spf=pass (google.com: domain of sender@email.com designates 209.85.220.41 as permitted sender) smtp.mailfrom=sender@email.com;
       dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=company.domian
Received: from mail-sor-f41.google.com (mail-sor-f41.google.com. [209.85.220.41])
        by mx.google.com with SMTPS id bl11-20020a05622a244b00b003a6f7bb72e7sor12164830qtb.30.2023.01.12.01.00.12
        for <sentfrom@email.com>
        (Google Transport Security);
        Thu, 12 Jan 2023 01:00:12 -0800 (PST)
Received-SPF: pass (google.com: domain of sender@email.com designates 209.85.220.41 as permitted sender) client-ip=209.85.220.41;
Authentication-Results: mx.google.com;
       dkim=pass header.i=@company.domian header.s=google header.b=dYz5iwXD;
       spf=pass (google.com: domain of sender@email.com designates 209.85.220.41 as permitted sender) smtp.mailfrom=sender@email.com;
       dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=company.domian
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=company.domian; s=google;
        h=to:subject:message-id:date:from:mime-version:from:to:cc:subject
         :date:message-id:reply-to;
        bh=sGXp/4VEuGH8w3jB8jdeuNHVXdS+Yj7oA1LlAK8dA6I=;
        b=dYz5iwXD4cqY8c16jwVFWlWY3BMp4BGKta5LuAX8NGob2MlDIHmz65IuNMaJ5Gtt8M
         ELBVGFKJsvN/G/Hzo4s/36oe0h6XuBuHTH+7UX0N8UpP2C2EBxkG+ozkfxcoTerFtLZs
         0GvYqaYiE9C2SHDPUxJxDlXm/XlqM02VRfVAUdf3X2MTK4VjCnXGFZOfpjS560yQPiYs
         JHkdZmMpFeeyhXD060P5jXG7fN3aX/HX1S113bu7Wk5XpeRJVHhiGQF2PJ9oysghmoWA
         a4BrPEEFen2MDr0ALgp3otiYfdPwpCelwVQ49l57mBXPwN21lFg9RyOMAxGdvwulZvpq
         LS7Q==
X-Google-Smtp-Source: AMrXdXuSyDEfwoWTa+ZHWEtMcUVGPi5hxO2EuEiYfvFZZ7qcKE1opseXoieEdJ1eoQgHhe1wBezJfI4AJcVSKlD97o4=
X-Received: by 2002:ac8:4403:0:b0:3a7:f2b0:c4c0 with SMTP id
 j3-20020ac84403000000b003a7f2b0c4c0mr4263499qtn.490.1673514011560; Thu, 12
 Jan 2023 01:00:11 -0800 (PST)
MIME-Version: 1.0
From: =?UTF-8?Q?D=C3=A1niel_-------?= <sender@email.com>
Date: Thu, 12 Jan 2023 09:59:58 +0100
Message-ID: <CABAg3zhMC1uip26ZR_E7rus_+8t73cOPwFSNRh7GZ6S9oJpV7w@mail.gmail.com>
Subject: teszt message
To: sentfrom@email.com
Content-Type: multipart/alternative; boundary="0000000000002fa67b05f20d584e"

--0000000000002fa67b05f20d584e
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi,

This is a test message, I put a hypen in here: -
The message bellow is cuted out by osticket.

--=20

------- D=C3=A1niel

CTO | company Kft.

Telefon: +0000000000

<https://domain.com/> <https://www.facebook.com/company/>
<https://www.instagram.com/company/>



=C3=89RTES=C3=8DT=C3=89S BIZALMAS LEVELEZ=C3=89SHEZ

Az ebben az e-mailben tal=C3=A1lhat=C3=B3 inform=C3=A1ci=C3=B3k bizalmasak.=
 Csak a megjel=C3=B6lt
c=C3=ADmzettekhez sz=C3=B3l, =C3=A9s a hozz=C3=A1f=C3=A9r=C3=A9s harmadik s=
zem=C3=A9lyek sz=C3=A1m=C3=A1ra meg nem
engedett. Amennyiben nem =C3=96n a lev=C3=A9l t=C3=A9nyleges c=C3=ADmzettje=
, akkor nem hozhatja
nyilv=C3=A1noss=C3=A1gra, nem m=C3=A1solhatja, nem tov=C3=A1bb=C3=ADthatja =
illetve m=C3=A1s m=C3=B3don sem
haszn=C3=A1lhatja az ebben az e-mailben tal=C3=A1lhat=C3=B3 inform=C3=A1ci=
=C3=B3kat, illetve azokra
nem is t=C3=A1maszkodhat. Az ilyen jelleg=C5=B1 jogosulatlan felhaszn=C3=A1=
l=C3=A1s
jogellenes.  Amennyiben t=C3=A9vesen kapta meg ezt az e-mailt, k=C3=A9rj=C3=
=BCk, hogy
azonnal =C3=A9rtes=C3=ADtse a felad=C3=B3t, valamint t=C3=A1vol=C3=ADtsa el=
 a levelet =C3=A9s =C3=B6sszes
m=C3=A1solat=C3=A1t sz=C3=A1m=C3=ADt=C3=B3g=C3=A9pes rendszer=C3=A9b=C5=91l=
.



PRIVACY NOTICE FOR CONFIDENTIAL COMMUNICATIONS

The information contained in this e-mail is confidential. It is intended
only for the stated addressee(s) and access to it by any other person is
unauthorised. If you are not an addressee, you must not disclose, copy,
circulate or in any other way use or rely on the information contained in
this e-mail. Such unauthorised use may be unlawful. If you have received
this e-mail in error, please inform us immediately and delete it and all
copies from your system.

--0000000000002fa67b05f20d584e
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi,<div><br></div><div>This is a test message, I put a hyp=
en=C2=A0in here: -=C2=A0</div><div>The message bellow=C2=A0is cuted=C2=A0ou=
t by osticket.=C2=A0</div><div><div><br></div>-- <br><div dir=3D"ltr" class=
=3D"gmail_signature" data-smartmail=3D"gmail_signature"><div dir=3D"ltr"><d=
iv dir=3D"ltr"><div dir=3D"ltr"><span><div dir=3D"ltr" style=3D"margin-left=
:0pt" align=3D"left"></div><div><span style=3D"font-size:8pt;font-family:Ar=
ial;color:rgb(0,0,0);background-color:transparent;font-style:italic;vertica=
l-align:baseline;white-space:pre-wrap"><span style=3D"font-style:normal;whi=
te-space:normal"><div dir=3D"ltr" align=3D"left" style=3D"margin-left:0pt">=
<table style=3D"border:none;border-collapse:collapse"><colgroup><col width=
=3D"332"></colgroup><tbody><tr style=3D"height:29.329705pt"><td style=3D"bo=
rder:1pt solid rgb(255,255,255);vertical-align:top;padding:5pt;overflow:hid=
den"><p dir=3D"ltr" style=3D"line-height:1.2;margin-top:0pt;margin-bottom:0=
pt"><span style=3D"font-size:11pt;font-family:Poppins,sans-serif;color:rgb(=
255,69,0);font-weight:600;vertical-align:baseline;white-space:pre-wrap">Bat=
ta D=C3=A1niel</span></p><p dir=3D"ltr" style=3D"line-height:1.2;margin-top=
:0pt;margin-bottom:0pt"><span style=3D"font-size:11pt;font-family:Poppins,s=
ans-serif;color:rgb(51,51,51);vertical-align:baseline;white-space:pre-wrap"=
>CTO | company Kft.</span></p><p dir=3D"ltr" style=3D"line-height:1.2;ma=
rgin-top:0pt;margin-bottom:0pt"><span style=3D"font-size:11pt;font-family:P=
oppins,sans-serif;color:rgb(51,51,51);vertical-align:baseline;white-space:p=
re-wrap">Telefon: +36 70 250 0127</span></p><br><p dir=3D"ltr" style=3D"lin=
e-height:1.2;margin-top:0pt;margin-bottom:0pt"><span style=3D"font-size:11p=
t;vertical-align:baseline;white-space:pre-wrap"><span style=3D"border:none;=
display:inline-block;overflow:hidden;width:200px;height:21px"><img src=3D"h=
ttps://lh5.googleusercontent.com/H1CfWFQ9I5oiQSDq17Y3IHzutCbtXzci8osQ2i3tg8=
CAhiguNNdBw--3cpEOyu5F5kbI0AQTcSGO1YLlJE0lF5eh8gOb1vdzfr3HLtD0VyFM9d-WzmNRh=
GwBol1AviwyaIXs-uARrv2n-NOMfgBhdRtkNI7ev6KP3DbdaB__iOBTRCrIuHfAHMZ5QAES7w" =
width=3D"200" height=3D"21" style=3D"margin-left:0px;margin-top:0px"></span=
></span></p></td></tr><tr style=3D"height:48.586399pt"><td style=3D"border:=
1pt solid rgb(255,255,255);vertical-align:top;padding:5pt;overflow:hidden">=
<p dir=3D"ltr" style=3D"line-height:1.2;margin-top:0pt;margin-bottom:10pt">=
<a href=3D"https://domain.com/" style=3D"text-decoration:none" target=3D"=
_blank"><span style=3D"font-size:11pt;color:rgb(17,85,204);text-decoration:=
underline;vertical-align:baseline;white-space:pre-wrap"><span style=3D"bord=
er:none;display:inline-block;overflow:hidden;width:34px;height:34px"><img s=
rc=3D"https://lh4.googleusercontent.com/953Fa4_nnDJz-XnwNDwKlpRUqdHjbS2fi8O=
KXWKLy6ZGiKTOVVORDc0KHfGHCf20vgM7pwlBneuygh_Zyig0CGNgVmX392LCHj7P0FFoCRmR9o=
NJNreiYB5QIrrIr4PwzXKGZBNoMdFdVeyEPZGQDNZhDTKYkVVc_0vdRp1qdGvJ3dr0al5knQY6B=
sYAXw" width=3D"34" height=3D"34" style=3D"margin-left:0px;margin-top:0px">=
</span></span></a><span style=3D"font-size:11pt;vertical-align:baseline;whi=
te-space:pre-wrap"> </span><a href=3D"https://www.facebook.com/company/" sty=
le=3D"text-decoration:none" target=3D"_blank"><span style=3D"font-size:11pt=
;color:rgb(17,85,204);text-decoration:underline;vertical-align:baseline;whi=
te-space:pre-wrap"><span style=3D"border:none;display:inline-block;overflow=
:hidden;width:34px;height:34px"><img src=3D"https://lh4.googleusercontent.c=
om/OsdAtb6MiHJskvvk5S4_8973mjI0V1Tu3MHaUhmVLPFFugjV0bGAdy7Wkh1DuGrARkaPv99Y=
7Qn_IgWii0zqBrqOQw1DyUpdlwlqS8qp6z66HJGwwA7vO3q61J9Hx-iofEdSfgJuF0cBjp7Iazf=
MMOTJY2vZpB_eUNioEIKNLOMcBgpIQRyamprG5V9DRg" width=3D"34" height=3D"34" sty=
le=3D"margin-left:0px;margin-top:0px"></span></span></a><span style=3D"font=
-size:11pt;vertical-align:baseline;white-space:pre-wrap"> </span><a href=3D=
"https://www.instagram.com/company/" style=3D"text-decoration:none" target=
=3D"_blank"><span style=3D"font-size:11pt;color:rgb(17,85,204);text-decorat=
ion:underline;vertical-align:baseline;white-space:pre-wrap"><span style=3D"=
border:none;display:inline-block;overflow:hidden;width:34px;height:34px"><i=
mg src=3D"https://lh5.googleusercontent.com/iFGcyAO7f1SPoMJ8PcJ5TFu1r-TP8Pp=
pwQP5PPCdybYXFOwa-WuuxblzAzuszDsgk0RKNM3iZepZa86VfV8UgA-NdfIFu9vycftiNN11JN=
-RJIBmlEPRAhYkz8X7iN3R5Us0vY0WpoJf4Fj82aFGYQbWmbwoGNOdmZk66MvPXRvgvZ9o3Xter=
NswwvOj7Q" width=3D"34" height=3D"34" style=3D"margin-left:0px;margin-top:0=
px"></span></span></a><span style=3D"font-size:11pt;vertical-align:baseline=
;white-space:pre-wrap">=C2=A0</span></p><br></td></tr></tbody></table></div=
><p dir=3D"ltr" style=3D"line-height:1.38;margin-top:0pt;margin-bottom:0pt"=
>=C2=A0</p><p dir=3D"ltr" style=3D"line-height:1.38;margin-top:0pt;margin-b=
ottom:0pt"><span style=3D"font-size:8pt;font-weight:700;font-style:italic;v=
ertical-align:baseline;white-space:pre-wrap">=C3=89RTES=C3=8DT=C3=89S BIZAL=
MAS LEVELEZ=C3=89SHEZ</span></p><p dir=3D"ltr" style=3D"line-height:1.38;ma=
rgin-top:0pt;margin-bottom:0pt"><span style=3D"font-size:8pt;font-style:ita=
lic;vertical-align:baseline;white-space:pre-wrap">Az ebben az e-mailben tal=
=C3=A1lhat=C3=B3 inform=C3=A1ci=C3=B3k bizalmasak. Csak a megjel=C3=B6lt c=
=C3=ADmzettekhez sz=C3=B3l, =C3=A9s a hozz=C3=A1f=C3=A9r=C3=A9s harmadik sz=
em=C3=A9lyek sz=C3=A1m=C3=A1ra meg nem engedett. Amennyiben nem =C3=96n a l=
ev=C3=A9l t=C3=A9nyleges c=C3=ADmzettje, akkor nem hozhatja nyilv=C3=A1noss=
=C3=A1gra, nem m=C3=A1solhatja, nem tov=C3=A1bb=C3=ADthatja illetve m=C3=A1=
s m=C3=B3don sem haszn=C3=A1lhatja az ebben az e-mailben tal=C3=A1lhat=C3=
=B3 inform=C3=A1ci=C3=B3kat, illetve azokra nem is t=C3=A1maszkodhat. Az il=
yen jelleg=C5=B1 jogosulatlan felhaszn=C3=A1l=C3=A1s jogellenes.=C2=A0 Amen=
nyiben t=C3=A9vesen kapta meg ezt az e-mailt, k=C3=A9rj=C3=BCk, hogy azonna=
l =C3=A9rtes=C3=ADtse a felad=C3=B3t, valamint t=C3=A1vol=C3=ADtsa el a lev=
elet =C3=A9s =C3=B6sszes m=C3=A1solat=C3=A1t sz=C3=A1m=C3=ADt=C3=B3g=C3=A9p=
es rendszer=C3=A9b=C5=91l.</span></p><p dir=3D"ltr" style=3D"line-height:1.=
38;margin-top:0pt;margin-bottom:0pt">=C2=A0</p><p dir=3D"ltr" style=3D"line=
-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style=3D"font-size:8pt=
;font-weight:700;font-style:italic;vertical-align:baseline;white-space:pre-=
wrap">PRIVACY NOTICE FOR CONFIDENTIAL COMMUNICATIONS</span></p><p dir=3D"lt=
r" style=3D"line-height:1.38;margin-top:0pt;margin-bottom:0pt"><span style=
=3D"font-size:8pt;font-style:italic;vertical-align:baseline;white-space:pre=
-wrap">The information contained in this e-mail is confidential. It is inte=
nded only for the stated addressee(s) and access to it by any other person =
is unauthorised. If you are not an addressee, you must not disclose, copy, =
circulate or in any other way use or rely on the information contained in t=
his e-mail. Such unauthorised use may be unlawful. If you have received thi=
s e-mail in error, please inform us immediately and delete it and all copie=
s from your system.=C2=A0</span></p></span></span></div></span></div></div>=
</div></div></div></div>

--0000000000002fa67b05f20d584e--

                                                bttd

                                                I did some testing and no matter what the email came in just fine for me.

                                                Cheers.

                                                • bttd replied to this.

                                                    bttd

                                                    I do not, no. We use the same ticket creation process for pipe and fetch and it worked just fine for me. At this point you'd need to do some debugging in the code to see where it's going wrong.

                                                    Cheers.

                                                    • bttd replied to this.

                                                        bttd

                                                        From the mailfetcher and just follow the pipeline to see when the content is empty.

                                                        Cheers.

                                                          Write a Reply...