Attachment Woe

Grizly

Edit:

Problem was with a specific sender: SAP NetWeaver, it wasn't sending a properly structured email so the osTickets code couldn't decipher it..

--------------------------------------------------------------

Mail Sending: phpMail

Mail Fetching/Receiving: POP and Gmail

PHP version: PHP 5.2.6-1+lenny16 with Suhosin-Patch 0.9.6.2

Operating System: Debian Lenny 5.0.8

Currently we are importing attachments just fine.. (well, the odd html one fails, but that is a different weirder issue)

Some PDF's come through fine.. by some I mean 99%.. there is just one system that sends us email'd PDF's with no message and a single attachment and it fails.

Message headers, space, attachment.. Bloody SAP! The sender comes through in capitals.. twice. like that, and the filename has spaces in it.

Weirdly though, THIS WORKED.. it truly did.. then it started to fail, and I can't figure it out.

I'm attempting to add some logging to the ticket-parser so I can see where it fails when an attachment doesn't get processed or denied.. currently there is no real error/message etc.

What can I do? How do I troubleshoot this single sender?

I've added the code from: https://github.com/osTicket/osTicket-1.7/pull/607/files(https://github.com/osTicket/osTicket-1.7/pull/607/files), however it doesn't seem to have fixed it for this.. hasn't introduced any other errors either, so thats ok.

example, with stuff that might identify the sender/receiver redacted:

Delivered-To: orders@mydomain.me

Received: by 10.114.173.203 with SMTP id bm11csp38002ldc;

Sun, 21 Jul 2013 18 -0700 (PDT)

X-Received: by 10.224.64.68 with SMTP id d4mr5181406qai.73.1374457409908;

Sun, 21 Jul 2013 18 -0700 (PDT)

Return-Path: <user@domain.com>

Received: from mail1.bemta8.messagelabs.com (mail1.bemta8.messagelabs.com. )

by mx.google.com with ESMTPS id w11si9723273qeu.9.2013.07.21.18.43.29

for <orders@mydomain.me>

(version=TLSv1 cipher=RC4-SHA bits=128/128);

Sun, 21 Jul 2013 18 -0700 (PDT)

Received-SPF: neutral (google.com: IP is neither permitted nor denied by best guess record for domain of user@domain.com) client-ip=IP;

Authentication-Results: mx.google.com;

spf=neutral (google.com: IP is neither permitted nor denied by best guess record for domain of user@domain.com) smtp.mail=user@domain.com

Return-Path: <user@domain.com>

Received: from by server-10.bemta-8.messagelabs.com id 40/9F-06587-04E8CE15; Mon, 22 Jul 2013 01 +0000

X-Env-Sender: user@domain.com

X-Msg-Ref: server-14.tower-220.messagelabs.com!1374457277!3275355!72

X-Originating-IP:

X-StarScan-Received:

X-StarScan-Version: 6.9.9; banners=domain.com,-,-

X-VirusChecked: Checked

Received: (qmail 12910 invoked from network); 22 Jul 2013 01 -0000

Received: from unknown (HELO a.sul.local) (IP)

by server-14.tower-220.messagelabs.com with AES128-SHA encrypted SMTP; 22 Jul 2013 01 -0000

Received: from a.sul.local (10.0.50.79) by b.sul.local

(10.20.20.30) with Microsoft SMTP Server id 8.3.83.0; Mon, 22 Jul 2013

11 +1000

Date: Mon, 22 Jul 2013 11 +1000

From: USERNAME USERNAME <username@domain.com.au>

Subject: Subject

To: <orders@mycompany.me>

Message-ID: <redacted>

MIME-Version: 1.0

Importance: Normal

X-Priority: 3 (Normal)

X-Mailer: SAP NetWeaver 7.03

Content-Type: application/pdf; name="filename oddly with spaces.pdf"

Content-Transfer-Encoding: base64

Content-Description: Subject

.... BASE64 DATA ....

...

...==

EOF

Grizly

Alright, so just to compare, the exact attachment that failed on one message, sent from mutt, worked..

Original:

MIME-Version: 1.0

Importance: Normal

X-Priority: 3 (Normal)

X-Mailer: SAP NetWeaver 7.03

Content-Type: application/pdf; name="Purchase Order number XXX.pdf"

Content-Transfer-Encoding: base64

Content-Description: Purchase Order number XXX

--- BASE 64 Stuff ---

Mutt:

MIME-Version: 1.0

Content-Type: multipart/mixed; boundary="zYM0uCDKw75PZbzx"

Content-Disposition: inline

User-Agent: Mutt/1.5.21 (2010-09-15)

--zYM0uCDKw75PZbzx

Content-Type: text/plain; charset=us-ascii

Content-Disposition: inline

TEST MESSAGE FROM GRIZLY

--zYM0uCDKw75PZbzx

Content-Type: application/pdf

Content-Disposition: attachment; filename="Purchase Order number XXX.pdf"

Content-Transfer-Encoding: base64

--- BASE 64 STUFF ---

Without any message, like original:

MIME-Version: 1.0

Content-Type: multipart/mixed; boundary="SLDf9lqlvOQaIe6s"

Content-Disposition: inline

User-Agent: Mutt/1.5.21 (2010-09-15)

--SLDf9lqlvOQaIe6s

Content-Type: text/plain; charset=us-ascii

Content-Disposition: inline

--SLDf9lqlvOQaIe6s

Content-Type: application/pdf

Content-Disposition: attachment; filename="Purchase Order number XXX.pdf"

Content-Transfer-Encoding: base64

--- BASE 64 STUFF ---

The stupid NetWeaver thing isn't setting a content-disposition header.. so the mime-decode function can't interpret the message format? gah.. my head feels like its full of cotton-wool..

Line 299 in class.mailfetch.php:

} elseif($part->ifparameters && $part->parameters && $part->type > 0) { //inline attachments without disposition.

Line 347 of include/pearr/Mail/mimeDecode.php

$this->_include_bodies ? $return->body = ($this->_decode_bodies ? $this->_decodeBody($body, $content_transfer_encoding) : $body) : null;

What? Is this too terse or is my brain dead?

Its not seeing the attachment, not saving it or something.. how can I log/debug this?

Grizly

Alright, it might actually be the originating senders email address! (We got another one)

Message-ID:

MIME-Version: 1.0

Importance: Normal

X-Priority: 3 (Normal)

X-Mailer: SAP NetWeaver 7.03

Content-Type: application/pdf; name="SAPLMEDRUCK /20130722/141831.pdf"

Content-Transfer-Encoding: base64

Content-Description: SAPLMEDRUCK /20130722/141831

JVBERi... more base64 stuff.

So, it seems that the SAP server is sending from @[deleted]0.yoda.sul.local

I'm sure that was a great idea for whichever SAP admin configured it.. but it seems to be not helpful for me.. I'm manually uploading pdf's 100 times a day right now, possibly why I can't think.

Grizly

My logging code on class.mailfetch is being hit:

//See if message is from them.... bastards

if( (substr($mailinfo, -14) == 'yoda.sul.local')

or (substr($mailinfo, -21) == '..domain.com.au')){

error_log("Parsing THEM email.. ".$mailinfo);

$THEM = true;

}

Which means I can see when its trying to do one of these attachments.. but it doesn't recognise them:

//Save attachments if any.

if(($message

&& $ost->getConfig()->allowEmailAttachments()

&& ($struct = imap_fetchstructure($this->mbox, $mid))

&& $struct->parts

&& ($attachments=$this->getAttachments($struct))) or $THEM) {

if($THEM and count($attachments) == 0) error_log("class.mailfetch: THEM Message has no attachments!");

Sure enough, my email is getting loads of hits on "THEM Message has no attachments" etc..

Parsing THEM email.. Purchase Order number 4700181794

class.mailfetch: THEM Inside attach-importer

class.mailfetch: THEM Message has no attachments!

Parsing THEM email.. Purchase Order number 4700181792

class.mailfetch: THEM Inside attach-importer

class.mailfetch: THEM Message has no attachments!

class.mailfetch: (Recursive Search for Attachments initiated)

class.mailfetch: Found =?utf-8?B?TUVMOTE5NzUucGRm?=)

Which tells me the problem (if its with this software) is in class.mailfetch->getAttachments()

I'll try dumping the imap structure of the message.. see what its getting. :-(

class.mailfetch: Failed to analyze THEM attachment!1

class.mailfetch: (Recursive Search for Attachments initiated)

class.mailfetch: Found SO29473.pdf)

class.mailfetch: Found SO29473.csv)

class.mailfetch: Found _Certification_.txt)

Parsing THEM email.. Purchase Order number 4700182033

stdClass Object

(     => 3
     => 3
     => 1
     => PDF
     => 1
     => Purchase Order number 4700182033
     => 0
     => 25478
     => 0
     => 0
     => 1
     => Array
        (
             => stdClass Object
                (
                     => NAME
                     => Purchase Order number 4700182033.pdf
                )
        )

So.. added this:

}else{

if($part->subtype == 'PDF'){

//Attach the bastard anyway.. FFFS

$f = $part->parameters->value;

if(strlen($f)>1)

$filename = $f;

}

if($filename) {

Still nope.. the structure is missing the "parts" bit.. which is why its failling!

Changed the "//Save attachments if any." if statement:

&& ($struct->parts or $THEM)

THATS FIXED IT!

YAYAYAYAYAYAYAYAYAYAY.. what a headache.. solved!