OSTicket intermittently alerts assigned user: Page 2

OSTicket intermittently alerts assigned user

KevinTheJedi

Very good work! Yes, providing separate headers of working and non working from the same account will definitely help. As I suspect I think something in the headers is being seen as auto reply or bounce (for whatever reason).

Cheers.

TPK

KevinTheJedi

Hello Kevin,

Okay, so here I have a set of 3 headers, all from the same account... I obfuscated the domain name and my name but otherwise I left them alone...

The emails from the web client did not generate alerts, but the ones from my cell phone client, and my thunderbird client did generate alerts...

So, for whatever reason, the web mail client is being tagged as an autorespond, based on that $vars array passed into the function...

Also, if you want me to run some code against these headers within our environment, let me know how to do that... I sometimes wonder if this might be an environmental (or even a Windows) issue when it comes to parsing out these email headers, and it might not show up in your environment, etc... I imagine most folks running OSTicket are using a linux distro and not a lot are running on Windows...

I will attach the headers here...

(file deleted by admin)

TPK

@KevinTheJedi

Hello Kevin,

.. Or for anyone else that comes across this ...

So what I did was I commented out some of the code in /include/class.ticket.php so it would ignore the value in $vars['autorespond']... So my code there looks like this:

        $reopen = $autorespond; // Do not reopen bounces
        if ($autorespond && $message->isBounceOrAutoReply())
            $autorespond = $reopen= false;
        //elseif ($autorespond && isset($vars['autorespond']))
        //    $autorespond = $vars['autorespond'];

And did some more testing, and indeed now I am getting agent alerts from ticket replies from all of my accounts (yahoo, gmail, personal email via all portals, etc..).... Hopefully this will translate into alerts for all of our other helpdesk replies as well (I don't see why it shouldn't)...

So removing that check does help the issue... Although this really isn't a fix, its more like I am just masking the symptoms here and creating more risk and side effects... I'm guessing that probably puts me at a higher risk of an auto-respond infinite loop of ping-ponging emails... We have had that happen to us before, a few times, with Spiceworks... It was not pretty...

How much of a risk is it really?? It seems there are other catches in the code for that sort of thing...

Anyhow, I hope all is well.. I also hope that those headers I provided are at least somewhat helpful (are they?), and I would be happy to provide more if that would help..

Thanks much ....
...

HansAlbers

TPK Hi TPK, I'm having the same issue. Have you ever found the root cause and solution for this issue? We also ended up commenting out the exact same lines like you did in this post. Apparently $vars['autorespond'] gets set to FALSE somewhere for some reason but we have no clue where and why I could not find any code in any of the PHP files that would set it.

$reopen = $autorespond; // Do not reopen bounces if ($autorespond && $message->isBounceOrAutoReply()) $autorespond = $reopen= false; //elseif ($autorespond && isset($vars['autorespond'])) // $autorespond = $vars['autorespond'];

I have found that our osTicket does NOT send alerts to our agents when a user has replied using the Outlook client replying from a Live.nl email address, while an alert IS sent to the agent if the same user replies from the same live.nl email address using Bluemail client on an android phone. Commenting out those two lines like you did, does cause the alerts to be sent to our agents again but I'm unsure about the risks of leaving it this way.
Any suggestions or info is appreciated! Hans

KevinTheJedi

@TPK

My apologies, a lot going on right now and don't have much free time. I didn't see anything weird in the headers. You can follow the trail upwards to see where autorespond is being set to true and you should find what's causing it.

Cheers.

TPK

@KevinTheJedi

Hello,

So I have been looking into this a bit more lately... IT is slow going (I have to send an email every time, and dump variables out), but I think I have more information...

So the problem stems from the "passive" entry in the $vars object (edit: "passive" is found in $mailinfo, the $autorespond is found in $vars)...

This is happening because sometimes the lookupByEmailHeaders function is being called, and sometimes it is not... When it is not being called, then the "passive" variable is never set. Passive is set in the lookupByEmailHeaders function class.thread line 1498.

When the passive entry is not set, then the autorespond in vars is set to true [class.thread line 476], and that will cause the alert email to send (see again class.ticket line lines 3192, 3193)

When the passive entry is set in lookupByEmailHeaders, that will cause autorespond to evaluate to false [class.thread line 476], so that will cause the alert email to not send...

So when I comment out lines 3192 and 3193 in class.ticket, this will tell the code to ignore what it sees in vars['autorespond'], and will leave the $autorespond variable alone, and the alert emails will send to the agents..

What I don't know is why sometimes lookupByEmailHeaders is being called, and sometimes it is not being called, but I think that is not the real issue here (perhaps sometimes it should be called, and sometimes not??)... Could the real problem be a flaw in a variable (passive) not being set sometimes, when it should always be set (either true or false)...

Anyhow, does this help at all??

Another thing I am sorta wondering here... Am I the only one that is having trouble with these alert emails?? How is that possible?? Does no one else care about alert emails telling the agents that a ticket has been updated?? How can this be specific to just my environment??... I am not trying to be pejorative here, I really think that this is just affecting me, otherwise I think many others would be noticing this problem... I just don't see how it can affect only me and our installation here.. Its not like we are using anything out of the ordinary (web gmail, web yahoo mail, android clients, etc...)

... Anyhow, I hope this sheds some light on this (does it?)...

If you want me to, I can try to look a little deeper here, but I am not sure what I should really be checking here... In the meantime, we can mask the problem by commenting out those two lines, however I think there is something more fundamentally wrong here (it should really be fixed at the source of the issue)...

TPK

So, put another way, would the following be a legitimate fix for OSTicket, in class.thread , line 465:

From:

        $vars = array(
            'mid' =>    $mailinfo['mid'],
            'header' => $mailinfo['header'],
            'poster' => $mailinfo['name'],
            'origin' => 'Email',
            'source' => 'Email',
            'ip' =>     '',
            'reply_to' => $entry,
            'recipients' => $mailinfo['recipients'],
            'thread_entry_recipients' => $mailinfo['thread_entry_recipients'],
            'to-email-id' => $mailinfo['to-email-id'],
            'autorespond' => !isset($mailinfo['passive']),
        );

To:

        $vars = array(
            'mid' =>    $mailinfo['mid'],
            'header' => $mailinfo['header'],
            'poster' => $mailinfo['name'],
            'origin' => 'Email',
            'source' => 'Email',
            'ip' =>     '',
            'reply_to' => $entry,
            'recipients' => $mailinfo['recipients'],
            'thread_entry_recipients' => $mailinfo['thread_entry_recipients'],
            'to-email-id' => $mailinfo['to-email-id'],
            'autorespond' => ((!isset($mailinfo['passive']))  || (isset($mailinfo['passive']) && ($mailinfo['passive']))) ,
        );

So doing that seems to fix the problem (and the 2-line comment-out workaround is no longer needed)....

So what we are doing is checking if "passive" is set. IF not set, then set autorespond to true, and if it is actually set, then set autorespond to the true/false value of "passive"...

However is this a legitimate fix to OSTicket to solve this problem, and should be submitted to the source code repository?? Or does this break your intended purpose for the "passive" / "autorespond" flag ??

Seems reasonable to me, however I don't know what the real intention of "passive" here is...

... Anyhow, if this is legitimate, can we use that fix for future versions ?

KevinTheJedi

TPK

I will ask the team to see what the intention of passive threading is and get back to you.

Cheers.

KevinTheJedi

@TPK

After reviewing the code and your "fix" I can see that you basically just reversed the functionality of the call. So we are setting autorespond to true ONLY IF passive is not set. If it is set we don't want to autorespond. So your "fix" is not a fix it actually reverses intended functionality.

Once I have an update on passive threading I'll let you know.

Cheers.

TPK

@KevinTheJedi

Okay, so if that is the case, then really the fix belongs somewhere here:

class.thread.php -> lookupByEmailHeaders -> Line 1490

        // Passive threading - listen mode
        if (count($possibles)
                && ($entry = ThreadEntry::objects()
                    ->filter(array('email_info__mid__in' => array_map(
                        function ($a) { return "<$a>"; },
                    $possibles)))
                    ->first()
                )
         ) {
            $mailinfo['passive'] = true;
            return $entry;
        }

... Because incoming emails that are evaluated sometimes hit this logic when lookupByEmailHeaders is called, but lookupByEmailHeaders is not called on every email...

When lookupByEmailHeaders is not called, then passive is not set, and so we don't have a problem (autorespond will evaluate to true).

When lookupByEmailHeaders is called, then passive is set here (seemingly every time, at least every time I have tested web based email clients so far), and the agent alerts do not go out.

... So I am trying to evaluate what is happening with that conditional logic there, and I think that is where I get a little stuck (I just cannot follow what it is doing there) ....

So what if I did a variable dump of $possibles, and $entry, and $mailinfo right before the return $entry, for web-based emails that fall into this conditional, and submitted that, would that be enough to figure out what is happening here for these emails??

Emails that work (do trigger an agent alert) never fall into this code because lookupByEmailHeaders is never called to begin with (and so passive is never set)... Is that a problem too??

... Thanks much

KevinTheJedi

TPK

What's the difference with ones hitting lookupByEmailHeaders() vs not? I would think the ones not hitting lookupByEmailHeaders() is ones done via the UI as all emails that are fetched go through lookupByEmailHeaders() method - this is how we determine what thread it belongs to (if any).

Cheers.

KevinTheJedi

TPK

And to confirm, you are replying to an alert sent from the system every time correct? In order for this to work properly you need to make sure that as a User you are replying to an alert sent from the system.

Is there anything special you are doing? Like are you forwarding mail to the other recipients? Are they on a mailinglist, etc.?

Cheers.

TPK

KevinTheJedi

Actually, now that I am looking at this a bit closer, I can see how the emails are routed here, and when you bring up forwards and mail lists, etc... I think I have have spotted a potential issue here...

See, when we bring in emails, we have OSTicket go directly to our google account via IMAP... Technically, we are under a google apps umbrella account, and the OSTicket general helpdesk email is an account under our google apps account with its own identity...

... However ...

When we send email out, rather than go through google, we have a localized Microsoft SMTP server here, and so all of our outbound SMTP traffic flows through there, which in turn smart-hosts it over to our SMTP relay provider (MailJet)... Mailjet then delivers the email to the recipients...

So I think somewhere in that process, the headers might be modified or messed with (??) either by our SMTP server or by Mailjet ?? Could that be the cause of our issue here? If so that could explain why this appears to not affect a lot of folks (Although, using a smarthost/edge server or SMTP relay provider for outgoing email is probably not all that uncommon)...

Anyhow, I believe in the past we tried to set up OSTicket to mail out directly from google apps, however we had some deliverability issues... However now I know that we need to turn on that "less secure apps" setting in google for this account. With that setting we should be allowed to send directly out using google and SMTP, routing through there instead of our local SMTP and MailJet...

Could this be the source of our problem?? Has this sort of thing caused these sorts of issues??

Perhaps we should give that a try...

TPK

Oh yes... I am sending the email back from a alert that was sent out from the agent reply to a ticket via the UI.... That email alert would get sent out, once to yahoo mail, and once to my personal email. I would respond to each from the yahoo web mail, and from the android client for my personal email respectively. Agent would receive alert from the personal mail response, and not receive alert from the yahoo mail response... This happens consistently each time (at least for this particular ticket)..

So here is what I did...

I will attach a archive with a bunch of files here... I've tried to obfuscate my name (which is also in my email addresses) in the email as best I could... IF this obfuscation is causing issues (I don't think it would) then I can provide the original files instead if you like..

I reverted all files back to OSTicket stock 1.16.3 (fyi the only plugin we are using is the filesystem attachments), and then modified the lookupByEmailHeaders functions in the class.thread.php. I modified the code to dump out various variables to files at various points in the code, including a backttrace at the top of the function calls...

I then ran two examples, once with a response from yahoo mail (which fails at alerts), and once with a response from my android client personal email (which succeeds at alerts)...

... And my apologies ... It looks like I got confused by the two definitions for lookupByEmailHeaders in the code, and it seems that the lookupByEmailHeaders (the second one, with two parameters) is always being called for all emails... I am so sorry for that... That was wrong information by me... IT turns out that the second one (with two parameters) is the only one being called, at least with the two examples I am providing, but it is being called every time an email is processed, however it is exiting the function differently depending on the email client being used to respond (and so the files provided will be slightly different for the 2 examples)....

Rather than explain each file and what I captured, I will also include my modified class.thread.php file, which will show you each file I am dumping to and what I captured...

So there is some very interesting data in there, but what I think is the most notable is the match_message file (for each) showing the results of the matching to the "possibles"... This shows empty for the web client and yet it finds stuff from the android client...

Anyhow, here are the two file sets, and the modified source file..

Does this help??

I am happy to put in some more var dumps if you would like to see more stuff...

... Thanks much

... Howard

20220818-var-dumps-redact-output.zip

44kB

KevinTheJedi

@TPK

Yes of course. If the headers are modified this could cause issues as it potentially wouldn’t have the original Message-ID, In-Reply-To, and References headers we use for lookup.

Cheers.

TPK

Hello Hans,

So what fixed it was simply setting up our outbound SMTP server to go through google apps, rather than through our 3rd party SMTP provider (which we use generally for system generated outbound emails)..

I don't like setting up google as a SMTP provider, and Google goes out of its way to prevent it (although there are work-arounds at Google to make it work), but eventually we could get that to work, and it seemed to alleviate most of the e-mail notification issues..

I think the combination of receiving emails via google apps (where our domain email is hosted) and sending through a different SMTP provider for outbound (in this case, MailJet) created some discrepancies in the headers that OSTicket was unable to parse, and caused the autorespond variable to set like that... I tried very hard (at that time) to walk through the code to see where that was happening, but it got so complicated (especially without formal debugging tools) that I eventually just gave up and resigned to work around the problem as I mention above...

There are still issues with email alerts (at least for agents) in the software. For example, no matter what we did, when we tried to use Teams to set up assignments for our 2-person helpdesk, we just could not get email alerts to work properly (and notify the two of us), so we ditched using teams. Instead now what we do is just assign the other person as a collaborator on the ticket when we need to, and so we get notified that way (yes, its icky)... This is sort of a kludge and is somewhat inconvenient (because now and agent is seen as an end-user rather than an agent), but I think its the best we can do with it...

If there is a better way to do this and have email alerts work, I would love to hear it... With our prior helpdesk, we were able to just assign multiple agents (or have a primary agent, and a CC agent) to the ticket... Here we can "sort-of" do the same thing by using the collaborator work-around...

.... It should be noted that we are still on an old version (1.16.3), and so some of this might have been improved since the last time I tried to work on this ... I suppose I should try to update and re-visit this ...

... Not sure if this helps or not ...

... Howard

« Previous Page