KevinTheJedi

Okay, will do... But I have a couple of questions, based mostly on my ignorance on how the system fundamentally works, and what is allowed within PHP. Please forgive these noob-ish questions, I just want to make sure that I get everything exactly right...

1) Do I replace the entire subject on replies with "Testing123", or should I put "Testing123" after or before the existing subject (which should be something like "Re: Test ticket from yahoo [#06172]") ... I am not sure exactly how OSTicket connects email responses back to the ticket, but if it uses the subject line with the ticket number then perhaps I need to keep that in there somewhere (??)

2) Since this is a windows system, the path to OSTicket files is going to be "C:\inetpub\wwwroot\Helpdesk".. I know that most C-type languages use backslash as the escape character, so should I change that to "C:\inetpub\wwwroot\Helpdesk" or "C:/inetpub/wwwroot/Helpdesk" or leave it as is??

.... Edit ... double back-slashes didn't survive the forum post formatting, so what I was wondering is if I need to use double-backslash instead of single back-slash for the path name ...

@TPK

  1. You will create a new ticket for the new test. Put Testing123 anywhere in the initial email's subject line. I think when you reply all (as collabs) it doesn't let you change the subject but if it does just make sure Testing123 is somewhere in the subject line.
  2. Sorry, forgot you're on windows. Yes, you can do backslash. I believe normal 1 backslash should work. Of course it's been forever since I touched windows so it might need double backslashes.

Cheers.

KevinTheJedi

So here is what happens...

I attempted to put the code in there, and I would never get anything in the var_dump.txt file and I would no longer get agent alerts from the ticket responses, when appending "Testing123" to the subject...

So its possible that I might not have permissions properly set up for the var_dump.txt file, or I might not have formulated the filename properly within the put_contents function, however I don't think that is where the problem lies...

I think there is something wrong with the code, and when it attempts to evaluate what is in the "IF" statement there, because no matter what I do, I cannot get the agent to receive an alert as long as the code is in there, even when I do not put the Testing123 into the subject line (which should ostensibly mean that the code within the if statement should not fire, yes??)

I have done all this testing with my home account (the one that seemingly works)... When I email the ticket from that account (without any modification to the subject line), and the extra code present, I get a message confirmation email back (saying that my reply has been noted), however the agent does not get an alert anymore...

Whenever I remove the code, I get both a message confirmation back, and the agent does get an alert...

I will include a screenshot here of what the code looks like out of class.ticket.php

So anyhow, regardless of how I formulate the string for the var_dump file, that code should never be executed if there is no "Testing123" in the subject line anyhow, yet I am still getting no agent alert when the code is present, and always I am getting an agent alert when the code is absent...

I also attempted to modify the "$ticket->getSubject()" to "$this->getSubject()" (since that seems to be used in this file), but that didn't make any difference either...

Not sure exactly what I am doing wrong here, but seemingly the presence of this code might be causing an error (??)...

I included a screenshot above, does what I have there seem okay ??

@KevinTheJedi

Sorry Kevin, I think we cross-posted there...

To be clear, the testing I was doing was on my original test ticket (the one that generated the headers I provided earlier), so I did not create a new ticket (per your most recent post)...

I was using forward-slashes for the filename there, however like I mentioned, I was not even changing the subject line at all (not putting in the "Testing123") and just the presence of that code (under those conditions) was causing the agent not to receive an alert (from my home account, which works), and would always receive an alert when the code was absent...

So I think something is wrong and that code is throwing an error (even with the code not branching into the if statement)...

Is there something else wrong??

Perhaps if I create a video of me doing all of that it might make more sense?? Should I do that??

... Would you be willing to remote in here with me? That might make this easier too (I am good with whatever)...

@TPK

Okay, well it seems like you'll need to do some var_dump()ing without all the extra stuff (see snippet below), temporarily disable cronjob/scheduled task (so you don't have multiple cron jobs running), and just run cron manually via CLI:
php C:\inetpub\wwwroot\Helpdesk\api\cron.php

There you will get output to the command prompt.

       var_dump('Subject: ', isset($vars['subject']) ? $vars['subject'] : $ticket->getSubject());
       var_dump('Poster: ', $vars['email']);
       var_dump('Alerts: ', $alerts);
       var_dump('Autorespond: ', $autorespond);

Cheers.

  • TPK replied to this.

    KevinTheJedi

    Okay, now we are getting somewhere šŸ™‚

    I replied to my test ticket from my (Home) account (the one that works), and I ran the cron manually... See the output below:

    C:\Users\redacted>"C:\Program Files (x86)\PHP\v7.4\php.exe" C:\inetpub\wwwroot\Helpdesk\api\cron.php
    string(9) "Subject: "
    string(22) "Test ticket from yahoo"
    string(8) "Poster: "
    NULL
    string(8) "Alerts: "
    bool(true)
    string(13) "Autorespond: "
    bool(true)

    So "Poster" is null? Can that be right?

    KevinTheJedi

    Yep, I see that I am still running php 7.4 for my automated cron job (when we should be running 8.0)... I need to fix that in my windows scheduled task...

    However when running with php 8.0 I get the same thing (doesn't matter)...

    C:\>"C:\Program Files (x86)\PHP\v8.0\php.exe" C:\inetpub\wwwroot\Helpdesk\api\cron.php
    string(9) "Subject: "
    string(22) "Test ticket from yahoo"
    string(8) "Poster: "
    NULL
    string(8) "Alerts: "
    bool(true)
    string(13) "Autorespond: "
    bool(true)

    KevinTheJedi

    Also, when I send a reply in from my yahoo account, I get no output from running the cron job from the command prompt... Nothing at all.. And I get no agent alert or confirmation response back to yahoo..

    ... Yet the reply does seem to process into the ticket ... I can see it there in the portal ...

    So it looks like this code is never hit, when I reply from my yahoo account (but is hit when I reply from my home account)... And it looks like the email is being processed for the ticket ...

    Is that helpful??

    ... Again, I appreciate this so much, thank you!

    Oye, it seems I must have hit some kind of posting limit, and now my most recent posts are "awaiting approval"....

    Anyhow, hopefully you will be able to see my posts...

    And to be clear, when I did get the output back, when I posted from my home account, I did get the agent alert and the message confirmation... So that did work...

    I got no output when replying from the yahoo mail user, and did not get the agent alert or the message confirmation...

    KevinTheJedi

    Okay,

    So I have done a lot of digging into the postMessage function, and after many var_dumps and emails going back and forth, I discovered that the issue boils down to the $vars being passed into the function with $vars['autorespond'] set to true on some emails replies coming in, and $vars['autorespond'] set to false on othersā€¦

    I can prove this because at the very top of the function I put this line:

    var_dump('VarsAutoRespondAtTop:', isset($vars['autorespond']) ? $vars['autorespond'] : 'Autorespond var not set');

    and when running the cron job manually at a command prompt, I would get true or false depending on where the email reply was coming fromā€¦ (vars[ā€˜autorespondā€™] was always set, I would never get ā€œvar not setā€ from that)ā€¦

    ā€¦ Now earlier, I mentioned that the yahoo mails coming in were not showing anything in the outputā€¦ Somehow, when I was working with it later in the night, the yahoo mails WERE showing outputā€¦ I think what might have been happening is that even though I turned off the chron job (or the Windows scheduled task), the helpdesk was still seeing activity from the other member in our support group. Because of that, I think emails were still being fetched (because of the activity?), and it was (I guess) perfectly correlated with when I was testing yahoo mail and when I was testing my home mail ā€¦ At least that is what I think was going on because when I was testing this off-hours I was getting responses from the var_dumps each time I sent in an emailā€¦

    One other thing about the home mailā€¦. I have several portals for that email, I can use my Thunderbird email client, or I can use the email client on my cell phone, or the zoho web mail portal for my accountā€¦ So when I reply to a helpdesk email from the Thunderbird client or my email client off my cell phone, I would get agent alerts every timeā€¦ When I reply to a helpdesk email from the zoho web mail portal, I would not get an agent reply every timeā€¦. So, even from the same email account, just the email client I am using does seem to make a differenceā€¦ I must have tested this at least 10 times from each client, replying to the same exact agent response email, and I get the same results every timeā€¦ And, of course, email responses generating from the yahoo web mail client, or the gmail web mail client, never result in an agent alert in any of the testsā€¦

    Anyhow, I spent a while testing var_dumps for everything, in and around the area of code we were testing before, and I found that after this code is run:

            $reopen = $autorespond; // Do not reopen bounces
            if ($autorespond && $message->isBounceOrAutoReply())
                $autorespond = $reopen= false;
            elseif ($autorespond && isset($vars['autorespond']))
                $autorespond = $vars['autorespond'];

    The $autorespond would show true or false depending on the email sourceā€¦ Then I narrowed it down to the $vars['autorespond']; , and from there I noticed that the function was just being passed the vars set that way right in the vars parameter to the functionā€¦ (the IsBounceOrAutoReply would always return ā€œFalseā€ in all my tests, so that is not the issue here)..

    Anyhow, clearly there is something going on with emails depending on the source and client being used to make the repliesā€¦ I donā€™t know if this can be found in the headers, or somewhere else, and from here I am not sure where to goā€¦

    The good news here, I think, is that I can finally narrow the issue down to some very specific circumstancesā€¦ Before it all seemed kind of random and we would sometimes get alerts and sometimes not, and this drove us crazyā€¦ Now I can reliably reproduce the issue and I get the same results every time I take certain actions.. Also, I think I know (at least in a general sense) where in the PHP code this issue lies, even though I have no idea what to check for next or how to fix itā€¦

    This is a lot more than I knew just yesterdayā€¦

    So what do you think??

    If you would like, I can send you headers again, except this time from the same email account (just from different portals), as we are getting different results depending on the portalā€¦ Would that be helpful??

      TPK

      Very good work! Yes, providing separate headers of working and non working from the same account will definitely help. As I suspect I think something in the headers is being seen as auto reply or bounce (for whatever reason).

      Cheers.

      • TPK replied to this.

        KevinTheJedi

        Hello Kevin,

        Okay, so here I have a set of 3 headers, all from the same account... I obfuscated the domain name and my name but otherwise I left them alone...

        The emails from the web client did not generate alerts, but the ones from my cell phone client, and my thunderbird client did generate alerts...

        So, for whatever reason, the web mail client is being tagged as an autorespond, based on that $vars array passed into the function...

        Also, if you want me to run some code against these headers within our environment, let me know how to do that... I sometimes wonder if this might be an environmental (or even a Windows) issue when it comes to parsing out these email headers, and it might not show up in your environment, etc... I imagine most folks running OSTicket are using a linux distro and not a lot are running on Windows...

        I will attach the headers here...

        (file deleted by admin)

        5 days later

        @KevinTheJedi

        Hello Kevin,

        .. Or for anyone else that comes across this ...

        So what I did was I commented out some of the code in /include/class.ticket.php so it would ignore the value in $vars['autorespond']... So my code there looks like this:

                $reopen = $autorespond; // Do not reopen bounces
                if ($autorespond && $message->isBounceOrAutoReply())
                    $autorespond = $reopen= false;
                //elseif ($autorespond && isset($vars['autorespond']))
                //    $autorespond = $vars['autorespond'];

        And did some more testing, and indeed now I am getting agent alerts from ticket replies from all of my accounts (yahoo, gmail, personal email via all portals, etc..).... Hopefully this will translate into alerts for all of our other helpdesk replies as well (I don't see why it shouldn't)...

        So removing that check does help the issue... Although this really isn't a fix, its more like I am just masking the symptoms here and creating more risk and side effects... I'm guessing that probably puts me at a higher risk of an auto-respond infinite loop of ping-ponging emails... We have had that happen to us before, a few times, with Spiceworks... It was not pretty...

        How much of a risk is it really?? It seems there are other catches in the code for that sort of thing...

        Anyhow, I hope all is well.. I also hope that those headers I provided are at least somewhat helpful (are they?), and I would be happy to provide more if that would help..

        Thanks much ....
        ...

          @TPK

          My apologies, a lot going on right now and don't have much free time. I didn't see anything weird in the headers. You can follow the trail upwards to see where autorespond is being set to true and you should find what's causing it.

          Cheers.

          3 months later

          @KevinTheJedi

          Hello,

          So I have been looking into this a bit more lately... IT is slow going (I have to send an email every time, and dump variables out), but I think I have more information...

          So the problem stems from the "passive" entry in the $vars object (edit: "passive" is found in $mailinfo, the $autorespond is found in $vars)...

          This is happening because sometimes the lookupByEmailHeaders function is being called, and sometimes it is not... When it is not being called, then the "passive" variable is never set. Passive is set in the lookupByEmailHeaders function class.thread line 1498.

          When the passive entry is not set, then the autorespond in vars is set to true [class.thread line 476], and that will cause the alert email to send (see again class.ticket line lines 3192, 3193)

          When the passive entry is set in lookupByEmailHeaders, that will cause autorespond to evaluate to false [class.thread line 476], so that will cause the alert email to not send...

          So when I comment out lines 3192 and 3193 in class.ticket, this will tell the code to ignore what it sees in vars['autorespond'], and will leave the $autorespond variable alone, and the alert emails will send to the agents..

          What I don't know is why sometimes lookupByEmailHeaders is being called, and sometimes it is not being called, but I think that is not the real issue here (perhaps sometimes it should be called, and sometimes not??)... Could the real problem be a flaw in a variable (passive) not being set sometimes, when it should always be set (either true or false)...

          Anyhow, does this help at all??

          Another thing I am sorta wondering here... Am I the only one that is having trouble with these alert emails?? How is that possible?? Does no one else care about alert emails telling the agents that a ticket has been updated?? How can this be specific to just my environment??... I am not trying to be pejorative here, I really think that this is just affecting me, otherwise I think many others would be noticing this problem... I just don't see how it can affect only me and our installation here.. Its not like we are using anything out of the ordinary (web gmail, web yahoo mail, android clients, etc...)

          ... Anyhow, I hope this sheds some light on this (does it?)...

          If you want me to, I can try to look a little deeper here, but I am not sure what I should really be checking here... In the meantime, we can mask the problem by commenting out those two lines, however I think there is something more fundamentally wrong here (it should really be fixed at the source of the issue)...

          So, put another way, would the following be a legitimate fix for OSTicket, in class.thread , line 465:

          From:

                  $vars = array(
                      'mid' =>    $mailinfo['mid'],
                      'header' => $mailinfo['header'],
                      'poster' => $mailinfo['name'],
                      'origin' => 'Email',
                      'source' => 'Email',
                      'ip' =>     '',
                      'reply_to' => $entry,
                      'recipients' => $mailinfo['recipients'],
                      'thread_entry_recipients' => $mailinfo['thread_entry_recipients'],
                      'to-email-id' => $mailinfo['to-email-id'],
                      'autorespond' => !isset($mailinfo['passive']),
                  );

          To:

                  $vars = array(
                      'mid' =>    $mailinfo['mid'],
                      'header' => $mailinfo['header'],
                      'poster' => $mailinfo['name'],
                      'origin' => 'Email',
                      'source' => 'Email',
                      'ip' =>     '',
                      'reply_to' => $entry,
                      'recipients' => $mailinfo['recipients'],
                      'thread_entry_recipients' => $mailinfo['thread_entry_recipients'],
                      'to-email-id' => $mailinfo['to-email-id'],
                      'autorespond' => ((!isset($mailinfo['passive']))  || (isset($mailinfo['passive']) && ($mailinfo['passive']))) ,
                  );

          So doing that seems to fix the problem (and the 2-line comment-out workaround is no longer needed)....

          So what we are doing is checking if "passive" is set. IF not set, then set autorespond to true, and if it is actually set, then set autorespond to the true/false value of "passive"...

          However is this a legitimate fix to OSTicket to solve this problem, and should be submitted to the source code repository?? Or does this break your intended purpose for the "passive" / "autorespond" flag ??

          Seems reasonable to me, however I don't know what the real intention of "passive" here is...

          ... Anyhow, if this is legitimate, can we use that fix for future versions ?

            TPK

            I will ask the team to see what the intention of passive threading is and get back to you.

            Cheers.

            @TPK

            After reviewing the code and your "fix" I can see that you basically just reversed the functionality of the call. So we are setting autorespond to true ONLY IF passive is not set. If it is set we don't want to autorespond. So your "fix" is not a fix it actually reverses intended functionality.

            Once I have an update on passive threading I'll let you know.

            Cheers.

            @KevinTheJedi

            Okay, so if that is the case, then really the fix belongs somewhere here:

            class.thread.php -> lookupByEmailHeaders -> Line 1490

                    // Passive threading - listen mode
                    if (count($possibles)
                            && ($entry = ThreadEntry::objects()
                                ->filter(array('email_info__mid__in' => array_map(
                                    function ($a) { return "<$a>"; },
                                $possibles)))
                                ->first()
                            )
                     ) {
                        $mailinfo['passive'] = true;
                        return $entry;
                    }

            ... Because incoming emails that are evaluated sometimes hit this logic when lookupByEmailHeaders is called, but lookupByEmailHeaders is not called on every email...

            When lookupByEmailHeaders is not called, then passive is not set, and so we don't have a problem (autorespond will evaluate to true).

            When lookupByEmailHeaders is called, then passive is set here (seemingly every time, at least every time I have tested web based email clients so far), and the agent alerts do not go out.

            ... So I am trying to evaluate what is happening with that conditional logic there, and I think that is where I get a little stuck (I just cannot follow what it is doing there) ....

            So what if I did a variable dump of $possibles, and $entry, and $mailinfo right before the return $entry, for web-based emails that fall into this conditional, and submitted that, would that be enough to figure out what is happening here for these emails??

            Emails that work (do trigger an agent alert) never fall into this code because lookupByEmailHeaders is never called to begin with (and so passive is never set)... Is that a problem too??

            ... Thanks much

              TPK

              What's the difference with ones hitting lookupByEmailHeaders() vs not? I would think the ones not hitting lookupByEmailHeaders() is ones done via the UI as all emails that are fetched go through lookupByEmailHeaders() method - this is how we determine what thread it belongs to (if any).

              Cheers.