Problems Fetching Emails

PaSta79

Hi everyone,I need some help - I have osTicket v1.9.12 installed and running - the instance is running since July now without any issues.Since last week we noticed that not all emails were fetched into tickets so I logged on to the email account manually and yes there were 83 emails still there - so I tried to run the rcron.php manually and I receive the following error:#!/usr/bin/php -q

HTTP/1.1 500 Internal Server Error

Date: Tue, 27 Oct 2015 18 GMT

Server: Apache

Set-Cookie: OSTSESSID=19ee9644380c366bad4c5b09990551cb; expires=Wed,

28-Oct-2015 18 GMT; Max-Age=86400; path=/;

domain=www.xyz-support.com; secure

Expires: Thu, 19 Nov 1981 08 GMT

Cache-Control: no-store, no-cache, must-revalidate, post-check=0,

pre-check=0

Pragma: no-cache

Vary: Accept-Encoding

Connection: close

Transfer-Encoding: chunked

Content-Type: text/html; charset=utf-8

I already changed the PHP version from 5.6 back to 5.5 - no difference - I also deleted the mailbox and recreated it...same issue.I also changed the settings from POP to POP+SSL and even tried IMAP - same result - I already have 10 emails in there that are not fetched into tickets for some reason.....I already contacted the technical guys for my server and asked them to check the log file for the 500 error but they said they cannot find any :-(

PaSta79

I got some more Info from my provider - they did run the cron.php directly on the server and they receive the following error which probably causes the problem but why ?PHP Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 81 bytes) in /www/htdocs/w0123a92/xyz-support.com/include/class.orm.php on line 1064

Chefkeks

PHP Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 81 bytes) in /www/htdocs/w0123a92/xyz-support.com/include/class.orm.php on line 1064Seems like there is not enough memory so the cron can be executed along with other applications running on the webserver. What is the memory limit currently?

PaSta79

php info shows:memory_limit --> Local Value 512M Master Value 512MThis is why I also wonder why I receive the message allowed memory size of 134217728 bytes exhausted.Is osticket changing that internally with a php ini set ?I also noticed now that some emails are imported some are not and when I look into the mailbox the mails are not deleted anymore.....

Chefkeks

Q: Is osticket changing that internally with a php ini set ?A: No, osTicket doesn't change that internally.Strange thing is that the memory_limit does not correspond with the error message, so there must be something wrong in my opinion. I quickly search for a script to show how much memory can be allocated really on a server and stumbled across this here:http://magazine.joomla.org/issues/issue-dec-2010/item/295-Are-you-getting-your-fair-share-of-PHP-memoryMy suggestion is to try it out and see what amount of memory is shown / can be allocated. You maybe need to change the value in the file (you can download there) since if I see it correctly it will only allocate up to 100MB, but I guess you'll find it out ;)Let us know the results.

PaSta79

This is the result:You have allocated 504M (528727280) memory in this php scriptYou have allocated 505M (529775856) memory in this php scriptYou have allocated 506M (530824432) memory in this php scriptYou have allocated 507M (531873008) memory in this php scriptYou have allocated 508M (532921584) memory in this php scriptYou have allocated 509M (533970160) memory in this php scriptYou have allocated 510M (535018736) memory in this php scriptYou have allocated 511M (536067312) memory in this php scriptFatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 536870913 bytes) in /www/htdocs/w0136a92/foa-support.com/memtest.php on line 15

Chefkeks

Ok, looks definitely good.But now in my mind comes something different. Often there are different php.ini config files for cli (command line) and webserver (e.g. apache). Maybe the cli php.ini has another value configured as the webserver php.ini and so the cron job cannot be executed correctly.Don't know if you can check the values / see the config file, but on linux the php.ini for cli is often under /etc/php5/cli/php.ini.

PaSta79

No I can not BUT I am running the Cron Job using the webserver - I am calling the /api/rcron.php - the problem is just that if I do this I only receive the message from my first posting - (internal error 500) - only if I run the cron.php using cli I receive the message of memory size exhausted.

PaSta79

So it actually seems that once an email is there - I receive that error above - the email is then not fetched into a ticket and with the next run it is not tried to be fetched again.So I am really facing an issue here which I can't fix or find out what the problem is.My provider sent me a confirmation that nothing was changed on the server or database environment within the last 4 weeks and the problem started to happen last week.

ntozier

does the email have an attachment?does your web host use a different version of php for CLI?

PaSta79

No not really:CLI: PHP 5.5.26-nmm1 (cli) (built: Jun 16 2015 19)Apache: PHP Version 5.5.26-nmm1 (built: Jun 16 2015 17 )I never used cli for osticket before - I just did this to php cron.php to see if there is any error message since the rcron.php only gives back an internal server error - this is how I got to that second message.Yes the emails sometimes have attachments - sometimes they don't - I sent some test emails with and without attachments and some were imported some not.It is really very strange - the problem is that the emails that were imported correctly are not even deleted from the mailbox (also tried to switch to move to folder) since I guess the script is aborted before deletion - so at the end of the day I have around 100 emails in the mailbox and have no idea what is in osticket and what isn't.I already wrote an email to your support address - if this isn't working - is there a way of you delivering professional support ?

PaSta79

I had my hoster raise the Memory Limit also for cli to see what happens but still:PHP Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 32 bytes) in /www/htdocs/w1236a92/xyz-support.com/include/class.filter.php on line 325Or this one:PHP Fatal error: Allowed memory size of 536870912 bytes exhausted

(tried to allocate 81 bytes) in

/www/htdocs/w0123a92/xyz-support.com/include/class.orm.php on line 1064So it seems that these classes for some reason start to overflow - I am pretty sure I could assign as much memory as I wanted and it won't be enough....

Chefkeks

Based on the error lines in the log/www/htdocs/w1236a92/xyz-support.com/include/class.filter.php on line 325/www/htdocs/w0123a92/xyz-support.com/include/class.orm.php on line 1064Code online:https://github.com/osTicket/osTicket-1.8/blob/develop/include/class.filter.php#L325https://github.com/osTicket/osTicket-1.8/blob/develop/include/class.orm.php#L1064and the code behind that lines I think the following:Line 325 of class.filter.php is part of a function that changes the senders email address to the reply-to email address. Line 1064 in class.orm.php seems to set some kind of output and maybe related here too.Do you have any filter (because of class.filter.php line 325 error) which shall replace the senders email address with the email address specified in the reply-to field?If so, disable it and test again.If not, but if you have other filters, you can also try to disable all filters and see if that helps.

PaSta79

We have version v1.9.12 but anyway - you are right we do have such a filter since we had the issue that some companies send us emails from no-reply@xyz.com with the reply-to address accounting@xyz.com - so we had a filter that changes that.I have just disabled this filter - I will have an eye on the console for next 1-2 hours and report what happened.But even if this was the issue - shouldn't the filter work ?Perhaps I set it up wrong but it did work for quiet a while.....

PaSta79

It really seems that it worked - all emailes in the mailbox were now fetched.The filter I disabled is the following:

29-10-2015 09-21-46.jpg

ntozier

You know kinda think of it someone else reported an issue very similar to this and the problem was their filter too. I think that you might be on to something. Please try deleting the filter and recreating it exactly as it was and see if that helps.

Chefkeks

I wouldn't set it that way:Reply-To-Email ... does not contain ... NULLMy guess is that the system cannot handle "NULL" and this causes the issue.Instead I'd try the following since every mail must contain @ symbol:Reply-To-Email ... contains ... @

PaSta79

I tried setting the rule to Reply-To-Email ... contains ... @and....boom I got the same error of memory overflow again :-(

Chefkeks

Wow that's really weird.

Can you answer me some other questions which hopefully can help:

How often do you fetch emails?

How many accounts do you fetch?

PaSta79

We do a fetch every 5 minutes and fetch 2 mailboxes.