localhost is another way of saying 127.0.0.1 or in simpler terms the server that the software resides on.
With out knowing your mail settings, and default MTA settings in osTicket we wouldn't be able to provide much more information. I can tell you that the error says that osTicket tries to send the email, the mailserver (localhost) says that the email you are using is invalid.