There is no setting to enable/disable Replies from Agents. That is only controlled in one place and that’s the dropdown above the reply box. Either your SMTP is failing for the mail responsible for sending replies, you have an issue with your Response/Reply email template, or there is an issue outside of osTicket.
Cheers.