*/5 * * * * nobody pathtopublichtml/public_html/api/cron.php
I pretty sure you have to remove the "nobody". Specifying a user (like this "nobody") is only supported for "root". Your Cron is running as your user.
But am I supposed to be editing the standard cron.php to respond to this file.
Why? But as http://osticket.com/wiki/POP3/IMAP_Settings(http://osticket.com/wiki/POP3/IMAP_Settings) suggests, you should include the path to the PHP CLI to run cron.php.
There it also says "Please refer to your hosting manual or contact your hosting provider for instructions" ;)