When you say that you are prompted to log in with domain credentials, do you mean you get a pop up box like "Windows Security"? Or do you mean OST's login page?
If you mean the former, then you want to check the following:
1. If you open the web.config file in the directory /path/to/osticket/scp
Look to see if someone set manual password protection. It would look something like:
<configuration>
<system.webServer>
<security>
<authorization>
<add accessType="Deny" users="?" />
</authorization>
</security>
</system.webServer>
</configuration>
2. If someone hasn't set manual password protection then make sure that anonymous user has access to that directory. If they do not then one has to authenticate to get the credentials necessary to load the page contained in said directory.
Fire up inetmgr (IIS Manager), in the left pane drill down to the directory under the website. (ie /site/scp). In the middle pane double click "Authorization Rules". The resulting screen should have a list of Authorization Rules. Mine says:
mode: Allow
Users: All Users
Entry Type: Inherited
You can change things in here to permit specific users, groups, etc to be the ones capable of accessing it.
I hope this points you in the right direction. :)