Server Information
osTicket Version v1.12.2 (a5d898b) — Up to date
Web Server Software Microsoft-IIS/10.0
MySQL Version 8.0.17
PHP Version 7.3.7

Same error:
authentication failure [SMTP: STARTTLS failed (code: 220, response: 2.0.0 SMTP server ready)]

\include\pear\Net\SMTP.php has not been updated, I believe the scope of $crypto_method on line 592 is the issue, and the scope of $result on line 594 might also need to be addressed..

EDIT: I am unable to check the mail server logs.

EDIT2: I believe that the modified error message I included confirms that those two variables have not been initialized within the current scope.

KevinTheJedi Is it not because @crypto_method on line 583 is not accessible from line 592?

SMTP.php:592 $crypto_method: int(57)

Socket.php:640 $enabled: bool(true)
Socket.php:640 $type: int(57)
Socket.php:646 $this->fp: resource(101) of type (stream)

SMTP.php:595 $result: bool(false)

How far down the rabbit hole should I go?
@stream_socket_enable_crypto($this->fp, $enabled, $type);

Edit2: Where is stream_socket_enable_crypto?

Edit: $crypto_method initialized on line 588:

$crypto_method = STREAM_CRYPTO_METHOD_TLS_CLIENT
                                 | @STREAM_CRYPTO_METHOD_TLSv1_1_CLIENT
                                 | @STREAM_CRYPTO_METHOD_TLSv1_2_CLIENT;

Note: I cannot check the mail server logs because I am using Office365. I have confirmed through my web UI the port number for SMTP, authenticates with STARTTLS..

Results from https://www.checktls.com

seconds test stage and result
[000.008] Connected to server
[000.015] <-- 220 BL0PR02CA0071.outlook.office365.com Microsoft ESMTP MAIL Service ready at Tue, 30 Jul 2019 20:23:12 +0000
[000.015] We are allowed to connect
[000.015] --> EHLO www6.CheckTLS.com
[000.023] <-- 250-BL0PR02CA0071.outlook.office365.com Hello [159.89.187.50]
250-SIZE 157286400
250-PIPELINING
250-DSN
250-ENHANCEDSTATUSCODES
250-STARTTLS
250-8BITMIME
250-BINARYMIME
250-CHUNKING
250 SMTPUTF8
[000.023] We can use this server
[000.023] TLS is an option on this server
[000.023] --> STARTTLS
[000.031] <-- 220 2.0.0 SMTP server ready
[000.031] STARTTLS command works on this server
[000.059] Connection converted to SSL
SSLVersion in use: TLSv1_2
Cipher in use: ECDHE-RSA-AES256-GCM-SHA384
Certificate 1 of 3 in chain: Cert VALIDATED: ok
Cert Hostname VERIFIED (smtp.office365.com = outlook.com | DNS:.clo.footprintdns.com | DNS:.nrb.footprintdns.com | DNS:.hotmail.com | DNS:.internal.outlook.com | DNS:.live.com | DNS:.office.com | DNS:.office365.com | DNS:.outlook.com | DNS:.outlook.office365.com | DNS:attachment.outlook.live.net | DNS:attachment.outlook.office.net | DNS:attachment.outlook.officeppe.net | DNS:ccs.login.microsoftonline.com | DNS:ccs-sdf.login.microsoftonline.com | DNS:hotmail.com | DNS:mail.services.live.com | DNS:office365.com | DNS:outlook.com | DNS:outlook.office.com | DNS:substrate.office.com | DNS:substrate-sdf.office.com | DNS:attachments.office.net | DNS:.fp.measure.office.com | DNS:attachments-sdf.office.net | DNS:outlook-sdf.graph.microsoft.com)
Not Valid Before: Jan 23 00:00:00 2019 GMT
Not Valid After: Jan 23 12:00:00 2021 GMT
subject= /C=US/ST=Washington/L=Redmond/O=Microsoft Corporation/CN=outlook.com
issuer= /C=US/O=DigiCert Inc/CN=DigiCert Cloud Services CA-1
Certificate 2 of 3 in chain: Cert VALIDATED: ok
Not Valid Before: Aug 4 12:00:00 2015 GMT
Not Valid After: Aug 4 12:00:00 2030 GMT
subject= /C=US/O=DigiCert Inc/CN=DigiCert Cloud Services CA-1
issuer= /C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert Global Root CA
Certificate 3 of 3 in chain: Cert VALIDATED: ok
Not Valid Before: Nov 10 00:00:00 2006 GMT
Not Valid After: Nov 10 00:00:00 2031 GMT
subject= /C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert Global Root CA
issuer= /C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert Global Root CA
[000.139] > EHLO www6.CheckTLS.com
[000.148] < 250-BL0PR02CA0071.outlook.office365.com Hello [159.89.187.50]
250-SIZE 157286400
250-PIPELINING
250-DSN
250-ENHANCEDSTATUSCODES
250-AUTH LOGIN XOAUTH2
250-8BITMIME
250-BINARYMIME
250-CHUNKING
250 SMTPUTF8
[000.148] TLS successfully started on this server

I've yet to succeed in setting up SMTP..

Office.com shows the following settings should work:

Server name: smtp.office365.com
Port: 587
Encryption method: STARTTLS

output from running "php -i" (only lines related to ssl):

Registered Stream Socket Transports => tcp, udp, ssl, tls, tlsv1.0, tlsv1.1, tlsv1.2
curl
SSL => Yes
MULTI_SSL => No
SSL Version => OpenSSL/1.1.1c
imap
SSL Support => enabled
mysqlnd
core SSL => supported
extended SSL => not supported
openssl
OpenSSL support => enabled
OpenSSL Library Version => OpenSSL 1.1.1c  28 May 2019
OpenSSL Header Version => OpenSSL 1.1.1c  28 May 2019
Openssl default config => C:\usr\local\ssl/openssl.cnf
Directive => Local Value => Master Value
openssl.cafile => no value => no value
openssl.capath => no value => no value
pgsql
SSL support => enabled
Phar
OpenSSL support => enabled

Different error displayed through osTicket after adding tls:// to smtp host

Sending Email via SMTP     
Failed to connect to tls://smtp.office365.com:587 [SMTP: Failed to connect socket: stream_socket_client(): unable to connect to tls://smtp.office365.com:587 (Unknown error) (code: -1, response: )]

from "php_errors.log":

PHP Warning:  stream_socket_client(): SSL operation failed with code 1. OpenSSL Error messages:
error:1408F10B:SSL routines:ssl3_get_record:wrong version number in C:\inetpub\wwwroot\osticket\include\pear\Net\Socket.php on line 166
PHP Warning:  stream_socket_client(): Failed to enable crypto in C:\inetpub\wwwroot\osticket\include\pear\Net\Socket.php on line 166
PHP Warning:  stream_socket_client(): unable to connect to tls://smtp.office365.com:587 (Unknown error) in C:\inetpub\wwwroot\osticket\include\pear\Net\Socket.php on line 166

osticket\include\pear\Net\Socket.php lines 148-171

        if ($options && function_exists('stream_context_create')) {
            if ($this->timeout) {
                $timeout = $this->timeout;
            } else {
                $timeout = 0;
            }
            $context = stream_context_create($options);

            // Since PHP 5 fsockopen doesn't allow context specification
            if (function_exists('stream_socket_client')) {
                $flags = STREAM_CLIENT_CONNECT;

                if ($this->persistent) {
                    $flags = STREAM_CLIENT_PERSISTENT;
                }

                $addr = $this->addr . ':' . $this->port;
                $fp   = stream_socket_client($addr, $errno, $errstr,
                                             $timeout, $flags, $context); // <--- Line 166
            } else {
                $fp = @$openfunc($this->addr, $this->port, $errno,
                                 $errstr, $timeout, $context);
            }
        } else {

Thanks for the response!

latest osTicket info from /scp/system.php

Server Information
osTicket Version	v1.12.2 (a5d898b) — v1.12.5 is available
Web Server Software	Microsoft-IIS/10.0
MySQL Version	8.0.17
PHP Version	7.3.7

I don't have [_] Use Separate Authentication. ?

packet dump via WireShark:

packetNumber,time,source,destination,protocol,length,Info
"114","9.996556",osTicketServer,mailServer,"SMTP","70","C: EHLO localhost"
"115","10.025890",mailServer,osTicketServer,"SMTP","165","[TCP Spurious Retransmission] S: 220 SN1PR12CA0109.outlook.office365.com Microsoft ESMTP MAIL Service ready at Fri, 13 Dec 2019 15:16:32 +0000"
"116","10.025935",osTicketServer,mailServer,"TCP","66","[TCP Dup ACK 114#1] 50615 → 587 [ACK] Seq=17 Ack=112 Win=262656 Len=0 SLE=1 SRE=112"
"117","10.026672",mailServer,osTicketServer,"SMTP","258","S: 250-SN1PR12CA0109.outlook.office365.com Hello [XXX.XXX.XXX.XXX] | 250-SIZE 157286400 | 250-PIPELINING | 250-DSN | 250-ENHANCEDSTATUSCODES | 250-STARTTLS | 250-8BITMIME | 250-BINARYMIME | 250-CHUNKING | 250 SMTPUTF8"
"118","10.027100",osTicketServer,mailServer,"SMTP","64","C: STARTTLS"
"119","10.058055",mailServer,osTicketServer,"SMTP","258","[TCP Spurious Retransmission] S: 250-SN1PR12CA0109.outlook.office365.com Hello [XXX.XXX.XXX.XXX] | 250-SIZE 157286400 | 250-PIPELINING | 250-DSN | 250-ENHANCEDSTATUSCODES | 250-STARTTLS | 250-8BITMIME | 250-BINARYMIME | 250-CHUNKING | 250 SMTPUTF8"
"120","10.058084",osTicketServer,mailServer,"TCP","66","[TCP Dup ACK 118#1] 50615 → 587 [ACK] Seq=27 Ack=316 Win=262400 Len=0 SLE=112 SRE=316"
"121","10.058702",mailServer,osTicketServer,"SMTP","83","S: 220 2.0.0 SMTP server ready"
"122","10.059973",osTicketServer,mailServer,"SMTP","571","C: \026\003\001\002\000\001\000\001\374\003\003(H\003b | 9B\274\217\2433\273\002D\305\227\301 \016\274\347\343\260.\241\025a\275\216\265&J\000\000\220\300/\300+\3000\300,\000\236\000\242\000\243\000\237\300'\300#\300\023\300\t\300(\300$\300\024\300 | \000g\0003\000@\000k\0008\0009\000\234\000\235\300\256\300\254\300\242\300\236\0002\300\240\300\234\000<\000/\300\257\300\255\300\243\300\237\000j\300\241\300\235\000=\0005\314\251\314\250\314\252\300]\300a\300W\300S\300\\300`\300V\300R\300s\300w\000\304\000\303\300r\300v\000\276\000\275\000\210\000\207\000E\000D\300Q\300P\000\300\000\272\000\204\000A\000\377\001\000\001C\000\000\000\021\000\017\000\000\f40.97.212.18\000\v\000\004\003\000\001\002\000 | \000\f\000 | \000\035\000\027\000\036\000\031\000\030\000#\000\000\000\026\000\000\000\027\000\000\000 | \0000\000.\004\003\005\003\006\003\b\a\b\b\b\t\b | \b\v\b\004\b\005\b\006\004\001\005\001\006\001\003\003\002\003\003\001\002\001\003\002\002\002\004\002\005\002\006\002\000\025\000\322\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000"
"123","10.084661",mailServer,osTicketServer,"SMTP","1514","S:  |  |  |  |  |  | 190123000000Z\027 | 210123120000Z0j1\v0\t\006\003U\004\006\023\002US1\0230\021\006\003U\004\b\023 |  |  |  |  |  |  |  |  |  |  |  | "
"124","10.126130",osTicketServer,mailServer,"TCP","54","50615 → 587 [ACK] Seq=544 Ack=1805 Win=262656 Len=0"
"125","10.131010",mailServer,osTicketServer,"SMTP","1514","[TCP Spurious Retransmission] S:  |  |  |  |  |  | 190123000000Z\027 | 210123120000Z0j1\v0\t\006\003U\004\006\023\002US1\0230\021\006\003U\004\b\023 |  |  |  |  |  |  |  |  |  |  |  | "
"126","10.131072",osTicketServer,mailServer,"TCP","66","[TCP Dup ACK 124#1] 50615 → 587 [ACK] Seq=544 Ack=1805 Win=262656 Len=0 SLE=345 SRE=1805"
"127","10.145610",mailServer,osTicketServer,"SMTP","1514","S:  |  |  |  |  |  |  |  |  |  |  | 150804120000Z\027 | 300804120000Z0K1\v0\t\006\003U\004\006\023\002US1\0250\023\006\003U\004 |  |  |  | "
"128","10.145970",mailServer,osTicketServer,"SMTP","1069","S:  |  |  |  |  |  | "
"129","10.146041",osTicketServer,mailServer,"TCP","54","50615 → 587 [ACK] Seq=544 Ack=4280 Win=262656 Len=0"
"130","10.255198",osTicketServer,mailServer,"TCP","61","50615 → 587 [PSH, ACK] Seq=544 Ack=4280 Win=262656 Len=7 [TCP segment of a reassembled PDU]"
"131","10.285520",mailServer,osTicketServer,"TCP","60","587 → 50615 [FIN, ACK] Seq=4280 Ack=551 Win=1048560 Len=0"
"132","10.285566",osTicketServer,mailServer,"SMTP","60","C: \025\003\003\000\002\002.RSET"
"133","10.315526",mailServer,osTicketServer,"TCP","60","587 → 50615 [RST, ACK] Seq=4281 Ack=557 Win=155216 Len=0"

I used mxtoolbox.com to find the hostname that ends with .mail.protection.outlook.com
Could not get SSL to work, but did get SMTP working without authentication.