There are two behaviours in SSL that seem to catch out a number of people.
The first is the use of close_notify.
close_notify is an operation in SSL that terminates the SSL session, providing a definite end to the stream that is being protected. As it provides an HMAC summarising the entire communication so far, it’s a solid, reliable record that your stream has not been interrupted in its journey to you, and that you have received the entire stream.
Consider close_notify to be an essential part of the stream, when you’re writing a stream, but think carefully about the possibilities when reading the stream.
In many protocols, there is some other component of the stream that can already indicate an end, and the HMAC protecting that component (which SSL considers to be simply data) can be relied upon to indicate that you have not been interrupted by evil-doers. HTTP, for instance, sends either a byte count, or uses chunked-encoding, where each chunk is counted. As a result, the close_notify doesn’t tell you anything more than you already know about the stream - it has finished. So, a lack of close_notify in an HTTPS stream, while a sign of technically poor SSL development, is not a fault in the use of SSL.
FTPS, on the other hand, is a different beast – data transfers under FTPS, if they are protected by SSL, start at the beginning of the SSL session, and finish at the end of the SSL connection. SSL connections can terminate either with a close_notify, or with a TCP FIN – closing the underlying stream. SSL’s design assumes that the TCP FIN can be forged – and indeed, so it can, in many environments. So, for a protocol like FTPS, a close_notify should really be treated as essential – though the last time I looked, out of a dozen FTPS clients I tried, only one actually sent the close_notify at the end of the upload.
Here’s a link to “how to send a close_notify at the end of an SSL connection“.
The second ‘gotcha’ is how to correctly handle client certificates.
An SSL server always has a certificate that it uses to identify itself. A client might have one or several certificates. Client certificates are requested by the SSL server if the SSL server wants them; in some cases, an SSL server might be able to accept that the client has no certificate.
For instance, going back to FTPS again, a connection to the FTPS server might be authenticated by certificate exchange, or – after the SSL session has been initiated – by using a username and password. Remember, sending a user name and password over SSL means that they are protected from snooping by everyone except the FTPS server administrator. So, an FTPS server might very reasonably ask for a client certificate, but not mind if the client hasn’t got one.
If a server can accept a certificate from the client, the only way it can indicate this is to ask for mutual authentication. Some clients are coded with the idea that a request from the server for mutual auth is actually a requirement for mutual auth, and they will throw up an error if they don’t have a client certificate, or the user hasn’t selected a certificate to use.
This is incorrect behaviour – if the client has been asked for a certificate, but cannot provide any, it should simply respond with a list of certificates that is empty. This only becomes an error condition if the server requires at least one certificate from the client.