After some investigation I found that they used a certificate created by the internal CA, not exactly best practice but it should work.
Network monitor reveals that communicator will not even try to download the address book, it only contacts the URL and then says goodbye real quick. Well, I finally figured it out.
Certificate contains ‘Certificate Revocation List Distribution Points’ with a URL that is only reachable from the internal network. When communicator cannot reach the CRL distribution point stated in the certificate it will not go further and simply don’t download the address book.
What probably could have been done here is to make the CRL available to internet and change the path in the certificate (that means creating a new certificate). Not so simply in practice.
A simpler way is to change the client behavior not to bother with the CRL. It is not the recommended way, but it will do the job.
Change the setting in Internet Explorer advanced settings; uncheck the “Check for server certificate revocation” checkbox and you should be fine.
The best solution to this is to use a certificate from a commercial certificate authority that is trusted by most computers. With a public trusted certificate, federation and PIC/PIM can be used if there is a need.
The good side is that customer got the address book download working and they will change there own certificate for publically trusted one with reachable CRL paths and also enable federation with some of there partners.