Mixing secure (SSL/https) and non-secure (http) on a secure page (https) can get you two kinds of warnings. Links to non-secure sites may generate a warning (when clicked) that the visitor is leaving a secure page and going to a non-secure page or site. They may even get a warning for going to another site, even if it's secure itself (it's on another domain). The other kind of warning is for non-secure images, CSS, Javascript, etc. files being requested by the browser as part of the page loading process. The visitor will usually be prompted as to whether or not they want to load the non-secure material.

I would suspect that the FB warning is not coming from the link itself, but from a logo image or the like being pulled in from the FB site or even your own site, explicitly under http (non-secure). In that case, you could keep a copy of the image in your secure site area, and modify the HTML to pull in your secure copy rather than the non-secure copy. Even if this works to prevent "non-secure content" warnings, your visitors will probably still get a warning that they are leaving your secure page when the click to go to FB. There's nothing that can be done about that, unless FB offers a secure (https) URL. The FB page is non-secure (http), and that's that.