Nss compat ossl

From FedoraProject

Revision as of 04:00, 13 September 2008 by Rcritten (Talk | contribs)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Contents

NSS compatibility for Open SSL apps

The purpose is library is to make converting an existing product that uses OpenSSL to use the NSS crypto library instead and to cause as few changes to the code you are trying to port as possible. Some changes are inevitable, particularly when crypto outside of SSL is being used, but for a general-purpose SSL client or server the goal is that 80% of the code can remain untouched.

In order to shake out the pieces of the API the following packages were converted using nss_compat_ossl:

  • stunnel
  • wget
  • libcurl

Each is lacking something, perhaps something important, but basic SSL support is done.

Currently Supports

  • Creating an SSL server listener and accepting requests
  • Creating an SSL client socket and making requests
  • Ciphers that should be compatible with OpenSSL
  • Client certificate authentication
  • Token password prompting/handling

Things to be done

  • We should import referenced certificates on the fly into our NSS database. A PKCS#11 module to do this has been started but requires NSS 3.12 so it is of limited use in the short-term.
  • Many missing pieces of the API

How To Use the Library

For the short term (see nss_compat_ossl#Reading PEM files) applications will need to use an NSS database. This consists of 3 files: cert8.db, key3.db and secmod.db located in the same directory. In order for the target to find the right database the environment variable SSL_DIR needs to be set to the location of your NSS database (unless an appropriate cert is installed in the default NSS database in /etc/pki/nssdb)

The code doesn't currently support file-based certificates. It uses the path of the certificate passed to SSL_CTX_use_certificate_file() and SSL_CTX_use_certificate_chain_file() as the nickname of the certificate in the NSS database. To list the certificates (and their nickname) in an NSS database you can use this:

% certutil -L -d /path/to/database

If you have a PKCS#12 file containing you can import it into your NSS database with:

% pk12util -i mycert.p12 -d /path/to/database

You can find more examples of the NSS command-line tools here: [1]

The library currently lack nice, importable autoconf rules. Developers will need to tell their application where to find the NSPR and NSS include and libraries. pkg-config with the package names of nss and nspr can be used to determine this.

The variables HAVE_NSS and HAVE_OPENSSL can be used to differentiate between NSS and openSSL in that 20% of cases not handled by nss_compat_ossl.

The include file "nss_compat_ossl.h" must be included, be careful to not include any openSSL header files.

Some specific things to watch out for:

  • OpenSSL CRL handling is very different from NSS so any OpenSSL CRL handling code should be ifdef'd out. NSS handles CRLs directly. Users can use the crlutil tool to load them into the NSS database.
  • The callbacks for info_callback and verify_callback are made but often those functions use very diverse OpenSSL calls that aren't supported yet (and may never be). These callbacks will likely all need to be rewritten for NSS.
  • Few of the BIO_ calls are implemented. If these are used extensively in the target application then some major rewriting may be needed. Best to request some assistance before proceeding.
  • nss_compate_ossl doesn't use OpenSSL structures in most cases so any programs trying to access specific elements may need to change, possibly to use accessor functions.
  • NSS supports two modes for its SSL cache: threaded and multi-process. The nss_compat_ossl code currently initializes the cache for multi-threaded operation. If you need multi-process you will need to call these in your application:

SSL_CTX_set_timeout(ctx, timeout); SSL_ShutdownServerSessionIDCache(); SSL_ConfigMPServerSIDCache(0, timeout, timeout, NULL);

Where Can I get the Source?

svn co http://svn.fedorahosted.org/svn/identity/common/trunk/nss_compat_ossl/

A tarball is available at [2]

The write-able repo is at svn+ssh://user@svn.fedorahosted.org//svn/hosted/identity/common/

Reading PEM files

Work on a PKCS#11 module that can load file-based certificates on-the-fly is in development. This module will let use more closely emulate OpenSSL with little-to-no changes on deployment.

It currently supports:

  • 8 slots. Slot 0 is designed to store all CA certificates and slots 1-7 will hold one certificate and one key pair. The choice of 8 slots is arbitrary and will likely be increased before shipping.
  • Can load encrypted private keys and prompt the user for a PIN (and verify the PIN)
  • Can handle SSL client and server encryption
  • On-par performance as the existing NSS built-in token

What it requires:

  • The API that this module uses, CKFW, is missing some features in the current version of NSS in Fedora 8 (3.11.x). NSS 3.12 will be needed to build this module.

Do applications automatically inherit FIPS 140-2 Compliance simply by linking with NSS?

No, as with using any FIPS validated module, all applications must comply with the security policy of that module document to claim FIPS conformance. The NSS security policy was written so that apps can easily comply with the requirements.

The main NSS FIPS site is http://wiki.mozilla.org/FIPS_Validation

The security policy page starts at http://wiki.mozilla.org/Section_C:_Cryptographic_Security_Policy

The policy document can be found at http://www.mozilla.org/projects/security/pki/nss/fips/secpolicy.pdf

Most requirements are met simply by turning on FIPS mode in NSS. The FIPS module automatically prevents most illegal operations with respect to FIPS compliance. The security policy points out those areas where it does not.

Sample Application

When learning something new there is little better than a concrete example to ease things along.

I'll use stunnel 4.15 as a starting point. It is what I used to help develop nss_compat_ossl.

The first thing we have to do is tell autoconf about nss_compat_ossl and possibly override any default OpenSSL settings. This includes any include directories, libraries and any definitions (like HAVE_OPENSSL).

Here is a diff of the stunnel configure.ac. I'll go into the details of what and why after the fold.

--- configure.ac.orig   2007-07-20 10:45:57.000000000 -0400
+++ configure.ac        2007-07-20 13:52:44.000000000 -0400
@@ -101,6 +101,9 @@

AC_MSG_NOTICE([**************************************** SSL] )
checkssldir() { :
+    if ! test -z "$nss_compat"; then
+        return 0
+    fi
if test -f "$1/include/openssl/ssl.h"
then AC_DEFINE(HAVE_OPENSSL)
ssldir="$1"
@@ -113,8 +116,30 @@
return 1
}

+OPT_NSS_COMPAT=no
AC_MSG_CHECKING([for SSL directory] )
+AC_ARG_WITH(nss_compat,
+[  --with-nss_compat=DIR   location of installed NSS compatibility SSL libraries/include files] , OPT_NSS_COMPAT=$withval)
+
+if test X"$OPT_NSS_COMPAT" != Xno; then
+    if test "x$OPT_NSS_COMPAT" = "xyes"; then
+     check=pkg-config --version 2>/dev/null
+     if test -n "$check"; then
+       addlib=pkg-config --libs nss
+       addcflags=pkg-config --cflags nss
+     fi
+    else
+      # Without pkg-config, we'll kludge in some defaults
+      addlib="-L$OPT_NSS_COMPAT/lib -lssl3 -lsmime3 -lnss3 -lplds4 -lplc4 -lnspr4 -lpthread -ldl"
+      addcflags="-I$OPT_NSS_COMPAT/include"
+    fi
+    CFLAGS="$CFLAGS $addcflags"
+    LIBS="$LIBS $addlib -lnss_compat_ossl"
+    nss_compat="yes"
+    AC_DEFINE(HAVE_NSS_COMPAT)
+    echo "Using nss_compat_ossl instead of OpenSSL"
+fi
AC_ARG_WITH(ssl,
[  --with-ssl=DIR          location of installed SSL libraries/include files] ,
[
]@@ -130,7 +155,7 @@
done
 
)
-if test -z "$ssldir"
+if test -z "$ssldir" -a -z "$nss_compat"
then AC_MSG_RESULT([Not found] )
echo
echo "Couldn't find your SSL library installation dir"

The stunnel configure.ac does things a little differently than others I've seen. They declare a separate function to try to find OpenSSL then set things up from there. What we need to do is short circuit this. There is no AC_UNDEFINE so our stuff has to come first.

The first change in the file is in checkssldir(). This bails out if nss_compat_ossl has been selected. We check for a global variable. The interesting stuff comes next.

The meat is in the --with-nss_compat field. If it it gets set to a directory then we use that path (like --with-nss_compat=/usr/local) and we set CFLAGS and LIBS as best we can. If it is set without a DIR (e.g --with-nss_compat) then we use pkg-config to get things setup for us which is more likely to produce the results we want.

The important things to set are:

AC_DEFINE(HAVE_NSS_COMPAT) LIBS to include the NSS and NSPR libraries and potentially the path they are installed in (if not in /usr/lib) CFLAGS or INCLUDES to include the NSS and NSPR include file locations

The change of "if test -z "$ssldir -a -z "$nss_compat" we want to crap out if neither OpenSSL nor nss_compat_ossl are set.

Now that we can configure things properly you would run something like:

% autoconf % ./configure --with-nss_compat

You should probably verify that the Makefiles define -DHAVE_NSS_COMPAT=1 and that LIBS and CFLAGS are sane.

Next we move onto the code changes. You can tackle it either as an iterative or analytical process. I tend to use the iterative approach myself. This involves trying to build and tackling errors as they come up.

In the stunnel case we run into a problem quite early. common.h is trying to include some OpenSSL headers which it can't find. We need it to use our header instead:

--- common.h.orig       2007-07-20 12:21:27.000000000 -0400
+++ common.h    2007-07-20 12:23:23.000000000 -0400
@@ -287,6 +287,7 @@

/**************************************** OpenSSL headers */

+#ifndef HAVE_NSS_COMPAT
@@ -302,6 +303,9 @@
#endif
+#else
+#include <nss_compat_ossl/nss_compat_ossl.h>
+#endif

/**************************************** Other defines */

When using an iterative approach errors tend to appear as undeclared defines and functions such as this in ssl.c:

ssl.c: In function ‘init_compression’:
ssl.c:63: error: ‘COMP_METHOD’ undeclared (first use in this function)
ssl.c:63: error: (Each undeclared identifier is reported only once
ssl.c:63: error: for each function it appears in.)
ssl.c:63: error: ‘cm’ undeclared (first use in this function)
ssl.c:69: warning: implicit declaration of function ‘COMP_zlib’
ssl.c:74: warning: implicit declaration of function ‘COMP_rle’
ssl.c:81: error: ‘NID_undef’ undeclared (first use in this function)
ssl.c:85: warning: implicit declaration of function ‘SSL_COMP_add_compression_method’
gmake: *** [ssl.o]  Error 1

nss_compat_ossl doesn't support SSL compression so we need to skip this function. One way to do it is with:

--- ssl.c.orig  2007-07-20 14:09:42.000000000 -0400
+++ ssl.c       2007-07-20 14:11:06.000000000 -0400
@@ -59,6 +59,7 @@
}

static void init_compression(void) {
+#ifdef HAVE_OPENSSL
int id=0;
COMP_METHOD *cm=NULL;
char *name="unknown";
@@ -87,6 +88,7 @@
exit(1);
}
s_log(LOG_INFO, "Compression enabled using %s method", name);
+#endif
}

static int init_prng(void) {

You could easily use the reverse of this and use #ifndef HAVE_NSS_COMPAT. The choice is yours.

Our next obstacle is in options.c. A slew of missing BIO_ errors are reported, way to many to include here. A further investigation reveals that the function generates a base64 value from a string passed in. In this case NSS provides a single function to do this so we can patch this with:

--- options.c.orig      2007-07-20 13:40:24.000000000 -0400
+++ options.c   2007-07-20 13:48:19.000000000 -0400
@@ -1263,6 +1263,9 @@
}

static char *base64(char *str) { /* Allocate base64-encoded string */
+#ifdef HAVE_NSS_COMPAT
+     return BTOA_DataToAscii(str, strlen(str));
+#else
BIO *bio, *b64;
char *retval;
int len;
@@ -1284,6 +1287,7 @@
BIO_read(bio, retval, len);
BIO_free(bio);
return retval;
+#endif
}

The last set of problems is a bit more challenging. Lets break down each change in the diff.

The first change is in verify_init():

--- ctx.c.orig  2007-07-20 13:51:00.000000000 -0400
+++ ctx.c       2007-07-20 13:52:10.000000000 -0400
@@ -304,6 +304,7 @@
}

if(section->crl_file || section->crl_dir) { /* setup CRL store */
+#ifdef HAVE_OPENSSL
revocation_store=X509_STORE_new();
if(!revocation_store) {
sslerror("X509_STORE_new");
@@ -341,6 +342,7 @@
}
s_log(LOG_DEBUG, "CRL directory set to %s", section->crl_dir);
}
+#endif
}

SSL_CTX_set_verify(ctx, section->verify_level==SSL_VERIFY_NONE ?

NSS handles CRLs via its database. OpenSSL imports them from flat files on startup. So we can #ifdef around a fairly large block of code as NSS will handle all this for us. It does mean though that the user that runs the program will need to preload the CRL into the NSS database using /usr/bin/crlutil before starting stunnel. So this should be documented somewhere.

@@ -352,6 +354,7 @@

static int verify_callback(int preverify_ok, X509_STORE_CTX *callback_ctx) {
/* our verify callback function */
+#ifdef HAVE_OPENSSL
char txt[STRLEN] ;
X509_OBJECT ret;
SSL *ssl;
@@ -389,11 +392,13 @@
/* errnum=X509_STORE_CTX_get_error(ctx); */

s_log(LOG_NOTICE, "VERIFY OK: depth=%d, %s", callback_ctx->error_depth, txt);
+#endif
return 1; /* Accept connection */
}

This change is also very significant. In both OpenSSL and NSS you can write your own callback to verify certificates, CRLs, etc. nss_compat_ossl does a lot of this for the user so we can #ifdef around this code. It is possible to go ahead and define a verify_callback that does real work in both but the nss_compat_ossl API is missing a lot of functions typically found in this verification so you will have to proceed cautiously.

nss_compat_ossl automatically checks for certificate trust and valid dates. If there are other things you require you can continue to do them in a verify_callback() but it may be tricky.

NSS handles CRL checking automatically so we can ignore all of the following code:

/* Based on BSD-style licensed code of mod_ssl */
static int crl_callback(X509_STORE_CTX *callback_ctx) {
+#ifdef HAVE_OPENSSL
X509_STORE_CTX store_ctx;
X509_OBJECT obj;
X509_NAME *subject;
@@ -506,6 +511,7 @@
}
X509_OBJECT_free_contents(&obj);
}
+#endif
return 1; /* Accept connection */
}

And finally a very important change. nss_compat_ossl does not necessarily mimic the data structures of OpenSSL so any attempts to directly access parts of a structure will probably fail. In nss_compat_ossl there is only a logical difference between an SSL structure and a SSL context structure. So we can't pass in s->ctx. But we can pass in s to get what we want. This doesn't actually do much since we aren't calculating this data. I included this to demonstrate one possible workaround.

@@ -525,7 +531,11 @@
SSL_alert_type_string_long(ret),
SSL_alert_desc_string_long(ret));
else if(where==SSL_CB_HANDSHAKE_DONE)
+#ifdef HAVE_NSS_COMPAT
+        print_stats(s);
+#else
print_stats(s->ctx);
+#endif
}

static void print_stats(SSL_CTX *ctx) { /* print statistics */

Now stunnel should build but it isn't quite ready to run yet. With NSS you have to tell it where it can find the key and certificate databases. We use the environment variable SSL_DIR to specify one (the default is /etc/pki/nssdb).

Congratulations. You've ported your first application with just a handful of changes to the code.