[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Memory management problems in SFL 2.3



I have been testing an application that I have ported from SFL 2.1 to SFL
2.3.  As part of this process I have been running it with the "mpatrol"
heap checking library.  The following is a summary of the problems I have
encountered.  Some of these, together with the 3DES and RC2 performance
issue that I reported a while ago, are serious enough to prevent us being
able to deploy SFL 2.3 in place of SFL 2.1.  Others also affected SFL
2.1 but we hoped that a newer release would eliminate them.

These tests were done using GCC 3.3.2 on a Linux system, but nothing
about them suggests platform-dependence.

We would be interested in knowing when a new release is likely to be
available that remedies these deficiencies, plus the others that I
have reported over the last two weeks.

1. Memory leaks

Both signing and signature verification appear to have significant
memory leaks.  The signing memory leak does not occur in the most
trivial case, but does occur in either of the following circumstances:

a. I have set the flag to include the signer's certificates.
b. I have attached a signed attribute (signing time)

The verify leaks occur when processing the messages that exhibited
a leak during signing.  These leaks are reported in:

a. SNACC::AsnRvsBuf::AsnRvsBuf(SNACC::AsnBuf const&)
   - 8 occurrences per verify operation, 24 per sign operation
b. SNACC::AsnBuf::insert(SNACC::AsnBuf const&)
   - 16 occurrences per verify operation, 48 per sign operation

when using a 50kB message content.  Encryption and decryption do
not appear to suffer from leaks in the same way.  The attached
zip file contains some standalone test programs.  The leaks can be
observed by setting the #define REPEATS to a large value (say 1000)
and watching the process grow using "top".

There is also a leak occurring when reading the PKCS#12 file, in:

   CERT::CSM_Free3::ComputePkcs12MAC(CTIL::CSM_Buffer&,
     SNACC::AsnOid const&, char const*, CTIL::CSM_Buffer const&,
     int)

2. Memory overrun

There is a memory overrun in the code in sm_free3:

   CERT::CSM_Free3::Setup(SNACC::AsnOid)

that calls:

   CryptoPP::X917RNG::X917RNG(CryptoPP::BlockTransformation*,
     unsigned char const*, unsigned long)+160

This code expects to be passed a buffer with at least 8 bytes
whereas sm_free3 is allocating 5 bytes.  This bug also existed
in SFL 2.1.

3. Mismatched library calls

I have encountered three of these.  They are benign with GCC,
but might matter on some platforms.

a. SNACC::AsnFileSeg::AsnFileSeg(char const*) allocates member
   m_filename using strdup() (and hence malloc()) but frees it
   using delete.
b. CM_FreeErrInfo() uses free() to free memory allocated using
   new in CML::ErrorInfo::operator errorInfo_List*() const
c. SNACC::ConsStringDeck::~ConsStringDeck() uses delete rather
   than delete [] to free memory allocated using new[] in
   SNACC::AsnBuf::GetSeg(long).

4. Lack of thread safety

One of my tests attempts a series of encrypt, decrypt, sign and verify
operations in 8 threads at once.  This produces failures with SFL 2.3.
One of my colleagues managed, with some effort, to track down use of
a global variable in SFL 2.1, but the code in SFL 2.3 has changed
significantly.  The exceptions thrown in one run of the test included
the following:

SNACC::SnaccException:
  m_errorCode   6003
  what()   Tag [-335544320] is invalid for type SafeContents
  getCallStack()->function AsnSeqOf<T>:BDec()
  getCallStack()->file  ... SMPDist/include/esnacc/c++/asn-list.h

SNACC::SnaccException:
  m_errorCode   1017
  what()   encapsulated content digest != message digest attribute
  getCallStack()->function
Verify(pCSMIME,pOriginalEncapContent,encapContentInfo,pCerts,pSignedAttrs)
  getCallStack()->file  sm_MsgSignerInfo.cpp

std::exception thrown by crypto++ and not caught within SFL:
  what()   BER decode error

5. Lack of handling of std::exception exceptions thrown by crypto++

crypto++ 5.1 can throw std::exception in a variety of circumstances,
especially when presented with messages with faulty encoding.  The
CSM_Free3 CTIL does not catch these, hence if one occurs it will not
perform cleanup.  This problem also existed in SFL 2.1.

6. Less robust than SFL 2.1

I have been testing with a collection of CMS messages containing
malformed ASN.1, which are designed to test the vulnerability of systems
to denial-of-service attacks.  SFL 2.3 seems somewhat less robust
than SFL 2.1 in this respect, particularly its decryption code.  The
number of failures recorded were as follows:

   SFL 2.1  SFL 2.3
Total number of tests: 1345101  1345101
Assertion failure: 231  3
Segmentation fault: 4363  7073
Infinite loop:  128  214

There may have been more assertion failures with SFL 2.1 because part
of SFL 2.1 was built with debug options.

7. Memory usage processing large messages

If I supply input to the sign, verify, encrypt or decrypt operations
in a CSM_Buffer that is held in a file rather than in RAM, the operations
still allocate further RAM-based buffers rather than file-based ones
during processing.  Memory usage can peak at around 6x the size of the
content.  I haven't come across a way of making SFL allocate file-based
buffers to allow large messages to be processed without consuming lots of
RAM.  This was the same in SFL 2.1.


John Stark
Tel: +44 1223 566732
Mobile: +44 7968 110628
E-mail: jas@xxxxxxxxxxxx
Web: http://www.metanate.com

Attachment: files.zip
Description: Zip compressed data