Overloading Global operator new and delete

June 7, 2012 · by · in Programming

Programmers are often advised not to overload global operator new, and to stick to overloading just class specific operator new. The arguments against are many, but they basically boil down to that it is difficult to do it correctly. But if you can do it, it opens up a bunch of wonderful possibilities: you can write your own global heap management where you can better track heap usage, track memory leaks, implement thread affinity and make memory management optimizations specific to your usage pattern.

We’ll work through the issues of doing it right. But first the basics. To overload global operator new and delete you just need to implement the following functions:

   inline void* operator new     ( size_t size ) { return myAlloc( size ); }
   inline void* operator new[]   ( size_t size ) { return myAlloc( size ); }
   inline void  operator delete  ( void* ptr   ) { myFree( ptr ); }
   inline void  operator delete[]( void* ptr   ) { myFree( ptr ); }

This is deceptively simple to do. You don’t even need to find the prototypes for the overloaded functions. They are known to the compiler.

To do anything interesting, other than a straight pass through to malloc() and free() you will need to allocate some data structures using an overloaded new and delete. So now you have problems:

  • To construct and destroy the heap data structures you’ll be using operators that call in to the very data structures that are not yet constructed.
  • C++ does not guarantee, without some tricks, that your heap gets constructed first. So some of your global data structures will try to allocate from an unfinished heap.
  • Similarly, there is no guarantee that your heap gets destroyed last. So global data structures may call deleteafter your heap has been destroyed.

Interfacing to your Heap

Here is the heap interface we would like to have, in say mem.h

void* myAlloc( u32 size );
void  myFree( void* ptr );
void* myRealloc( void* mem, u32 size, u32 oldSize = U32_MAX );

But we’re going to have to implement it with function pointers:

typedef void*(*myAllocFunc)( u32 );
typedef void(*myFreeFunc)( void* );

extern myAllocFunc myAlloc_i;
extern myFreeFunc  myFree_i;
void*              myRealloc_i( void* mem, u32 size, u32 oldSize = U32_MAX );

This is so that we can change the alloc and free functions that get called depending on whether the heap is constructed or not. Before the heap is constructed, and after it is destructed we will call malloc() and free() directly. Our own allocator should be able to recognize its own pointers, such that if it receives a pointer that was allocated with malloc() we can still destroy it. But we will have to be careful never to pass pointers allocated with our own heap into free().

Here are the real allocation and free functions we’re going to use, in mem.cpp

static void* myAlloc_init( u32 size );
static void* myAlloc_init2( u32 size );
static void* myAlloc_run( u32 size );
static void* myAlloc_shutdown( u32 size );

static void myFree_run( void* mem );
static void myFree_shutdown( void* mem );

myAllocFunc myAlloc_i = myAlloc_init;
myFreeFunc  myFree_i  = myFree_run;

You can see that we initialize the allocation and deallocation functions to some pass throughs to malloc() and free(). During initialization of the heap, we will call in to myAlloc_init2(). While the heap is running we will call myAlloc_run(). And during shutdown, if we need to allocate memory, we will call in to myAlloc_shutdown(). All of them except myAlloc_run() are some variant of calling malloc().

Similarly, for the deallocator, we have a myFree_shutdown() that calls free() directly when we need to free anything during shutdown of the global heap. We don’t need a myFree_init() because we don’t expect to free anything during heap construction. But if you did – then you’d need to implement that too.

We’ll fill these functions in later.

Guaranteeing that your Heap Gets Constructed First

The trick here is to allocate the heap on first use. This means two things – prevent the default constructor of the global heap structures from running, and to call that constructor manually the first time any code tries to allocate memory.

So in mem.cpp:

class KxMemGlobals
{
public:
   // your global heap data structures here.
};

// we reserve some memory to hold the mem globals
static char memGlobals[sizeof(KxMemGlobals)] = { 0 };

// and we create a fake class reference to this memory.
static KxMemGlobals& gMem = 
   *(reinterpret_cast(&memGlobals[0]));

// implement our first allocation function that constructs 
// our heap on first use.
static void* myAlloc_init( u32 size )
{
   myAlloc_i = myAlloc_init2;
   static KxMemGlobals* mem = new(memGlobals, NULL) KxMemGlobals;
   myAlloc_i = myAlloc_run;

   return myAlloc_run( size );
}

// This gets called during the initialization of memGlobals only.
static void* myAlloc_init2( u32 size )
{
   return malloc( size );
}

// The normal allocator that runs after heap construction, and before
// its eventual destruction.
static void* myAlloc_run( u32 size )
{
   return gMem.alloc( size );
}

// Allocates memory during shutdown of the memory manager
static void* myAlloc_shutdown( u32 size )
{
   return malloc( size );
}

The constructor to our global heap is prevented from running by allocating its memory as a non-class static buffer. The constructor to our heap structures gets called manually the first time anything tries to allocate memory.

During allocation of the heap, we set the allocation function to myAlloc_init2(). When the heap is fully constructed, we set the allocation function to myAlloc_run(). This way we guarantee that all the code in our program that calls newor delete will use our own heap – except for the heap itself, which is constructed using malloc()

myAlloc_shutdown() also just calls malloc() in this example. But we want a separate function. In reality you may want to do something a little different here. For one, you probably should not be allocating memory after heap destruction. We’ll be making sure heap destruction is the last thing our program does – so no allocations should be happening after heap shutdown. You may want to assert on, or report any such allocations. If you are trapping signals, you may get an allocation in a late firing signal handler – for example if you get a crash during the heap destruction itself and you’ve trapped SIGSEGV.

Making sure the Heap gets Destructed Last

We’ve allocated the memory manager using placement new, which means that the compiler will also not call the destructor automatically. You have to call it manually, which is great, because we want to control when it gets called anyway. To make sure it gets called last, we register the destructor with atexit() during heap construction.

static void kxMemExit()
{
   gMem.~KxMemGlobals();

   // leakCheckReport();
}

KxMemGlobals::KxMemGlobals()
{
   atexit( kxMemExit );

   // construct your heap here
}

KxMemGlobals::~KxMemGlobals()
{
   myAlloc_i = myAlloc_shutdown;

   // destruct your heap here

   myFree_i = myFree_shutdown;
}

static void myFree_shutdown( void* mem )
{
   free( mem );
}

static void myFree_run( void* mem )
{
   gMem.free( mem );
}

So, first we create an atMemExit() function that calls the heap destructor manually. This is also a good place to report memory leaks, if you’ve implemented that. By now everything should be destructed. Make sure your leak reporter is not a member of your heap of course. We register this function with atexit() at the top of the heap constructor. You want to do this as soon as possible, because multiple functions can we registered this way. They get called in reverse order of registration. So if there is any chance a subsidiary data structure also wants to call an exit function, you want to make sure the heap’s exit function gets called last.

There is a small chance that the first data structure constructed globally might register an exit function before its first call to new. Your program will crash if that atexit function does any memory management, since that function will get called after the heap is destroyed. Of course if that function both allocates and frees memory it will be fine, since they will be matched calls to malloc() and free(). But if it tries to free memory allocated after heap construction you will crash.

In the heap destructor you first set the allocation function pointer to your shutdown allocation function. From this point on you can safely destroy the heap, since any new (and likely erroneous) allocs will safely go to malloc() to be called by free() later.

From this point forward you are in a critical period where the heap is half-constructed. If your heap allocated new memory with its own allocator while it was running, you need to remove it carefully – without calling delete directly, since calls to delete will still be routed to the heap which is half destructed.

Finally we set the pointer to the deallocation routines to the pass through to free with myFree_shutdown(). Again, your code should not be calling this anyway, since this function is the last thing that runs ( except possibly a late firing signal or exception handler. )

Now you just have to implement a decent heap replacement. There is no point in doing this unless you intend to implement a heap that is efficient, thread safe, and has good reporting features like leak checking, corruption detection, and usage reporting.

Calling the Heap Directly

Once implemented, your heap will be called automatically whenever you use new and delete. Even third-party libraries will now be routed into your heap.

But you may also want to call myAlloc(), myFree() and myRealloc() directly. Especially if you want to use realloc, which doesn’t get called by default in C++.

It is a good idea to call these using preprocessor definitions. This way you can route the calls to special versions, for example if you want to make special builds with features like leak detection, that you don’t want to happen all the time:

typedef void*(*myAllocFunc)( u32 );
typedef void(*myFreeFunc)( void* );

extern myAllocFunc myAlloc_i;
extern myFreeFunc  myFree_i;
void*              myRealloc_i( void* mem, u32 size, u32 oldSize = U32_MAX );

#define myAlloc   myAlloc_i
#define myFree    myFree_i
#define myRealloc myRealloc_i

You may also notice that we don’t need to call myRealloc() through a function pointer. Realloc will only ever get called after the heap is constructed because you have to call myAlloc() first, and that will build the heap. And you probably don’t want to call myRealloc() after system shutdown either. If you did – then make a function pointer for it, and route the calls to a myRealloc_shutdown() that calls realloc(). Even then you’ll need to make sure you’re only realloc’ing memory that was allocated post heap destruction through malloc() and not some leaked buffer that depends on your heap.

Another Reason to use Function Pointers

Recent versions of g++ have a bug where, in optimized builds, overloaded global new and delete sometimes don’t get called if they are declared inline. Making this work is another reason to implement this with function pointers.

Resources:

How can I overload new & delete in C++ An example of overloading global new and delete
Advanced Memory Handling: New and Delete Some more code examples of overloading new and delete
What’s the static initialization order fiasco? Controlling the order of global constructor calls
library interposers An alternative strategy to replace the global heap
Any reason to overload global new and delete? A discussion on the benefits of overloading new and delete
overloading new/delete A thread advising never to overload global new and delete
Tags: ,