Qfyilyi

While working on an embedded project, I have encountered a function, which is called thousands of times in application's lifetime, often in loops, dozens of times per second. I wondered if I can reduce its cost and I found out, that most of its parameters are known during compilation.

Let me illustrate it with an example.

Original hpp/cpp files can be approximated like this:

original.hpp:

void example(bool arg1, bool arg2, const char* data);

original.cpp:

#include "ex1.hpp"

#include <iostream>



void example(bool arg1, bool arg2, const char* data)

{

    if (arg1 && arg2)

    {

        std::cout << "Both true " << data << std::endl;

    }

    else if (!arg1 && arg2) 

    {

        std::cout << "False and true " << data << std::endl;

    }

    else if (arg1 && !arg2) 

    {

        std::cout << "True and false " << data << std::endl;

    }

    else 

    {

        std::cout << "Both false " << data << std::endl;

    }

}

Let's assume, that every single time the function is called, arg1 and arg2 are known during compilation. Argument data isn't, and for variety of reasons its processing cannot be put in header file.

However, all those if statements can be handled by the compiler with a little bit of template magic:

magic.hpp:

template<bool arg1, bool arg2>

void example(const char* data);

magic.cpp:

#include "ex1.hpp"    

#include <iostream>



template<bool arg1, bool arg2>

struct Processor;



template<>

struct Processor<true, true>

{

    static void process(const char* data)

    {

        std::cout << "Both true " << data << std::endl;

    }

};



template<>

struct Processor<false, true>

{

    static void process(const char* data)

    {

        std::cout << "False and true " << data << std::endl;

    }

};



template<>

struct Processor<true, false>

{

    static void process(const char* data)

    {

        std::cout << "True and false " << data << std::endl;

    }

};



template<>

struct Processor<false, false>

{

    static void process(const char* data)

    {

        std::cout << "Both false " << data << std::endl;

    }

};



template<bool arg1, bool arg2>

void example(const char* data)

{

    Processor<arg1, arg2>::process(data);

}



template void example<true, true>(const char*);

template void example<false, true>(const char*);

template void example<true, false>(const char*);

template void example<false, false>(const char*);

As you can see, even on this tiny example cpp file got significantly bigger compared to the original. But I did remove a few assembler instructions!

Now, in my real-life case things are a bit more complex, because instead of two bool arguments I have enums and structures. Long story short, all combinations give me about one thousand combinations, so I have that many instances of line template void example<something>(const char*);

Of course I do not generate them manually, but with macros, yet still cpp file gets humongous, compared to the original and object file is even worse.

All this in the name of removing several if and one switch statements.

My question is: is size the only problem with the template-magic approach? I wonder if there is some hidden cost with using so many versions of the same function. Did I really saved some resources, or just the opposite?

The problem with an increased binary size is almost never the storage of the file itself - the problem is that more code means a lower % of the program instructions are available in cache at any point, leading to cache misses. If you're calling the same instantiation in a tight loop, then having it do less work is great. But if you're constantly bouncing around between different template instantiations, then the cost of going to main memory to load instructions may be far higher than what you save by removing some instructions from inside the function.

This kind of thing can be VERY difficult to predict, though. The way to find the sweet spot in this (and any) type of optimization is to measure. It is also likely to change across platforms - especially in an embedded world.