Runtime vs Compilation speed – The new deal

Years ago on many projects we would gladly accept a longer compile time for a faster execution at runtime. However with the acceptance of continuous integration gaining momentum, it’s not “always” a clearly acceptable trade-off. With more frequent code submissions, it’s become desirable to compile and test each submission as independently as possible. This (ideally) allows any compilation errors or test failures to be rapidly traceable to the rogue coder’s submission. Now the trade-off hasn’t just changed. It’s mostly gone. We need to both have fast compile times, and blazing execution. Luckily, this is where judicious software design can help.

The idea of keyword inline is to speed up code execution by inserting the definition of the inlined function directly a the site of the function call. You can read about in the standard [dcl.inline / 10.1.6]. For a function definition of a single line, we can see how avoiding a function call can be beneficial. That single line of code gets inserted directly into the calling code. But, there a plenty of reasons why you mostly don’t want to use inline, unless that function to be inlined is very simple and short as described above.

So far I’ve mentioned using keyword inline, but a function can also be implicitly inlined. Such is the case when a function is defined in the header file, as described in the standard [ class.mfct / 12.2.1.1]

“A member function may be defined (11.4) in its class definition, in which case it is an inline member function (10.1.6), or it may be defined outside of its class definition if it has already been declared but not defined in its class definition.”

struct ExplicitInline
{
    ExplicitInline();
    ~ExplicitInline();

…

    inline void InlinedFunction();
    int a;
};

// Could have been defined in the header
// along with the declaration.
void ExplicitInline::InlinedFunction() 
{ return a; }

OR

struct ImplitInline
{
    ImplitInline();
    ~ImplitInline();

…

    void ImplitInline() { return a; }
    int a;
};

So far this looks fairly innocuous. However the reason you want to define your functions outside their header file are:

1. You may still have a function definition in the translation unit. Why this affects compilation / linking efficiency: You may still have a function definition present in the translation unit if the function is ODR-used [basic.def.odr / 6.2]. This means it increases the size of the translation unit. This in turn means the compiler has to spend extra time evaluating that the inline call is appropriate, copy the function definition where it is called. We’re therefore wasting precious compilation and linking time. Best to wait until a profiler has identified this function for optimization.

2. Another reason for avoiding inlined functions is maintenance. Why this affects compilation / linking efficiency: As other devs might change your well thought out designs, a your once short and tidy inlined functions may grow. As this happens they may not be such good candidates for inlining, yet the inline keyword may not be re-evaluated and end up costing more than planned. Another reason that keeps coming back is that changes to header files often cause a full recompilation of the code base using this header. Best to avoid that unless necessary.

Finalthoughts: Best to avoid early optimization and define your functions outside of the header. If the profiler indicates possible optimization, you can easily add the inline keyword in your headerafterwards.

Further reading: This page from the chromium projects discusses some best practices related to this post as well as a few other interesting items.