Built-in Macros in the Linux Kernel

The Linux kernel uses several built-in macros to optimize performance. Two commonly used macros are defined in include/linux/compiler.h:

#define likely(x)       __builtin_expect((x),1)
#define unlikely(x)     __builtin_expect((x),0)        

• likely(x): Used when a condition is expected to be true most of the time, helping the compiler optimize branch prediction.
• unlikely(x): Used when a condition is expected to be false most of the time, aiding in branch prediction optimization.

These macros internally use the built-in function __builtin_expect(). While __builtin_expect() has a name that directly suggests its function, likely and unlikely provide a more readable and intuitive way to use it in the kernel code.

Understanding __builtin_expect()

long __builtin_expect (long exp, long c);        

This is a GCC built-in function that provides branch prediction hints to the compiler. It helps optimize performance by positioning rarely used code paths at the end of a function, improving cache efficiency and reducing branch mispredictions.

• Unlike likely and unlikely, __builtin_expect() can also be used in user-space applications.
• It is particularly useful in performance-critical scenarios where conditional branching is predictable.

Debugging and Branch Profiling

When debugging or profiling branch execution, the kernel uses an extended version of these macros:

include/linux/compiler.h
#if defined(CONFIG_TRACE_BRANCH_PROFILING) \
    && !defined(DISABLE_BRANCH_PROFILING) && !defined(__CHECKER__)
void ftrace_likely_update(struct ftrace_likely_data *f, int val,
			  int expect, int is_constant);

#define likely_notrace(x)	__builtin_expect(!!(x), 1)
#define unlikely_notrace(x)	__builtin_expect(!!(x), 0)

#define __branch_check__(x, expect, is_constant) ({			\
			long ______r;					\
			static struct ftrace_likely_data		\
				__aligned(4)				\
				__section("_ftrace_annotated_branch")	\
				______f = {				\
				.data.func = __func__,			\
				.data.file = __FILE__,			\
				.data.line = __LINE__,			\
			};						\
			______r = __builtin_expect(!!(x), expect);	\
			ftrace_likely_update(&______f, ______r,		\
					     expect, is_constant);	\
			______r;					\
		})

/*
 * Using __builtin_constant_p(x) to ignore cases where the return
 * value is always the same.  This idea is taken from a similar patch
 * written by Daniel Walker.
 */
# ifndef likely
#  define likely(x)	(__branch_check__(x, 1, __builtin_constant_p(x)))
# endif
# ifndef unlikely
#  define unlikely(x)	(__branch_check__(x, 0, __builtin_constant_p(x)))
# endif        

These macros help track branch execution statistics when debugging is enabled.

__builtin_constant_p()

int __builtin_constant_p(x);        

• This built-in function returns 1 if the argument x is known to be a compile-time constant, otherwise it returns 0.
• It is useful in macros that need to differentiate between compile-time constants and runtime values.

Using likely() and unlikely() in Linux System Programs

Since likely() and unlikely() are kernel-defined macros, using them in user-space programs will result in compilation errors. However, you can use __builtin_expect() directly in general C applications.

If a condition in a branch statement is highly predictable, using __builtin_expect() can improve performance. However, if branch predictability is uncertain, it's best to let the compiler handle optimization.

Example Program

Below is a simple test program to demonstrate __builtin_expect() in action:

#include <stdio.h>

int test_func(int input)
{
    if (__builtin_expect(input > 0, 1))
    {
        input += 8;
        input *= 4;
    }
    else
    {
        input -= 8;
        input /= 4;
    }
    return input;
}

int main()
{
    printf("Output: %d\n", test_func(3));
    return 0;
}        

In this example:

• The if condition is expected to be true (input > 0), so __builtin_expect() is used to hint this to the compiler.
• This allows the compiler to optimize the execution flow for better performance.

Using branch prediction optimizations like likely(), unlikely(), and __builtin_expect() can significantly improve performance in large-scale applications, particularly in performance-critical areas such as data processing and system-level programming.