Jorge Niedbalski: Merging memory pages with same content.

Since Linux Kernel version 2.6.32 there is a mechanism for de-duplication
and sharing of memory pages called Kernel Same page Merging.

KSM enables dynamic sharing of identical pages found in different
memory areas, even if they are not shared by fork.

If KSM is enabled on the system, there is a daemon running in
ring-0 called ksmd which scans memory areas marked as mergeable with identical content,
if a coincidence is found, ksmd replaces that page with a single write-protected page.

The way an application can advise the kernel on which
pages are candidates to be merged is by using the madvise system call with
the MADV_MERGEABLE flag.

I did a quick testing on this kernel feature using a small C program.

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/mman.h>
#include <unistd.h>
#include <fcntl.h>

#define N_PAGES 3

void p_s(void) {  
    int fd;
    char buff[1];
    sleep(3);
    fd  = open("/sys/kernel/mm/ksm/pages_sharing", O_RDONLY);
    read(fd, &buff, 1);
    printf("Sharing pages: %d\n", atoi(buff));
    close(fd);
}

void main(void) {  
    int i
    size_t p_size = sysconf(_SC_PAGE_SIZE);

    void **pages = (void **)calloc(N_PAGES, sizeof(void *));

    //I create 3 pages and fill them with zeroes
    for(i = 0;i < N_PAGES; i++) {
        ((pages[i] = mmap(NULL, p_size, PROT_READ | PROT_WRITE | PROT_EXEC, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0)) && (pages[i])) ?
             memset(pages[i], 0, p_size) && madvise(pages[i], p_size, MADV_MERGEABLE) : exit(-1);
    }

    p_s();
    print("modifying pages contents");

    //Then I modify the pages to some "random" value
    for (i = 0; i < N_PAGES; i++){memset(pages[i], i+1337, 1);};
    p_s();
}

So, we create 3 PRIVATE pages, we fill them with zeroes, and
then we mark them as candidates to be merged.

After that, we give some time for the KSM scan to happen, then KSM merges
the pages, then the program modifies the pages content with some 'random values'
and we wait for the scan to happen and finally the KSM process un-merge the pages.

After compiling, the resulting stdout is:

$ ./a.out
Sharing pages: 3  
modifying page contents  
Sharing pages: 0  

You can also review some other runtime KSM parameters by running:

$ sudo grep . /sys/kernel/mm/ksm/pages_*
/sys/kernel/mm/ksm/pages_shared:0
/sys/kernel/mm/ksm/pages_sharing:0
/sys/kernel/mm/ksm/pages_to_scan:100
/sys/kernel/mm/ksm/pages_unshared:0
/sys/kernel/mm/ksm/pages_volatile:0

At this point I guess that you are asking yourself if this can be applied
to any memory page even without being advised by the program itself.

The answer is yes, but using UKSM, unfortunately
those patches are not merged on the mainstream kernel yet.

In conclusion this is a good feature offered by the kernel
for optimizing memory consumption between processes that share the same data
, its main drawback is that this is also an expensive process in terms of CPU usage.

In a next article, I'll be covering how I played a bit with these
primitives but using Golang.

References:

• Madvise man page
• Mmap man page
• KSM wikipedia