• Kirill Tkhai's avatar
    mm: make counting of list_lru_one::nr_items lockless · 0c7c1bed
    Kirill Tkhai authored
    During the reclaiming slab of a memcg, shrink_slab iterates over all
    registered shrinkers in the system, and tries to count and consume
    objects related to the cgroup.  In case of memory pressure, this behaves
    bad: I observe high system time and time spent in list_lru_count_one()
    for many processes on RHEL7 kernel.
    
    This patch makes list_lru_node::memcg_lrus rcu protected, that allows to
    skip taking spinlock in list_lru_count_one().
    
    Shakeel Butt with the patch observes significant perf graph change.  He
    says:
    
    ========================================================================
    Setup: running a fork-bomb in a memcg of 200MiB on a 8GiB and 4 vcpu
    VM and recording the trace with 'perf record -g -a'.
    
    The trace without the patch:
    
    +  34.19%     fb.sh  [kernel.kallsyms]  [k] queued_spin_lock_slowpath
    +  30.77%     fb.sh  [kernel.kallsyms]  [k] _raw_spin_lock
    +   3.53%     fb.sh  [kernel.kallsyms]  [k] list_lru_count_one
    +   2.26%     fb.sh  [kernel.kall...
    0c7c1bed
list_lru.c 13.4 KB