TWxSoftware/twx-linux
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

net: use napi_skb_cache even in process context

Browse Source 
This is a followup of commit e20dfbad8a ("net: fix napi_consume_skb()
with alien skbs").

Now the per-cpu napi_skb_cache is populated from TX completion path,
we can make use of this cache, especially for cpus not used
from a driver NAPI poll (primary user of napi_cache).

We can use the napi_skb_cache only if current context is not from hard irq.

With this patch, I consistently reach 130 Mpps on my UDP tx stress test
and reduce SLUB spinlock contention to smaller values.

Note there is still some SLUB contention for skb->head allocations.

I had to tune /sys/kernel/slab/skbuff_small_head/cpu_partial
and /sys/kernel/slab/skbuff_small_head/min_partial depending
on the platform taxonomy.

Signed-off-by: Eric Dumazet <edumazet@google.com>
Reviewed-by: Jason Xing <kerneljasonxing@gmail.com>
Tested-by: Jason Xing <kerneljasonxing@gmail.com>
Reviewed-by: Kuniyuki Iwashima <kuniyu@google.com>
Link: https://patch.msgid.link/20251116202717.1542829-4-edumazet@google.com
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
This commit is contained in:
Eric Dumazet
2025-11-16 20:27:17 +00:00
committed by Jakub Kicinski
parent 294e638259
commit 21664814b8
1 changed files with 5 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
net/core/skbuff.c
+5
View File
@@ -666,7 +666,12 @@ struct sk_buff *__alloc_skb(unsigned int size, gfp_t gfp_mask,
skb = napi_skb_cache_get(true);
if (unlikely(!skb))
return NULL;
} else if (!in_hardirq() && !irqs_disabled()) {
local_bh_disable();
skb = napi_skb_cache_get(false);
local_bh_enable();
}
if (!skb) {
fallback:
skb = kmem_cache_alloc_node(cache, gfp_mask & ~GFP_DMA, node);