TWxSoftware/twx-linux
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
28ab173e96
twx-linux/kernel
History
Alexei Starovoitov 1c2a088a66 bpf: x64: add JIT support for multi-function programs 
Typical JIT does several passes over bpf instructions to
compute total size and relative offsets of jumps and calls.
With multitple bpf functions calling each other all relative calls
will have invalid offsets intially therefore we need to additional
last pass over the program to emit calls with correct offsets.
For example in case of three bpf functions:
main:
  call foo
  call bpf_map_lookup
  exit
foo:
  call bar
  exit
bar:
  exit

We will call bpf_int_jit_compile() indepedently for main(), foo() and bar()
x64 JIT typically does 4-5 passes to converge.
After these initial passes the image for these 3 functions
will be good except call targets, since start addresses of
foo() and bar() are unknown when we were JITing main()
(note that call bpf_map_lookup will be resolved properly
during initial passes).
Once start addresses of 3 functions are known we patch
call_insn->imm to point to right functions and call
bpf_int_jit_compile() again which needs only one pass.
Additional safety checks are done to make sure this
last pass doesn't produce image that is larger or smaller
than previous pass.

When constant blinding is on it's applied to all functions
at the first pass, since doing it once again at the last
pass can change size of the JITed code.

Tested on x64 and arm64 hw with JIT on/off, blinding on/off.
x64 jits bpf-to-bpf calls correctly while arm64 falls back to interpreter.
All other JITs that support normal BPF_CALL will behave the same way
since bpf-to-bpf call is equivalent to bpf-to-kernel call from
JITs point of view.

Signed-off-by: Alexei Starovoitov <ast@kernel.org>
Acked-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
2017-12-17 20:34:36 +01:00
..
bpf bpf: x64: add JIT support for multi-function programs 2017-12-17 20:34:36 +01:00
cgroup Merge branch 'for-4.15' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup 2017-11-15 14:29:44 -08:00
configs
debug
events bpf/tracing: fix kernel/events/core.c compilation error 2017-12-13 22:44:10 +01:00
gcov License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
irq Merge branch 'irq-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2017-11-26 14:39:20 -08:00
livepatch Merge branch 'for-linus' of ssh://gitolite.kernel.org/pub/scm/linux/kernel/git/jikos/livepatching 2017-11-15 10:21:58 -08:00
locking kmemcheck: remove annotations 2017-11-15 18:21:04 -08:00
power mm: remove __GFP_COLD 2017-11-15 18:21:06 -08:00
printk Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/pmladek/printk 2017-11-21 05:28:13 -10:00
rcu Merge branch 'timers-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2017-11-13 17:56:58 -08:00
sched Merge branch 'misc.compat' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2017-11-17 11:54:55 -08:00
time - final batch of "non trivial" timer conversions (multi-tree dependencies, 2017-11-23 16:29:05 +01:00
trace bpf/tracing: fix kernel/events/core.c compilation error 2017-12-13 22:44:10 +01:00
.gitignore
acct.c Merge branch 'linus' into locking/core, to resolve conflicts 2017-11-07 10:32:44 +01:00
async.c
audit_fsnotify.c
audit_tree.c Merge branch 'fsnotify' of git://git.kernel.org/pub/scm/linux/kernel/git/jack/linux-fs 2017-11-14 14:08:20 -08:00
audit_watch.c
audit.c Audit: remove unused audit_log_secctx function 2017-11-10 16:08:47 -05:00
audit.h audit/stable-4.15 PR 20171113 2017-11-15 13:28:48 -08:00
auditfilter.c audit: filter PATH records keyed on filesystem magic 2017-11-10 16:08:56 -05:00
auditsc.c audit/stable-4.15 PR 20171113 2017-11-15 13:28:48 -08:00
backtracetest.c
bounds.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
capability.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
compat.c sched_rr_get_interval(): move compat to native, get rid of set_fs() 2017-09-20 00:30:57 -04:00
configs.c
context_tracking.c
cpu_pm.c
cpu.c cpu/hotplug: Reset node state after operation 2017-10-21 16:11:30 +02:00
crash_core.c kdump: print a message in case parse_crashkernel_mem resulted in zero bytes 2017-11-17 16:10:03 -08:00
crash_dump.c
cred.c
delayacct.c
dma.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
elfcore.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
exec_domain.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
exit.c locking/atomics: COCCINELLE/treewide: Convert trivial ACCESS_ONCE() patterns to READ_ONCE()/WRITE_ONCE() 2017-10-25 11:01:08 +02:00
extable.c kprobes, x86/alternatives: Use text_mutex to protect smp_alt_modules 2017-11-07 12:20:09 +01:00
fork.c pid: remove pidhash 2017-11-17 16:10:04 -08:00
freezer.c
futex_compat.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
futex.c Merge branch 'linus' into core/urgent, to pick up dependent commits 2017-11-04 08:53:04 +01:00
groups.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
hung_task.c
irq_work.c Merge branch 'irq-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip 2017-11-13 17:33:11 -08:00
jump_label.c jump_label: Invoke jump_label_test() via early_initcall() 2017-11-14 08:41:41 +01:00
kallsyms.c kallsyms: take advantage of the new '%px' format 2017-11-29 10:30:13 -08:00
kcmp.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
Kconfig.freezer
Kconfig.hz
Kconfig.locks
Kconfig.preempt
kcov.c kcov: support comparison operands collection 2017-11-17 16:10:04 -08:00
kexec_core.c
kexec_file.c resource: Provide resource struct in resource walk callback 2017-11-07 15:35:57 +01:00
kexec_internal.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
kexec.c
kmod.c kmod: move #ifdef CONFIG_MODULES wrapper to Makefile 2017-09-08 18:26:51 -07:00
kprobes.c add infrastructure for tagging functions as error injectable 2017-12-12 08:56:26 -08:00
ksysfs.c
kthread.c treewide: Remove TIMER_FUNC_TYPE and TIMER_DATA_TYPE casts 2017-11-21 16:35:54 -08:00
latencytop.c
Makefile License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
memremap.c memremap: add scheduling point to devm_memremap_pages 2017-10-03 17:54:25 -07:00
module_signing.c
module-internal.h
module.c add infrastructure for tagging functions as error injectable 2017-12-12 08:56:26 -08:00
notifier.c
nsproxy.c
padata.c treewide: setup_timer() -> timer_setup() 2017-11-21 15:57:07 -08:00
panic.c kernel/panic.c: add TAINT_AUX 2017-11-17 16:10:04 -08:00
params.c kernel/params.c: improve STANDARD_PARAM_DEF readability 2017-10-03 17:54:26 -07:00
pid_namespace.c pid: remove pidhash 2017-11-17 16:10:04 -08:00
pid.c pid: remove pidhash 2017-11-17 16:10:04 -08:00
profile.c
ptrace.c
range.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
reboot.c kernel/reboot.c: add devm_register_reboot_notifier() 2017-11-17 16:10:04 -08:00
relay.c
resource.c x86/mm, resource: Use PAGE_KERNEL protection for ioremap of memory pages 2017-11-07 15:35:58 +01:00
seccomp.c Merge branch 'linus' into locking/core, to resolve conflicts 2017-11-07 10:32:44 +01:00
signal.c Merge branch 'akpm' (patches from Andrew) 2017-11-17 16:56:17 -08:00
smp.c smp/core: Use lockdep to assert IRQs are disabled/enabled 2017-11-08 11:13:50 +01:00
smpboot.c watchdog/core, powerpc: Lock cpus across reconfiguration 2017-10-04 10:53:54 +02:00
smpboot.h License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
softirq.c kmemcheck: rip it out 2017-11-15 18:21:05 -08:00
stacktrace.c
stop_machine.c
sys_ni.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
sys.c arm64 updates for 4.15 2017-11-15 10:56:56 -08:00
sysctl_binary.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
sysctl.c kernel/sysctl.c: code cleanups 2017-11-17 16:10:03 -08:00
task_work.c Merge branch 'linus' into locking/core, to resolve conflicts 2017-11-07 10:32:44 +01:00
taskstats.c
test_kprobes.c kprobes: Disable the jprobes test code 2017-10-20 11:02:54 +02:00
torture.c
tracepoint.c
tsacct.c
ucount.c
uid16.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
umh.c kernel/umh.c: optimize 'proc_cap_handler()' 2017-11-17 16:10:01 -08:00
up.c
user_namespace.c Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace 2017-11-16 12:20:15 -08:00
user-return-notifier.c
user.c userns: use union in {g,u}idmap struct 2017-10-31 17:22:58 -05:00
utsname_sysctl.c
utsname.c
watchdog_hld.c Merge branch 'linus' into core/urgent, to pick up dependent commits 2017-11-04 08:53:04 +01:00
watchdog.c Merge branch 'linus' into sched/core, to pick up fixes 2017-11-08 10:17:15 +01:00
workqueue_internal.h Merge branch 'for-4.14-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/wq 2017-11-06 12:26:49 -08:00
workqueue.c treewide: Remove TIMER_FUNC_TYPE and TIMER_DATA_TYPE casts 2017-11-21 16:35:54 -08:00