New attempt after the steps changed (info mtree -f in L1 qemu-monitor)

[HanzhongPeng]

System setup
L0 ubuntu 2204
L1 debian-12-nocloud-amd64-daily-20240827-1852.qcow2
L2 debian-12-nocloud-amd64-daily-20240827-1852.qcow2

L0 start L1 (qemu 8.0.0)
qemu/build/qemu-system-x86_64 -machine q35 -accel kvm -m 2G
-cpu host,-pku,-xsaves,-kvmclock,-kvm-pv-unhalt
-netdev user,id=u1,hostfwd=tcp::2222-:22
-device virtio-net,netdev=u1 -smp 1 -serial stdio
-hda debian-12-nocloud-amd64-daily-20240827-1852.qcow2
-monitor telnet:127.0.0.1:55556,server,nowait

L1 start L2(qemu 6.1.0-hw/nvme/ctrl.c:6146 timer_mod(sq->timer, qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL));)
qemu-6.1.0/build/qemu-system-x86_64
-machine type=q35,accel=kvm -cpu host -m 1G
-hda debian-12-nocloud-amd64-daily-20240827-1852.qcow2
-device nvme,drive=disk0,serial=1234,cmb_size_mb=64
-drive file=null-co://,if=none,format=raw,id=disk0
-device ich9-intel-hda
-netdev user,id=net0,hostfwd=tcp::2222-:22
-device e1000,netdev=net0
-nographic
-monitor telnet:127.0.0.1:55556,server,nowait

[L2] lspci -v
[L1] telnet localhost 55556
[L1 qemu-monitor] info mtree -f
[L2] snap
[L0 qemu-monitor] dump-guest-memory /home/iie/Desktop/phz/tools/snapshots/0103/mem
[L0 qemu-monitor] info registers
[L0] sudo dmesg | grep "VMCS.*last" | cut -f2 -d"(" | cut -f1 -d ")"

• in hyperpill after chaging 4 lines
  #define nvme_mmio_region 0xf4070000 // FIXME:
  #define leak_buf 0x3023000
  #define cqes 0x3024000
  #define cmds 0x3025000

CC=clang CXX=clang++ make
CC=clang CXX=clang++ make tests

KVM=1 MANUAL_RANGES=$SNAPSHOT_BASE/mtree RANGE_REGEX="nvme" $PROJECT_ROOT/tests/run_hyperpill.sh cve-2021-3947

[1735903978] NEW_PC: 7f86278c52f0 ()
[1735903978] NEW_PC: 7f86278c52fd ()
[1735903978] NEW_PC: 7f862782845e ()
[1735903978] NEW_PC: 7f86278284c8 ()
[1735903978] NEW_PC: 7f86278284a7 ()
[1735903978] NEW_PC: 7f86278b767f ()
[1735903978] NEW_PC: 55cb047625b5 ()
[1735903978] NEW_PC: 7f8627324e40 ()
[1735903978] NEW_PC: 7f8627324e80 ()
PIO READ ADDR: 3f9
PIO READ ADDR: 3fd
^C
Program received signal SIGINT, Interrupt.
fuzz_instr_before_execution (i=0x555555bcdd50 <iC+2723664>) at main.cc:170
170 }
(gdb) bt
#0 fuzz_instr_before_execution (i=0x555555bcdd50 <iC+2723664>) at main.cc:170
#1 0x00005555556488cb in BX_CPU_C::cpu_loop () at ../../bochs/cpu/cpu.cc:129
#2 0x00005555555f4415 in start_cpu () at main.cc:71
#3 0x00005555555f40d3 in test_mmio_write (addr=4273930260, size=2, data=0)
at tests/tests.h:13
#4 main (argc=1, argv=0x7fffffffe028) at tests/cve-2021-3947.cc:96
(gdb)

And it stuck on the last three step of mmio write,whole result can be downloaded from google drive
The snapshot and the result in the google drive zip
https://drive.google.com/file/d/1yu1ePNpii48zUFNpK_mAZ9WYUFT4QH5D/view?usp=sharing

[cyruscyliu]

probably due xtest that we found is not emulated by Bochs

since xtest is enabled by two CPU features: hle and rtm, you can disable xtest by removing these CPU features when launching L1 VM.

qemu/build/qemu-system-x86_64 -machine q35 -accel kvm -m 2G \
    -cpu host,-pku,-xsaves,-kvmclock,-kvm-pv-unhalt,-hle,-rtm \
    -netdev user,id=u1,hostfwd=tcp::2222-:22 \
    -device virtio-net,netdev=u1 -smp 1 -serial stdio \
    -hda debian-12-nocloud-amd64-daily-20240827-1852.qcow2 \
    -monitor telnet:127.0.0.1:55556,server,nowait

[cyruscyliu]

to verify, run lscpu and check if the flags include hle, rtm or both

[cyruscyliu]

As it turns out, leak_buf/cqes/cmds should be page aligned.