Based on kernel version 3.9. Page generated on 2013-05-02 23:16 EST.
1 The PPC KVM paravirtual interface 2 ================================= 3 4 The basic execution principle by which KVM on PowerPC works is to run all kernel 5 space code in PR=1 which is user space. This way we trap all privileged 6 instructions and can emulate them accordingly. 7 8 Unfortunately that is also the downfall. There are quite some privileged 9 instructions that needlessly return us to the hypervisor even though they 10 could be handled differently. 11 12 This is what the PPC PV interface helps with. It takes privileged instructions 13 and transforms them into unprivileged ones with some help from the hypervisor. 14 This cuts down virtualization costs by about 50% on some of my benchmarks. 15 16 The code for that interface can be found in arch/powerpc/kernel/kvm* 17 18 Querying for existence 19 ====================== 20 21 To find out if we're running on KVM or not, we leverage the device tree. When 22 Linux is running on KVM, a node /hypervisor exists. That node contains a 23 compatible property with the value "linux,kvm". 24 25 Once you determined you're running under a PV capable KVM, you can now use 26 hypercalls as described below. 27 28 KVM hypercalls 29 ============== 30 31 Inside the device tree's /hypervisor node there's a property called 32 'hypercall-instructions'. This property contains at most 4 opcodes that make 33 up the hypercall. To call a hypercall, just call these instructions. 34 35 The parameters are as follows: 36 37 Register IN OUT 38 39 r0 - volatile 40 r3 1st parameter Return code 41 r4 2nd parameter 1st output value 42 r5 3rd parameter 2nd output value 43 r6 4th parameter 3rd output value 44 r7 5th parameter 4th output value 45 r8 6th parameter 5th output value 46 r9 7th parameter 6th output value 47 r10 8th parameter 7th output value 48 r11 hypercall number 8th output value 49 r12 - volatile 50 51 Hypercall definitions are shared in generic code, so the same hypercall numbers 52 apply for x86 and powerpc alike with the exception that each KVM hypercall 53 also needs to be ORed with the KVM vendor code which is (42 << 16). 54 55 Return codes can be as follows: 56 57 Code Meaning 58 59 0 Success 60 12 Hypercall not implemented 61 <0 Error 62 63 The magic page 64 ============== 65 66 To enable communication between the hypervisor and guest there is a new shared 67 page that contains parts of supervisor visible register state. The guest can 68 map this shared page using the KVM hypercall KVM_HC_PPC_MAP_MAGIC_PAGE. 69 70 With this hypercall issued the guest always gets the magic page mapped at the 71 desired location. The first parameter indicates the effective address when the 72 MMU is enabled. The second parameter indicates the address in real mode, if 73 applicable to the target. For now, we always map the page to -4096. This way we 74 can access it using absolute load and store functions. The following 75 instruction reads the first field of the magic page: 76 77 ld rX, -4096(0) 78 79 The interface is designed to be extensible should there be need later to add 80 additional registers to the magic page. If you add fields to the magic page, 81 also define a new hypercall feature to indicate that the host can give you more 82 registers. Only if the host supports the additional features, make use of them. 83 84 The magic page layout is described by struct kvm_vcpu_arch_shared 85 in arch/powerpc/include/asm/kvm_para.h. 86 87 Magic page features 88 =================== 89 90 When mapping the magic page using the KVM hypercall KVM_HC_PPC_MAP_MAGIC_PAGE, 91 a second return value is passed to the guest. This second return value contains 92 a bitmap of available features inside the magic page. 93 94 The following enhancements to the magic page are currently available: 95 96 KVM_MAGIC_FEAT_SR Maps SR registers r/w in the magic page 97 98 For enhanced features in the magic page, please check for the existence of the 99 feature before using them! 100 101 MSR bits 102 ======== 103 104 The MSR contains bits that require hypervisor intervention and bits that do 105 not require direct hypervisor intervention because they only get interpreted 106 when entering the guest or don't have any impact on the hypervisor's behavior. 107 108 The following bits are safe to be set inside the guest: 109 110 MSR_EE 111 MSR_RI 112 113 If any other bit changes in the MSR, please still use mtmsr(d). 114 115 Patched instructions 116 ==================== 117 118 The "ld" and "std" instructions are transormed to "lwz" and "stw" instructions 119 respectively on 32 bit systems with an added offset of 4 to accommodate for big 120 endianness. 121 122 The following is a list of mapping the Linux kernel performs when running as 123 guest. Implementing any of those mappings is optional, as the instruction traps 124 also act on the shared page. So calling privileged instructions still works as 125 before. 126 127 From To 128 ==== == 129 130 mfmsr rX ld rX, magic_page->msr 131 mfsprg rX, 0 ld rX, magic_page->sprg0 132 mfsprg rX, 1 ld rX, magic_page->sprg1 133 mfsprg rX, 2 ld rX, magic_page->sprg2 134 mfsprg rX, 3 ld rX, magic_page->sprg3 135 mfsrr0 rX ld rX, magic_page->srr0 136 mfsrr1 rX ld rX, magic_page->srr1 137 mfdar rX ld rX, magic_page->dar 138 mfdsisr rX lwz rX, magic_page->dsisr 139 140 mtmsr rX std rX, magic_page->msr 141 mtsprg 0, rX std rX, magic_page->sprg0 142 mtsprg 1, rX std rX, magic_page->sprg1 143 mtsprg 2, rX std rX, magic_page->sprg2 144 mtsprg 3, rX std rX, magic_page->sprg3 145 mtsrr0 rX std rX, magic_page->srr0 146 mtsrr1 rX std rX, magic_page->srr1 147 mtdar rX std rX, magic_page->dar 148 mtdsisr rX stw rX, magic_page->dsisr 149 150 tlbsync nop 151 152 mtmsrd rX, 0 b <special mtmsr section> 153 mtmsr rX b <special mtmsr section> 154 155 mtmsrd rX, 1 b <special mtmsrd section> 156 157 [Book3S only] 158 mtsrin rX, rY b <special mtsrin section> 159 160 [BookE only] 161 wrteei [0|1] b <special wrteei section> 162 163 164 Some instructions require more logic to determine what's going on than a load 165 or store instruction can deliver. To enable patching of those, we keep some 166 RAM around where we can live translate instructions to. What happens is the 167 following: 168 169 1) copy emulation code to memory 170 2) patch that code to fit the emulated instruction 171 3) patch that code to return to the original pc + 4 172 4) patch the original instruction to branch to the new code 173 174 That way we can inject an arbitrary amount of code as replacement for a single 175 instruction. This allows us to check for pending interrupts when setting EE=1 176 for example. 177 178 Hypercall ABIs in KVM on PowerPC 179 ================================= 180 1) KVM hypercalls (ePAPR) 181 182 These are ePAPR compliant hypercall implementation (mentioned above). Even 183 generic hypercalls are implemented here, like the ePAPR idle hcall. These are 184 available on all targets. 185 186 2) PAPR hypercalls 187 188 PAPR hypercalls are needed to run server PowerPC PAPR guests (-M pseries in QEMU). 189 These are the same hypercalls that pHyp, the POWER hypervisor implements. Some of 190 them are handled in the kernel, some are handled in user space. This is only 191 available on book3s_64. 192 193 3) OSI hypercalls 194 195 Mac-on-Linux is another user of KVM on PowerPC, which has its own hypercall (long 196 before KVM). This is supported to maintain compatibility. All these hypercalls get 197 forwarded to user space. This is only useful on book3s_32, but can be used with 198 book3s_64 as well.