1*f77c0515SJanosch Frank /* 2*f77c0515SJanosch Frank * s390x smp 3*f77c0515SJanosch Frank * Based on Linux's arch/s390/kernel/smp.c and 4*f77c0515SJanosch Frank * arch/s390/include/asm/sigp.h 5*f77c0515SJanosch Frank * 6*f77c0515SJanosch Frank * Copyright (c) 2019 IBM Corp 7*f77c0515SJanosch Frank * 8*f77c0515SJanosch Frank * Authors: 9*f77c0515SJanosch Frank * Janosch Frank <frankja@linux.ibm.com> 10*f77c0515SJanosch Frank * 11*f77c0515SJanosch Frank * This code is free software; you can redistribute it and/or modify it 12*f77c0515SJanosch Frank * under the terms of the GNU General Public License version 2. 13*f77c0515SJanosch Frank */ 14*f77c0515SJanosch Frank #include <libcflat.h> 15*f77c0515SJanosch Frank #include <asm/arch_def.h> 16*f77c0515SJanosch Frank #include <asm/sigp.h> 17*f77c0515SJanosch Frank #include <asm/page.h> 18*f77c0515SJanosch Frank #include <asm/barrier.h> 19*f77c0515SJanosch Frank #include <asm/spinlock.h> 20*f77c0515SJanosch Frank #include <asm/asm-offsets.h> 21*f77c0515SJanosch Frank 22*f77c0515SJanosch Frank #include <alloc.h> 23*f77c0515SJanosch Frank #include <alloc_page.h> 24*f77c0515SJanosch Frank 25*f77c0515SJanosch Frank #include "smp.h" 26*f77c0515SJanosch Frank #include "sclp.h" 27*f77c0515SJanosch Frank 28*f77c0515SJanosch Frank static char cpu_info_buffer[PAGE_SIZE] __attribute__((__aligned__(4096))); 29*f77c0515SJanosch Frank static struct cpu *cpus; 30*f77c0515SJanosch Frank static struct cpu *cpu0; 31*f77c0515SJanosch Frank static struct spinlock lock; 32*f77c0515SJanosch Frank 33*f77c0515SJanosch Frank extern void smp_cpu_setup_state(void); 34*f77c0515SJanosch Frank 35*f77c0515SJanosch Frank int smp_query_num_cpus(void) 36*f77c0515SJanosch Frank { 37*f77c0515SJanosch Frank struct ReadCpuInfo *info = (void *)cpu_info_buffer; 38*f77c0515SJanosch Frank return info->nr_configured; 39*f77c0515SJanosch Frank } 40*f77c0515SJanosch Frank 41*f77c0515SJanosch Frank struct cpu *smp_cpu_from_addr(uint16_t addr) 42*f77c0515SJanosch Frank { 43*f77c0515SJanosch Frank int i, num = smp_query_num_cpus(); 44*f77c0515SJanosch Frank 45*f77c0515SJanosch Frank for (i = 0; i < num; i++) { 46*f77c0515SJanosch Frank if (cpus[i].addr == addr) 47*f77c0515SJanosch Frank return &cpus[i]; 48*f77c0515SJanosch Frank } 49*f77c0515SJanosch Frank return NULL; 50*f77c0515SJanosch Frank } 51*f77c0515SJanosch Frank 52*f77c0515SJanosch Frank bool smp_cpu_stopped(uint16_t addr) 53*f77c0515SJanosch Frank { 54*f77c0515SJanosch Frank uint32_t status; 55*f77c0515SJanosch Frank 56*f77c0515SJanosch Frank if (sigp(addr, SIGP_SENSE, 0, &status) != SIGP_CC_STATUS_STORED) 57*f77c0515SJanosch Frank return false; 58*f77c0515SJanosch Frank return !!(status & (SIGP_STATUS_CHECK_STOP|SIGP_STATUS_STOPPED)); 59*f77c0515SJanosch Frank } 60*f77c0515SJanosch Frank 61*f77c0515SJanosch Frank bool smp_cpu_running(uint16_t addr) 62*f77c0515SJanosch Frank { 63*f77c0515SJanosch Frank if (sigp(addr, SIGP_SENSE_RUNNING, 0, NULL) != SIGP_CC_STATUS_STORED) 64*f77c0515SJanosch Frank return true; 65*f77c0515SJanosch Frank /* Status stored condition code is equivalent to cpu not running. */ 66*f77c0515SJanosch Frank return false; 67*f77c0515SJanosch Frank } 68*f77c0515SJanosch Frank 69*f77c0515SJanosch Frank static int smp_cpu_stop_nolock(uint16_t addr, bool store) 70*f77c0515SJanosch Frank { 71*f77c0515SJanosch Frank struct cpu *cpu; 72*f77c0515SJanosch Frank uint8_t order = store ? SIGP_STOP_AND_STORE_STATUS : SIGP_STOP; 73*f77c0515SJanosch Frank 74*f77c0515SJanosch Frank cpu = smp_cpu_from_addr(addr); 75*f77c0515SJanosch Frank if (!cpu || cpu == cpu0) 76*f77c0515SJanosch Frank return -1; 77*f77c0515SJanosch Frank 78*f77c0515SJanosch Frank if (sigp_retry(addr, order, 0, NULL)) 79*f77c0515SJanosch Frank return -1; 80*f77c0515SJanosch Frank 81*f77c0515SJanosch Frank while (!smp_cpu_stopped(addr)) 82*f77c0515SJanosch Frank mb(); 83*f77c0515SJanosch Frank cpu->active = false; 84*f77c0515SJanosch Frank return 0; 85*f77c0515SJanosch Frank } 86*f77c0515SJanosch Frank 87*f77c0515SJanosch Frank int smp_cpu_stop(uint16_t addr) 88*f77c0515SJanosch Frank { 89*f77c0515SJanosch Frank int rc; 90*f77c0515SJanosch Frank 91*f77c0515SJanosch Frank spin_lock(&lock); 92*f77c0515SJanosch Frank rc = smp_cpu_stop_nolock(addr, false); 93*f77c0515SJanosch Frank spin_unlock(&lock); 94*f77c0515SJanosch Frank return rc; 95*f77c0515SJanosch Frank } 96*f77c0515SJanosch Frank 97*f77c0515SJanosch Frank int smp_cpu_stop_store_status(uint16_t addr) 98*f77c0515SJanosch Frank { 99*f77c0515SJanosch Frank int rc; 100*f77c0515SJanosch Frank 101*f77c0515SJanosch Frank spin_lock(&lock); 102*f77c0515SJanosch Frank rc = smp_cpu_stop_nolock(addr, true); 103*f77c0515SJanosch Frank spin_unlock(&lock); 104*f77c0515SJanosch Frank return rc; 105*f77c0515SJanosch Frank } 106*f77c0515SJanosch Frank 107*f77c0515SJanosch Frank int smp_cpu_restart(uint16_t addr) 108*f77c0515SJanosch Frank { 109*f77c0515SJanosch Frank int rc = -1; 110*f77c0515SJanosch Frank struct cpu *cpu; 111*f77c0515SJanosch Frank 112*f77c0515SJanosch Frank spin_lock(&lock); 113*f77c0515SJanosch Frank cpu = smp_cpu_from_addr(addr); 114*f77c0515SJanosch Frank if (cpu) { 115*f77c0515SJanosch Frank rc = sigp(addr, SIGP_RESTART, 0, NULL); 116*f77c0515SJanosch Frank cpu->active = true; 117*f77c0515SJanosch Frank } 118*f77c0515SJanosch Frank spin_unlock(&lock); 119*f77c0515SJanosch Frank return rc; 120*f77c0515SJanosch Frank } 121*f77c0515SJanosch Frank 122*f77c0515SJanosch Frank int smp_cpu_start(uint16_t addr, struct psw psw) 123*f77c0515SJanosch Frank { 124*f77c0515SJanosch Frank int rc = -1; 125*f77c0515SJanosch Frank struct cpu *cpu; 126*f77c0515SJanosch Frank struct lowcore *lc; 127*f77c0515SJanosch Frank 128*f77c0515SJanosch Frank spin_lock(&lock); 129*f77c0515SJanosch Frank cpu = smp_cpu_from_addr(addr); 130*f77c0515SJanosch Frank if (cpu) { 131*f77c0515SJanosch Frank lc = cpu->lowcore; 132*f77c0515SJanosch Frank lc->restart_new_psw.mask = psw.mask; 133*f77c0515SJanosch Frank lc->restart_new_psw.addr = psw.addr; 134*f77c0515SJanosch Frank rc = sigp(addr, SIGP_RESTART, 0, NULL); 135*f77c0515SJanosch Frank } 136*f77c0515SJanosch Frank spin_unlock(&lock); 137*f77c0515SJanosch Frank return rc; 138*f77c0515SJanosch Frank } 139*f77c0515SJanosch Frank 140*f77c0515SJanosch Frank int smp_cpu_destroy(uint16_t addr) 141*f77c0515SJanosch Frank { 142*f77c0515SJanosch Frank struct cpu *cpu; 143*f77c0515SJanosch Frank int rc; 144*f77c0515SJanosch Frank 145*f77c0515SJanosch Frank spin_lock(&lock); 146*f77c0515SJanosch Frank rc = smp_cpu_stop_nolock(addr, false); 147*f77c0515SJanosch Frank if (!rc) { 148*f77c0515SJanosch Frank cpu = smp_cpu_from_addr(addr); 149*f77c0515SJanosch Frank free_pages(cpu->lowcore, 2 * PAGE_SIZE); 150*f77c0515SJanosch Frank free_pages(cpu->stack, 4 * PAGE_SIZE); 151*f77c0515SJanosch Frank cpu->lowcore = (void *)-1UL; 152*f77c0515SJanosch Frank cpu->stack = (void *)-1UL; 153*f77c0515SJanosch Frank } 154*f77c0515SJanosch Frank spin_unlock(&lock); 155*f77c0515SJanosch Frank return rc; 156*f77c0515SJanosch Frank } 157*f77c0515SJanosch Frank 158*f77c0515SJanosch Frank int smp_cpu_setup(uint16_t addr, struct psw psw) 159*f77c0515SJanosch Frank { 160*f77c0515SJanosch Frank struct lowcore *lc; 161*f77c0515SJanosch Frank struct cpu *cpu; 162*f77c0515SJanosch Frank int rc = -1; 163*f77c0515SJanosch Frank 164*f77c0515SJanosch Frank spin_lock(&lock); 165*f77c0515SJanosch Frank 166*f77c0515SJanosch Frank if (!cpus) 167*f77c0515SJanosch Frank goto out; 168*f77c0515SJanosch Frank 169*f77c0515SJanosch Frank cpu = smp_cpu_from_addr(addr); 170*f77c0515SJanosch Frank 171*f77c0515SJanosch Frank if (!cpu || cpu->active) 172*f77c0515SJanosch Frank goto out; 173*f77c0515SJanosch Frank 174*f77c0515SJanosch Frank sigp_retry(cpu->addr, SIGP_INITIAL_CPU_RESET, 0, NULL); 175*f77c0515SJanosch Frank 176*f77c0515SJanosch Frank lc = alloc_pages(1); 177*f77c0515SJanosch Frank cpu->lowcore = lc; 178*f77c0515SJanosch Frank memset(lc, 0, PAGE_SIZE * 2); 179*f77c0515SJanosch Frank sigp_retry(cpu->addr, SIGP_SET_PREFIX, (unsigned long )lc, NULL); 180*f77c0515SJanosch Frank 181*f77c0515SJanosch Frank /* Copy all exception psws. */ 182*f77c0515SJanosch Frank memcpy(lc, cpu0->lowcore, 512); 183*f77c0515SJanosch Frank 184*f77c0515SJanosch Frank /* Setup stack */ 185*f77c0515SJanosch Frank cpu->stack = (uint64_t *)alloc_pages(2); 186*f77c0515SJanosch Frank 187*f77c0515SJanosch Frank /* Start without DAT and any other mask bits. */ 188*f77c0515SJanosch Frank cpu->lowcore->sw_int_grs[14] = psw.addr; 189*f77c0515SJanosch Frank cpu->lowcore->sw_int_grs[15] = (uint64_t)cpu->stack + (PAGE_SIZE * 4); 190*f77c0515SJanosch Frank lc->restart_new_psw.mask = 0x0000000180000000UL; 191*f77c0515SJanosch Frank lc->restart_new_psw.addr = (uint64_t)smp_cpu_setup_state; 192*f77c0515SJanosch Frank lc->sw_int_cr0 = 0x0000000000040000UL; 193*f77c0515SJanosch Frank 194*f77c0515SJanosch Frank /* Start processing */ 195*f77c0515SJanosch Frank rc = sigp_retry(cpu->addr, SIGP_RESTART, 0, NULL); 196*f77c0515SJanosch Frank if (!rc) 197*f77c0515SJanosch Frank cpu->active = true; 198*f77c0515SJanosch Frank 199*f77c0515SJanosch Frank out: 200*f77c0515SJanosch Frank spin_unlock(&lock); 201*f77c0515SJanosch Frank return rc; 202*f77c0515SJanosch Frank } 203*f77c0515SJanosch Frank 204*f77c0515SJanosch Frank /* 205*f77c0515SJanosch Frank * Disregarding state, stop all cpus that once were online except for 206*f77c0515SJanosch Frank * calling cpu. 207*f77c0515SJanosch Frank */ 208*f77c0515SJanosch Frank void smp_teardown(void) 209*f77c0515SJanosch Frank { 210*f77c0515SJanosch Frank int i = 0; 211*f77c0515SJanosch Frank uint16_t this_cpu = stap(); 212*f77c0515SJanosch Frank struct ReadCpuInfo *info = (void *)cpu_info_buffer; 213*f77c0515SJanosch Frank 214*f77c0515SJanosch Frank spin_lock(&lock); 215*f77c0515SJanosch Frank for (; i < info->nr_configured; i++) { 216*f77c0515SJanosch Frank if (cpus[i].active && 217*f77c0515SJanosch Frank cpus[i].addr != this_cpu) { 218*f77c0515SJanosch Frank sigp_retry(cpus[i].addr, SIGP_STOP, 0, NULL); 219*f77c0515SJanosch Frank } 220*f77c0515SJanosch Frank } 221*f77c0515SJanosch Frank spin_unlock(&lock); 222*f77c0515SJanosch Frank } 223*f77c0515SJanosch Frank 224*f77c0515SJanosch Frank /*Expected to be called from boot cpu */ 225*f77c0515SJanosch Frank extern uint64_t *stackptr; 226*f77c0515SJanosch Frank void smp_setup(void) 227*f77c0515SJanosch Frank { 228*f77c0515SJanosch Frank int i = 0; 229*f77c0515SJanosch Frank unsigned short cpu0_addr = stap(); 230*f77c0515SJanosch Frank struct ReadCpuInfo *info = (void *)cpu_info_buffer; 231*f77c0515SJanosch Frank 232*f77c0515SJanosch Frank spin_lock(&lock); 233*f77c0515SJanosch Frank sclp_mark_busy(); 234*f77c0515SJanosch Frank info->h.length = PAGE_SIZE; 235*f77c0515SJanosch Frank sclp_service_call(SCLP_READ_CPU_INFO, cpu_info_buffer); 236*f77c0515SJanosch Frank 237*f77c0515SJanosch Frank if (smp_query_num_cpus() > 1) 238*f77c0515SJanosch Frank printf("SMP: Initializing, found %d cpus\n", info->nr_configured); 239*f77c0515SJanosch Frank 240*f77c0515SJanosch Frank cpus = calloc(info->nr_configured, sizeof(cpus)); 241*f77c0515SJanosch Frank for (i = 0; i < info->nr_configured; i++) { 242*f77c0515SJanosch Frank cpus[i].addr = info->entries[i].address; 243*f77c0515SJanosch Frank cpus[i].active = false; 244*f77c0515SJanosch Frank if (info->entries[i].address == cpu0_addr) { 245*f77c0515SJanosch Frank cpu0 = &cpus[i]; 246*f77c0515SJanosch Frank cpu0->stack = stackptr; 247*f77c0515SJanosch Frank cpu0->lowcore = (void *)0; 248*f77c0515SJanosch Frank cpu0->active = true; 249*f77c0515SJanosch Frank } 250*f77c0515SJanosch Frank } 251*f77c0515SJanosch Frank spin_unlock(&lock); 252*f77c0515SJanosch Frank } 253