1*b2c0b2cbSRussell King /* 2*b2c0b2cbSRussell King * NMI backtrace support 3*b2c0b2cbSRussell King * 4*b2c0b2cbSRussell King * Gratuitously copied from arch/x86/kernel/apic/hw_nmi.c by Russell King, 5*b2c0b2cbSRussell King * with the following header: 6*b2c0b2cbSRussell King * 7*b2c0b2cbSRussell King * HW NMI watchdog support 8*b2c0b2cbSRussell King * 9*b2c0b2cbSRussell King * started by Don Zickus, Copyright (C) 2010 Red Hat, Inc. 10*b2c0b2cbSRussell King * 11*b2c0b2cbSRussell King * Arch specific calls to support NMI watchdog 12*b2c0b2cbSRussell King * 13*b2c0b2cbSRussell King * Bits copied from original nmi.c file 14*b2c0b2cbSRussell King */ 15*b2c0b2cbSRussell King #include <linux/cpumask.h> 16*b2c0b2cbSRussell King #include <linux/delay.h> 17*b2c0b2cbSRussell King #include <linux/kprobes.h> 18*b2c0b2cbSRussell King #include <linux/nmi.h> 19*b2c0b2cbSRussell King #include <linux/seq_buf.h> 20*b2c0b2cbSRussell King 21*b2c0b2cbSRussell King #ifdef arch_trigger_all_cpu_backtrace 22*b2c0b2cbSRussell King /* For reliability, we're prepared to waste bits here. */ 23*b2c0b2cbSRussell King static DECLARE_BITMAP(backtrace_mask, NR_CPUS) __read_mostly; 24*b2c0b2cbSRussell King static cpumask_t printtrace_mask; 25*b2c0b2cbSRussell King 26*b2c0b2cbSRussell King #define NMI_BUF_SIZE 4096 27*b2c0b2cbSRussell King 28*b2c0b2cbSRussell King struct nmi_seq_buf { 29*b2c0b2cbSRussell King unsigned char buffer[NMI_BUF_SIZE]; 30*b2c0b2cbSRussell King struct seq_buf seq; 31*b2c0b2cbSRussell King }; 32*b2c0b2cbSRussell King 33*b2c0b2cbSRussell King /* Safe printing in NMI context */ 34*b2c0b2cbSRussell King static DEFINE_PER_CPU(struct nmi_seq_buf, nmi_print_seq); 35*b2c0b2cbSRussell King 36*b2c0b2cbSRussell King /* "in progress" flag of arch_trigger_all_cpu_backtrace */ 37*b2c0b2cbSRussell King static unsigned long backtrace_flag; 38*b2c0b2cbSRussell King 39*b2c0b2cbSRussell King static void print_seq_line(struct nmi_seq_buf *s, int start, int end) 40*b2c0b2cbSRussell King { 41*b2c0b2cbSRussell King const char *buf = s->buffer + start; 42*b2c0b2cbSRussell King 43*b2c0b2cbSRussell King printk("%.*s", (end - start) + 1, buf); 44*b2c0b2cbSRussell King } 45*b2c0b2cbSRussell King 46*b2c0b2cbSRussell King void nmi_trigger_all_cpu_backtrace(bool include_self, 47*b2c0b2cbSRussell King void (*raise)(cpumask_t *mask)) 48*b2c0b2cbSRussell King { 49*b2c0b2cbSRussell King struct nmi_seq_buf *s; 50*b2c0b2cbSRussell King int i, cpu, this_cpu = get_cpu(); 51*b2c0b2cbSRussell King 52*b2c0b2cbSRussell King if (test_and_set_bit(0, &backtrace_flag)) { 53*b2c0b2cbSRussell King /* 54*b2c0b2cbSRussell King * If there is already a trigger_all_cpu_backtrace() in progress 55*b2c0b2cbSRussell King * (backtrace_flag == 1), don't output double cpu dump infos. 56*b2c0b2cbSRussell King */ 57*b2c0b2cbSRussell King put_cpu(); 58*b2c0b2cbSRussell King return; 59*b2c0b2cbSRussell King } 60*b2c0b2cbSRussell King 61*b2c0b2cbSRussell King cpumask_copy(to_cpumask(backtrace_mask), cpu_online_mask); 62*b2c0b2cbSRussell King if (!include_self) 63*b2c0b2cbSRussell King cpumask_clear_cpu(this_cpu, to_cpumask(backtrace_mask)); 64*b2c0b2cbSRussell King 65*b2c0b2cbSRussell King cpumask_copy(&printtrace_mask, to_cpumask(backtrace_mask)); 66*b2c0b2cbSRussell King 67*b2c0b2cbSRussell King /* 68*b2c0b2cbSRussell King * Set up per_cpu seq_buf buffers that the NMIs running on the other 69*b2c0b2cbSRussell King * CPUs will write to. 70*b2c0b2cbSRussell King */ 71*b2c0b2cbSRussell King for_each_cpu(cpu, to_cpumask(backtrace_mask)) { 72*b2c0b2cbSRussell King s = &per_cpu(nmi_print_seq, cpu); 73*b2c0b2cbSRussell King seq_buf_init(&s->seq, s->buffer, NMI_BUF_SIZE); 74*b2c0b2cbSRussell King } 75*b2c0b2cbSRussell King 76*b2c0b2cbSRussell King if (!cpumask_empty(to_cpumask(backtrace_mask))) { 77*b2c0b2cbSRussell King pr_info("Sending NMI to %s CPUs:\n", 78*b2c0b2cbSRussell King (include_self ? "all" : "other")); 79*b2c0b2cbSRussell King raise(to_cpumask(backtrace_mask)); 80*b2c0b2cbSRussell King } 81*b2c0b2cbSRussell King 82*b2c0b2cbSRussell King /* Wait for up to 10 seconds for all CPUs to do the backtrace */ 83*b2c0b2cbSRussell King for (i = 0; i < 10 * 1000; i++) { 84*b2c0b2cbSRussell King if (cpumask_empty(to_cpumask(backtrace_mask))) 85*b2c0b2cbSRussell King break; 86*b2c0b2cbSRussell King mdelay(1); 87*b2c0b2cbSRussell King touch_softlockup_watchdog(); 88*b2c0b2cbSRussell King } 89*b2c0b2cbSRussell King 90*b2c0b2cbSRussell King /* 91*b2c0b2cbSRussell King * Now that all the NMIs have triggered, we can dump out their 92*b2c0b2cbSRussell King * back traces safely to the console. 93*b2c0b2cbSRussell King */ 94*b2c0b2cbSRussell King for_each_cpu(cpu, &printtrace_mask) { 95*b2c0b2cbSRussell King int len, last_i = 0; 96*b2c0b2cbSRussell King 97*b2c0b2cbSRussell King s = &per_cpu(nmi_print_seq, cpu); 98*b2c0b2cbSRussell King len = seq_buf_used(&s->seq); 99*b2c0b2cbSRussell King if (!len) 100*b2c0b2cbSRussell King continue; 101*b2c0b2cbSRussell King 102*b2c0b2cbSRussell King /* Print line by line. */ 103*b2c0b2cbSRussell King for (i = 0; i < len; i++) { 104*b2c0b2cbSRussell King if (s->buffer[i] == '\n') { 105*b2c0b2cbSRussell King print_seq_line(s, last_i, i); 106*b2c0b2cbSRussell King last_i = i + 1; 107*b2c0b2cbSRussell King } 108*b2c0b2cbSRussell King } 109*b2c0b2cbSRussell King /* Check if there was a partial line. */ 110*b2c0b2cbSRussell King if (last_i < len) { 111*b2c0b2cbSRussell King print_seq_line(s, last_i, len - 1); 112*b2c0b2cbSRussell King pr_cont("\n"); 113*b2c0b2cbSRussell King } 114*b2c0b2cbSRussell King } 115*b2c0b2cbSRussell King 116*b2c0b2cbSRussell King clear_bit(0, &backtrace_flag); 117*b2c0b2cbSRussell King smp_mb__after_atomic(); 118*b2c0b2cbSRussell King put_cpu(); 119*b2c0b2cbSRussell King } 120*b2c0b2cbSRussell King 121*b2c0b2cbSRussell King /* 122*b2c0b2cbSRussell King * It is not safe to call printk() directly from NMI handlers. 123*b2c0b2cbSRussell King * It may be fine if the NMI detected a lock up and we have no choice 124*b2c0b2cbSRussell King * but to do so, but doing a NMI on all other CPUs to get a back trace 125*b2c0b2cbSRussell King * can be done with a sysrq-l. We don't want that to lock up, which 126*b2c0b2cbSRussell King * can happen if the NMI interrupts a printk in progress. 127*b2c0b2cbSRussell King * 128*b2c0b2cbSRussell King * Instead, we redirect the vprintk() to this nmi_vprintk() that writes 129*b2c0b2cbSRussell King * the content into a per cpu seq_buf buffer. Then when the NMIs are 130*b2c0b2cbSRussell King * all done, we can safely dump the contents of the seq_buf to a printk() 131*b2c0b2cbSRussell King * from a non NMI context. 132*b2c0b2cbSRussell King */ 133*b2c0b2cbSRussell King static int nmi_vprintk(const char *fmt, va_list args) 134*b2c0b2cbSRussell King { 135*b2c0b2cbSRussell King struct nmi_seq_buf *s = this_cpu_ptr(&nmi_print_seq); 136*b2c0b2cbSRussell King unsigned int len = seq_buf_used(&s->seq); 137*b2c0b2cbSRussell King 138*b2c0b2cbSRussell King seq_buf_vprintf(&s->seq, fmt, args); 139*b2c0b2cbSRussell King return seq_buf_used(&s->seq) - len; 140*b2c0b2cbSRussell King } 141*b2c0b2cbSRussell King 142*b2c0b2cbSRussell King bool nmi_cpu_backtrace(struct pt_regs *regs) 143*b2c0b2cbSRussell King { 144*b2c0b2cbSRussell King int cpu = smp_processor_id(); 145*b2c0b2cbSRussell King 146*b2c0b2cbSRussell King if (cpumask_test_cpu(cpu, to_cpumask(backtrace_mask))) { 147*b2c0b2cbSRussell King printk_func_t printk_func_save = this_cpu_read(printk_func); 148*b2c0b2cbSRussell King 149*b2c0b2cbSRussell King /* Replace printk to write into the NMI seq */ 150*b2c0b2cbSRussell King this_cpu_write(printk_func, nmi_vprintk); 151*b2c0b2cbSRussell King pr_warn("NMI backtrace for cpu %d\n", cpu); 152*b2c0b2cbSRussell King show_regs(regs); 153*b2c0b2cbSRussell King this_cpu_write(printk_func, printk_func_save); 154*b2c0b2cbSRussell King 155*b2c0b2cbSRussell King cpumask_clear_cpu(cpu, to_cpumask(backtrace_mask)); 156*b2c0b2cbSRussell King return true; 157*b2c0b2cbSRussell King } 158*b2c0b2cbSRussell King 159*b2c0b2cbSRussell King return false; 160*b2c0b2cbSRussell King } 161*b2c0b2cbSRussell King NOKPROBE_SYMBOL(nmi_cpu_backtrace); 162*b2c0b2cbSRussell King #endif 163