1 /*- 2 * Copyright (c) 2015-2016 Svatopluk Kraus 3 * Copyright (c) 2015-2016 Michal Meloun 4 * All rights reserved. 5 * 6 * Redistribution and use in source and binary forms, with or without 7 * modification, are permitted provided that the following conditions 8 * are met: 9 * 1. Redistributions of source code must retain the above copyright 10 * notice, this list of conditions and the following disclaimer. 11 * 2. Redistributions in binary form must reproduce the above copyright 12 * notice, this list of conditions and the following disclaimer in the 13 * documentation and/or other materials provided with the distribution. 14 * 15 * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND 16 * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 17 * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 18 * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE 19 * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 20 * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 21 * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 22 * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 23 * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 24 * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 25 * SUCH DAMAGE. 26 */ 27 28 #include "opt_platform.h" 29 30 #include <sys/param.h> 31 #include <sys/systm.h> 32 33 #include <machine/cpu.h> 34 #include <machine/cpufunc.h> 35 #include <machine/intr.h> 36 37 /* 38 * arm_irq_memory_barrier() 39 * 40 * Ensure all writes to device memory have reached devices before proceeding. 41 * 42 * This is intended to be called from the post-filter and post-thread routines 43 * of an interrupt controller implementation. A peripheral device driver should 44 * use bus_space_barrier() if it needs to ensure a write has reached the 45 * hardware for some reason other than clearing interrupt conditions. 46 * 47 * The need for this function arises from the ARM weak memory ordering model. 48 * Writes to locations mapped with the Device attribute bypass any caches, but 49 * are buffered. Multiple writes to the same device will be observed by that 50 * device in the order issued by the cpu. Writes to different devices may 51 * appear at those devices in a different order than issued by the cpu. That 52 * is, if the cpu writes to device A then device B, the write to device B could 53 * complete before the write to device A. 54 * 55 * Consider a typical device interrupt handler which services the interrupt and 56 * writes to a device status-acknowledge register to clear the interrupt before 57 * returning. That write is posted to the L2 controller which "immediately" 58 * places it in a store buffer and automatically drains that buffer. This can 59 * be less immediate than you'd think... There may be no free slots in the store 60 * buffers, so an existing buffer has to be drained first to make room. The 61 * target bus may be busy with other traffic (such as DMA for various devices), 62 * delaying the drain of the store buffer for some indeterminate time. While 63 * all this delay is happening, execution proceeds on the CPU, unwinding its way 64 * out of the interrupt call stack to the point where the interrupt driver code 65 * is ready to EOI and unmask the interrupt. The interrupt controller may be 66 * accessed via a faster bus than the hardware whose handler just ran; the write 67 * to unmask and EOI the interrupt may complete quickly while the device write 68 * to ack and clear the interrupt source is still lingering in a store buffer 69 * waiting for access to a slower bus. With the interrupt unmasked at the 70 * interrupt controller but still active at the device, as soon as interrupts 71 * are enabled on the core the device re-interrupts immediately: now you've got 72 * a spurious interrupt on your hands. 73 * 74 * The right way to fix this problem is for every device driver to use the 75 * proper bus_space_barrier() calls in its interrupt handler. For ARM a single 76 * barrier call at the end of the handler would work. This would have to be 77 * done to every driver in the system, not just arm-specific drivers. 78 * 79 * Another potential fix is to map all device memory as Strongly-Ordered rather 80 * than Device memory, which takes the store buffers out of the picture. This 81 * has a pretty big impact on overall system performance, because each strongly 82 * ordered memory access causes all L2 store buffers to be drained. 83 * 84 * A compromise solution is to have the interrupt controller implementation call 85 * this function to establish a barrier between writes to the interrupt-source 86 * device and writes to the interrupt controller device. 87 * 88 * This takes the interrupt number as an argument, and currently doesn't use it. 89 * The plan is that maybe some day there is a way to flag certain interrupts as 90 * "memory barrier safe" and we can avoid this overhead with them. 91 */ 92 void 93 arm_irq_memory_barrier(uintptr_t irq) 94 { 95 96 dsb(); 97 cpu_l2cache_drain_writebuf(); 98 } 99