summaryrefslogtreecommitdiffstats
path: root/arch/arm/mm/dma-mapping.c (follow)
Commit message (Collapse)AuthorAgeFilesLines
* ARM: 9418/1: dma-mapping: Use iommu_paging_domain_alloc()Jason Gunthorpe2024-09-041-2/+4
| | | | | | | | | | | | | Since arm_iommu_create_mapping() now accepts the device, let's replace iommu_domain_alloc() with iommu_paging_domain_alloc() to retire the former. Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com> Reviewed-by: Jason Gunthorpe <jgg@nvidia.com> Reviewed-by: Vasant Hegde <vasant.hegde@amd.com> Acked-by: Michael S. Tsirkin <mst@redhat.com> Acked-by: Jeff Johnson <quic_jjohnson@quicinc.com> Signed-off-by: Jason Gunthorpe <jgg@ziepe.ca> Signed-off-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
* ARM: 9417/1: dma-mapping: Pass device to arm_iommu_create_mapping()Jason Gunthorpe2024-09-041-4/+4
| | | | | | | | | | | | | | | | All users of ARM IOMMU mappings create them for a particular device, so change the interface to accept the device rather than forcing a vague indirection through a bus type. This prepares for making a similar change to iommu_domain_alloc() itself. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com> Reviewed-by: Jason Gunthorpe <jgg@nvidia.com> Reviewed-by: Vasant Hegde <vasant.hegde@amd.com> Acked-by: Michael S. Tsirkin <mst@redhat.com> Acked-by: Jeff Johnson <quic_jjohnson@quicinc.com> Signed-off-by: Jason Gunthorpe <jgg@ziepe.ca> Signed-off-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
* dma-mapping: Simplify arch_setup_dma_ops()Robin Murphy2024-04-261-7/+9
| | | | | | | | | | | | | | | | The dma_base, size and iommu arguments are only used by ARM, and can now easily be deduced from the device itself, so there's no need to pass them through the callchain as well. Acked-by: Rob Herring <robh@kernel.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Michael Kelley <mhklinux@outlook.com> # For Hyper-V Reviewed-by: Jason Gunthorpe <jgg@nvidia.com> Tested-by: Hanjun Guo <guohanjun@huawei.com> Signed-off-by: Robin Murphy <robin.murphy@arm.com> Acked-by: Catalin Marinas <catalin.marinas@arm.com> Link: https://lore.kernel.org/r/5291c2326eab405b1aa7693aa964e8d3cb7193de.1713523152.git.robin.murphy@arm.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
* Merge tag 'iommu-updates-v6.8' of ↵Linus Torvalds2024-01-191-5/+5
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu Pull iommu updates from Joerg Roedel: "Core changes: - Fix race conditions in device probe path - Retire IOMMU bus_ops - Support for passing custom allocators to page table drivers - Clean up Kconfig around IOMMU_SVA - Support for sharing SVA domains with all devices bound to a mm - Firmware data parsing cleanup - Tracing improvements for iommu-dma code - Some smaller fixes and cleanups ARM-SMMU drivers: - Device-tree binding updates: - Add additional compatible strings for Qualcomm SoCs - Document Adreno clocks for Qualcomm's SM8350 SoC - SMMUv2: - Implement support for the ->domain_alloc_paging() callback - Ensure Secure context is restored following suspend of Qualcomm SMMU implementation - SMMUv3: - Disable stalling mode for the "quiet" context descriptor - Minor refactoring and driver cleanups Intel VT-d driver: - Cleanup and refactoring AMD IOMMU driver: - Improve IO TLB invalidation logic - Small cleanups and improvements Rockchip IOMMU driver: - DT binding update to add Rockchip RK3588 Apple DART driver: - Apple M1 USB4/Thunderbolt DART support - Cleanups Virtio IOMMU driver: - Add support for iotlb_sync_map - Enable deferred IO TLB flushes" * tag 'iommu-updates-v6.8' of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu: (66 commits) iommu: Don't reserve 0-length IOVA region iommu/vt-d: Move inline helpers to header files iommu/vt-d: Remove unused vcmd interfaces iommu/vt-d: Remove unused parameter of intel_pasid_setup_pass_through() iommu/vt-d: Refactor device_to_iommu() to retrieve iommu directly iommu/sva: Fix memory leak in iommu_sva_bind_device() dt-bindings: iommu: rockchip: Add Rockchip RK3588 iommu/dma: Trace bounce buffer usage when mapping buffers iommu/arm-smmu: Convert to domain_alloc_paging() iommu/arm-smmu: Pass arm_smmu_domain to internal functions iommu/arm-smmu: Implement IOMMU_DOMAIN_BLOCKED iommu/arm-smmu: Convert to a global static identity domain iommu/arm-smmu: Reorganize arm_smmu_domain_add_master() iommu/arm-smmu-v3: Remove ARM_SMMU_DOMAIN_NESTED iommu/arm-smmu-v3: Master cannot be NULL in arm_smmu_write_strtab_ent() iommu/arm-smmu-v3: Add a type for the STE iommu/arm-smmu-v3: disable stall for quiet_cd iommu/qcom: restore IOMMU state if needed iommu/arm-smmu-qcom: Add QCM2290 MDSS compatible iommu/arm-smmu-qcom: Add missing GMU entry to match table ...
| * iommu: Remove struct iommu_ops *iommu from arch_setup_dma_ops()Jason Gunthorpe2023-12-121-5/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is not being used to pass ops, it is just a way to tell if an iommu driver was probed. These days this can be detected directly via device_iommu_mapped(). Call device_iommu_mapped() in the two places that need to check it and remove the iommu parameter everywhere. Reviewed-by: Jerry Snitselaar <jsnitsel@redhat.com> Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com> Reviewed-by: Moritz Fischer <mdf@kernel.org> Acked-by: Christoph Hellwig <hch@lst.de> Acked-by: Rob Herring <robh@kernel.org> Tested-by: Hector Martin <marcan@marcan.st> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com> Link: https://lore.kernel.org/r/1-v2-16e4def25ebb+820-iommu_fwspec_p1_jgg@nvidia.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
* | ARM: 9331/1: ARM/dma-mapping: replace kzalloc() and vzalloc() with kvzalloc()Chen Haonan2024-01-051-4/+1
|/ | | | | | | | | using kvzalloc() simplifies the code by avoiding the use of different memory allocation functions for different situations, making the code more uniform and readable. Signed-off-by: Chen Haonan <chen.haonan2@zte.com.cn> Signed-off-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
* arm: implement the new page table range APIMatthew Wilcox (Oracle)2023-08-251-13/+15
| | | | | | | | | | | | | | | | | | | Add set_ptes(), update_mmu_cache_range(), flush_dcache_folio() and flush_icache_pages(). Change the PG_dcache_clear flag from being per-page to per-folio which makes __dma_page_dev_to_cpu() a bit more exciting. Also add flush_cache_pages(), even though this isn't used by generic code (yet?) [m.szyprowski@samsung.com: fix potential endless loop in __dma_page_dev_to_cpu()] Link: https://lkml.kernel.org/r/20230809172737.3574190-1-m.szyprowski@samsung.com [willy@infradead.org: fix folio conversion in __dma_page_dev_to_cpu()] Link: https://lkml.kernel.org/r/20230823191852.1556561-1-willy@infradead.org Link: https://lkml.kernel.org/r/20230802151406.3735276-10-willy@infradead.org Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org> Signed-off-by: Marek Szyprowski <m.szyprowski@samsung.com> Acked-by: Mike Rapoport (IBM) <rppt@kernel.org> Reviewed-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk> Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
* Merge tag 'asm-generic-6.5' of ↵Linus Torvalds2023-07-061-1/+1
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | git://git.kernel.org/pub/scm/linux/kernel/git/arnd/asm-generic Pull asm-generic updates from Arnd Bergmann: "These are cleanups for architecture specific header files: - the comments in include/linux/syscalls.h have gone out of sync and are really pointless, so these get removed - The asm/bitsperlong.h header no longer needs to be architecture specific on modern compilers, so use a generic version for newer architectures that use new enough userspace compilers - A cleanup for virt_to_pfn/virt_to_bus to have proper type checking, forcing the use of pointers" * tag 'asm-generic-6.5' of git://git.kernel.org/pub/scm/linux/kernel/git/arnd/asm-generic: syscalls: Remove file path comments from headers tools arch: Remove uapi bitsperlong.h of hexagon and microblaze asm-generic: Unify uapi bitsperlong.h for arm64, riscv and loongarch m68k/mm: Make pfn accessors static inlines arm64: memory: Make virt_to_pfn() a static inline ARM: mm: Make virt_to_pfn() a static inline asm-generic/page.h: Make pfn accessors static inlines xen/netback: Pass (void *) to virt_to_page() netfs: Pass a pointer to virt_to_page() cifs: Pass a pointer to virt_to_page() in cifsglob cifs: Pass a pointer to virt_to_page() riscv: mm: init: Pass a pointer to virt_to_page() ARC: init: Pass a pointer to virt_to_pfn() in init m68k: Pass a pointer to virt_to_pfn() virt_to_page() fs/proc/kcore.c: Pass a pointer to virt_addr_valid()
| * ARM: mm: Make virt_to_pfn() a static inlineLinus Walleij2023-05-291-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Making virt_to_pfn() a static inline taking a strongly typed (const void *) makes the contract of a passing a pointer of that type to the function explicit and exposes any misuse of the macro virt_to_pfn() acting polymorphic and accepting many types such as (void *), (unitptr_t) or (unsigned long) as arguments without warnings. Doing this is a bit intrusive: virt_to_pfn() requires PHYS_PFN_OFFSET and PAGE_SHIFT to be defined, and this is defined in <asm/page.h>, so this must be included *before* <asm/memory.h>. The use of macros were obscuring the unclear inclusion order here, as the macros would eventually be resolved, but a static inline like this cannot be compiled with unresolved macros. The naive solution to include <asm/page.h> at the top of <asm/memory.h> does not work, because <asm/memory.h> sometimes includes <asm/page.h> at the end of itself, which would create a confusing inclusion loop. So instead, take the approach to always unconditionally include <asm/page.h> at the end of <asm/memory.h> arch/arm uses <asm/memory.h> explicitly in a lot of places, however it turns out that if we just unconditionally include <asm/memory.h> into <asm/page.h> and switch all inclusions of <asm/memory.h> to <asm/page.h> instead, we enforce the right order and <asm/memory.h> will always have access to the definitions. Put an inclusion guard in place making it impossible to include <asm/memory.h> explicitly. Link: https://lore.kernel.org/linux-mm/20220701160004.2ffff4e5ab59a55499f4c736@linux-foundation.org/ Signed-off-by: Linus Walleij <linus.walleij@linaro.org>
* | ARM: 9301/1: dma-mapping: hide unused dma_contiguous_early_fixup functionArnd Bergmann2023-06-191-0/+2
|/ | | | | | | | | | | | | | | | | The dma_contiguous_early_fixup() function is a global __weak stub with an arm specific override, but the declaration is in an #ifdef. If CONFIG_DMA_CMA is disabled, there is no caller and no prototype, which adds a warning for the definition: arm/mm/dma-mapping.c:261:13: error: no previous prototype for 'dma_contiguous_early_fixup' [-Werror=missing-prototypes] Enclose the definition in the same #ifdef as the prototype to avoid that and save a few bytes of .init.text. Reviewed-by: Kees Cook <keescook@chromium.org> Signed-off-by: Arnd Bergmann <arnd@arndb.de> Signed-off-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
* ARM/dma-mapping: const a pointer to bus_type in arm_iommu_create_mapping()Greg Kroah-Hartman2023-03-231-1/+1
| | | | | | | | | | | | | | | | | | | | Change the function arm_iommu_create_mapping() to take a pointer to a const bus_type as the function does not modify the variable the pointer points to at all, and the driver core bus functions it calls all expect a const * type. Cc: Russell King <linux@armlinux.org.uk> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Cc: "Russell King (Oracle)" <rmk+kernel@armlinux.org.uk> Cc: Arnd Bergmann <arnd@arndb.de> Cc: Robin Murphy <robin.murphy@arm.com> Cc: Kees Cook <keescook@chromium.org> Cc: Lukas Bulwahn <lukas.bulwahn@gmail.com> Cc: Ben Dooks <ben-linux@fluff.org> Cc: Linus Walleij <linus.walleij@linaro.org> Cc: linux-arm-kernel@lists.infradead.org Link: https://lore.kernel.org/r/20230313182918.1312597-33-gregkh@linuxfoundation.org Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
* iommu: Add a gfp parameter to iommu_map()Jason Gunthorpe2023-01-251-4/+7
| | | | | | | | | | | | | The internal mechanisms support this, but instead of exposting the gfp to the caller it wrappers it into iommu_map() and iommu_map_atomic() Fix this instead of adding more variants for GFP_KERNEL_ACCOUNT. Reviewed-by: Kevin Tian <kevin.tian@intel.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com> Reviewed-by: Mathieu Poirier <mathieu.poirier@linaro.org> Link: https://lore.kernel.org/r/1-v3-76b587fe28df+6e3-iommu_map_gfp_jgg@nvidia.com Signed-off-by: Joerg Roedel <jroedel@suse.de>
* dma-mapping: reject __GFP_COMP in dma_alloc_attrsChristoph Hellwig2022-11-211-17/+0
| | | | | | | | | | | | DMA allocations can never be turned back into a page pointer, so requesting compound pages doesn't make sense and it can't even be supported at all by various backends. Reject __GFP_COMP with a warning in dma_alloc_attrs, and stop clearing the flag in the arm dma ops and dma-iommu. Signed-off-by: Christoph Hellwig <hch@lst.de> Acked-by: Marek Szyprowski <m.szyprowski@samsung.com>
* Merge tag 'dma-mapping-6.1-2022-10-10' of ↵Linus Torvalds2022-10-101-2/+8
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | git://git.infradead.org/users/hch/dma-mapping Pull dma-mapping updates from Christoph Hellwig: - fix a regression in the ARM dma-direct conversion (Christoph Hellwig) - use memcpy_{from,to}_page (Fabio M. De Francesco) - cleanup the swiotlb MAINTAINERS entry (Lukas Bulwahn) - make SG table pool allocation less fragile (Masahiro Yamada) - don't panic on swiotlb initialization failure (Robin Murphy) * tag 'dma-mapping-6.1-2022-10-10' of git://git.infradead.org/users/hch/dma-mapping: ARM/dma-mapping: remove the dma_coherent member of struct dev_archdata ARM/dma-mappіng: don't override ->dma_coherent when set from a bus notifier lib/sg_pool: change module_init(sg_pool_init) to subsys_initcall MAINTAINERS: merge SWIOTLB SUBSYSTEM into DMA MAPPING HELPERS swiotlb: don't panic! swiotlb: replace kmap_atomic() with memcpy_{from,to}_page()
| * ARM/dma-mapping: remove the dma_coherent member of struct dev_archdataChristoph Hellwig2022-10-061-3/+1
| | | | | | | | | | | | | | | | | | Since commit ae626eb97376 ("ARM/dma-mapping: use dma-direct unconditionally") only the dma_coherent flag in struct device is used, so remove the now write only flag in struct dev_archdata. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
| * ARM/dma-mappіng: don't override ->dma_coherent when set from a bus notifierChristoph Hellwig2022-10-061-2/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Commit ae626eb97376 ("ARM/dma-mapping: use dma-direct unconditionally") caused a regression on the mvebu platform, wherein devices that are dma-coherent are marked as dma-noncoherent, because although mvebu_hwcc_notifier() after that commit still marks then as coherent, the arm_coherent_dma_ops() function, which is called later, overwrites this setting, since it is being called from drivers/of/device.c with coherency parameter determined by of_dma_is_coherent(), and the device-trees do not declare the 'dma-coherent' property. Fix this by defaulting never clearing the dma_coherent flag in arm_coherent_dma_ops(). Fixes: ae626eb97376 ("ARM/dma-mapping: use dma-direct unconditionally") Reported-by: Marek Behún <kabel@kernel.org> Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk> Tested-by: Marek Behún <kabel@kernel.org>
* | ARM: 9240/1: dma-mapping: Pass (void *) to virt_to_page()Linus Walleij2022-10-041-1/+1
|/ | | | | | | | | Pointers to virtual memory functions are (void *) but the __dma_update_pte() function is passing an unsigned long. Fix this up by explicit cast. Signed-off-by: Linus Walleij <linus.walleij@linaro.org> Signed-off-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
* Merge tag 'dma-mapping-5.20-2022-08-06' of ↵Linus Torvalds2022-08-061-579/+73
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | git://git.infradead.org/users/hch/dma-mapping Pull dma-mapping updates from Christoph Hellwig: - convert arm32 to the common dma-direct code (Arnd Bergmann, Robin Murphy, Christoph Hellwig) - restructure the PCIe peer to peer mapping support (Logan Gunthorpe) - allow the IOMMU code to communicate an optional DMA mapping length and use that in scsi and libata (John Garry) - split the global swiotlb lock (Tianyu Lan) - various fixes and cleanup (Chao Gao, Dan Carpenter, Dongli Zhang, Lukas Bulwahn, Robin Murphy) * tag 'dma-mapping-5.20-2022-08-06' of git://git.infradead.org/users/hch/dma-mapping: (45 commits) swiotlb: fix passing local variable to debugfs_create_ulong() dma-mapping: reformat comment to suppress htmldoc warning PCI/P2PDMA: Remove pci_p2pdma_[un]map_sg() RDMA/rw: drop pci_p2pdma_[un]map_sg() RDMA/core: introduce ib_dma_pci_p2p_dma_supported() nvme-pci: convert to using dma_map_sgtable() nvme-pci: check DMA ops when indicating support for PCI P2PDMA iommu/dma: support PCI P2PDMA pages in dma-iommu map_sg iommu: Explicitly skip bus address marked segments in __iommu_map_sg() dma-mapping: add flags to dma_map_ops to indicate PCI P2PDMA support dma-direct: support PCI P2PDMA pages in dma-direct map_sg dma-mapping: allow EREMOTEIO return code for P2PDMA transfers PCI/P2PDMA: Introduce helpers for dma_map_sg implementations PCI/P2PDMA: Attempt to set map_type if it has not been set lib/scatterlist: add flag for indicating P2PDMA segments in an SGL swiotlb: clean up some coding style and minor issues dma-mapping: update comment after dmabounce removal scsi: sd: Add a comment about limiting max_sectors to shost optimal limit ata: libata-scsi: cap ata_device->max_sectors according to shost->max_sectors scsi: scsi_transport_sas: cap shost opt_sectors according to DMA optimal limit ...
| * dma-mapping: update comment after dmabounce removalLukas Bulwahn2022-07-221-2/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | Commit e3217540c271 ("ARM/dma-mapping: remove dmabounce") removes the config DMABOUNCE. A comment to the function __dma_page_cpu_to_dev() refers to this removed config DMABOUNCE. Remove the obsolete explanation, but keep the recommendation not to use __dma_page_cpu_to_dev() and use dma_sync_* functions instead. Signed-off-by: Lukas Bulwahn <lukas.bulwahn@gmail.com> Acked-by: Arnd Bergmann <arnd@arndb.de> Signed-off-by: Christoph Hellwig <hch@lst.de>
| * ARM/dma-mapping: merge IOMMU opsRobin Murphy2022-07-071-24/+13
| | | | | | | | | | | | | | | | | | | | | | The dma_sync_* operations are now the only difference between the coherent and non-coherent IOMMU ops. Some minor tweaks to make those safe for coherent devices with minimal overhead, and we can condense down to a single set of DMA ops. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Christoph Hellwig <hch@lst.de> Tested-by: Marc Zyngier <maz@kernel.org>
| * ARM/dma-mapping: consolidate IOMMU ops callbacksRobin Murphy2022-07-071-183/+55
| | | | | | | | | | | | | | | | | | | | Merge the coherent and non-coherent callbacks down to a single implementation each, relying on the generic dev->dma_coherent flag at the points where the difference matters. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Christoph Hellwig <hch@lst.de> Tested-by: Marc Zyngier <maz@kernel.org>
| * ARM/dma-mapping: drop .dma_supported for IOMMU opsRobin Murphy2022-07-071-23/+0
| | | | | | | | | | | | | | | | | | | | | | When an IOMMU is present, we trust that it should be capable of remapping any physical memory, and since the device masks represent the input (virtual) addresses to the IOMMU it makes no sense to validate them against physical PFNs anyway. Signed-off-by: Robin Murphy <robin.murphy@arm.com> Signed-off-by: Christoph Hellwig <hch@lst.de> Tested-by: Marc Zyngier <maz@kernel.org>
| * ARM/dma-mapping: use dma-direct unconditionallyChristoph Hellwig2022-07-071-350/+15
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Use dma-direct unconditionally on arm. It has already been used for some time for LPAE and nommu configurations. This mostly changes the streaming mapping implementation and the (simple) coherent allocator for device that are DMA coherent. The existing complex allocator for uncached mappings for non-coherent devices is still used as is using the arch_dma_alloc/arch_dma_free hooks. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Arnd Bergmann <arnd@arndb.de> Acked-by: Andre Przywara <andre.przywara@arm.com> [highbank] Tested-by: Marc Zyngier <maz@kernel.org>
| * ARM/dma-mapping: use dma_to_phys/phys_to_dma in the dma-mapping codeChristoph Hellwig2022-07-071-12/+12
| | | | | | | | | | | | | | | | | | Use the helpers as expected by the dma-direct code in the old arm dma-mapping code to ease a gradual switch to the common DMA code. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Arnd Bergmann <arnd@arndb.de> Tested-by: Marc Zyngier <maz@kernel.org>
| * ARM/dma-mapping: mark various dma-mapping routines static in dma-mapping.cChristoph Hellwig2022-07-071-54/+46
| | | | | | | | | | | | | | | | | | | | With the dmabounce removal these aren't used outside of dma-mapping.c, so mark them static. Move the dma_map_ops declarations down a bit to avoid lots of forward declarations. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Arnd Bergmann <arnd@arndb.de> Tested-by: Marc Zyngier <maz@kernel.org>
* | ARM: 9218/1: dma-mapping: fix pointer/integer warningBen Dooks2022-07-281-1/+1
|/ | | | | | | | | | | Fix the use of a pointer assignment from integer where false is being used instead of NULL. Fix the following warning by usign NULL: arch/arm/mm/dma-mapping.c:712:52: warning: Using plain integer as NULL pointer Signed-off-by: Ben Dooks <ben-linux@fluff.org> Signed-off-by: Russell King (Oracle) <rmk+kernel@armlinux.org.uk>
* arm/xen: Introduce xen_setup_dma_ops()Oleksandr Tyshchenko2022-06-061-5/+2
| | | | | | | | | | | | | | | | | This patch introduces new helper and places it in new header. The helper's purpose is to assign any Xen specific DMA ops in a single place. For now, we deal with xen-swiotlb DMA ops only. The one of the subsequent commits in current series will add xen-grant DMA ops case. Also re-use the xen_swiotlb_detect() check on Arm32. Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org> [For arm64] Acked-by: Catalin Marinas <catalin.marinas@arm.com> Link: https://lore.kernel.org/r/1654197833-25362-2-git-send-email-olekstysh@gmail.com Signed-off-by: Juergen Gross <jgross@suse.com>
* ARM/dma-mapping: Remove CMA code when not built with CMAKees Cook2022-03-311-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The MAX_CMA_AREAS could be set to 0, which would result in code that would attempt to operate beyond the end of a zero-sized array. If CONFIG_CMA is disabled, just remove this code entirely. Found when building arm on GCC 10.x for several defconfigs (e.g. axm55xx_defconfig) under -Warray-bounds: arch/arm/mm/dma-mapping.c:396:22: warning: array subscript <unknown> is outside array bounds of 'struct dma_contig_early_reserve[0]' [-Warray-bounds] 396 | dma_mmu_remap[dma_mmu_remap_num].size = size; | ~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~ arch/arm/mm/dma-mapping.c:389:40: note: while referencing 'dma_mmu_remap' 389 | static struct dma_contig_early_reserve dma_mmu_remap[MAX_CMA_AREAS] __initdata; | ^~~~~~~~~~~~~ Cc: Russell King <linux@armlinux.org.uk> Cc: Logan Gunthorpe <logang@deltatee.com> Cc: Martin Oliveira <martin.oliveira@eideticom.com> Cc: David Hildenbrand <david@redhat.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Stephen Rothwell <sfr@canb.auug.org.au> Cc: Zi Yan <ziy@nvidia.com> Cc: Hari Bathini <hbathini@linux.ibm.com> Cc: Minchan Kim <minchan@kernel.org> Cc: Mike Kravetz <mike.kravetz@oracle.com> Cc: linux-arm-kernel@lists.infradead.org Link: https://lore.kernel.org/all/6243ee60.1c69fb81.16de6.7dbf@mx.google.com/ Signed-off-by: Kees Cook <keescook@chromium.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Link: https://lore.kernel.org/lkml/20220310070041.GA24874@lst.de Reviewed-by: David Hildenbrand <david@redhat.com> Link: https://lore.kernel.org/lkml/9059fa71-330f-f04f-b155-2850abb72a71@redhat.com
* ARM/dma-mapping: don't set failed sg dma_address to DMA_MAPPING_ERRORLogan Gunthorpe2021-08-091-1/+0
| | | | | | | | | | | | Setting the ->dma_address to DMA_MAPPING_ERROR is not part of the ->map_sg calling convention, so remove it. Link: https://lore.kernel.org/linux-mips/20210716063241.GC13345@lst.de/ Suggested-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Logan Gunthorpe <logang@deltatee.com> Cc: Russell King <linux@armlinux.org.uk> Cc: Thomas Bogendoerfer <tsbogend@alpha.franken.de> Signed-off-by: Christoph Hellwig <hch@lst.de>
* ARM/dma-mapping: return error code from .map_sg() opsMartin Oliveira2021-08-091-9/+16
| | | | | | | | | | | | | The .map_sg() op now expects an error code instead of zero on failure. In the case of a DMA_MAPPING_ERROR, -EIO is returned. Otherwise, -ENOMEM or -EINVAL is returned depending on the error from __map_sg_chunk(). Signed-off-by: Martin Oliveira <martin.oliveira@eideticom.com> Signed-off-by: Logan Gunthorpe <logang@deltatee.com> Cc: Russell King <linux@armlinux.org.uk> Cc: Thomas Bogendoerfer <tsbogend@alpha.franken.de> Signed-off-by: Christoph Hellwig <hch@lst.de>
* dma-mapping: merge <linux/dma-noncoherent.h> into <linux/dma-map-ops.h>Christoph Hellwig2020-10-061-1/+0
| | | | | | | Move more nitty gritty DMA implementation details into the common internal header. Signed-off-by: Christoph Hellwig <hch@lst.de>
* dma-mapping: remove <asm/dma-contiguous.h>Christoph Hellwig2020-10-061-1/+0
| | | | | | | Just provide a weak default definition of dma_contiguous_early_fixup and let arm override it. Signed-off-by: Christoph Hellwig <hch@lst.de>
* dma-mapping: merge <linux/dma-contiguous.h> into <linux/dma-map-ops.h>Christoph Hellwig2020-10-061-1/+0
| | | | | | | Merge dma-contiguous.h into dma-map-ops.h, after removing the comment describing the contiguous allocator into kernel/dma/contigous.c. Signed-off-by: Christoph Hellwig <hch@lst.de>
* dma-mapping: split <linux/dma-mapping.h>Christoph Hellwig2020-10-061-1/+1
| | | | | | | | | | | Split out all the bits that are purely for dma_map_ops implementations and related code into a new <linux/dma-map-ops.h> header so that they don't get pulled into all the drivers. That also means the architecture specific <asm/dma-mapping.h> is not pulled in by <linux/dma-mapping.h> any more, which leads to a missing includes that were pulled in by the x86 or arm versions in a few not overly portable drivers. Signed-off-by: Christoph Hellwig <hch@lst.de>
* dma-mapping: add a new dma_alloc_pages APIChristoph Hellwig2020-09-251-0/+4
| | | | | | | | | | | | | This API is the equivalent of alloc_pages, except that the returned memory is guaranteed to be DMA addressable by the passed in device. The implementation will also be used to provide a more sensible replacement for DMA_ATTR_NON_CONSISTENT flag. Additionally dma_alloc_noncoherent is switched over to use dma_alloc_pages as its backend. Signed-off-by: Christoph Hellwig <hch@lst.de> Acked-by: Thomas Bogendoerfer <tsbogend@alpha.franken.de> (MIPS part)
* ARM/dma-mapping: merge __dma_supported into arm_dma_supportedChristoph Hellwig2020-03-201-24/+17
| | | | | | | | Merge __dma_supported into its only caller, and move the resulting function so that it doesn't need a forward declaration. Also mark it static as there are no callers outside of dma-mapping.c. Signed-off-by: Christoph Hellwig <hch@lst.de>
* ARM/dma-mapping: take the bus limit into account in __dma_allocChristoph Hellwig2020-03-201-1/+1
| | | | | | | | | | The DMA coherent allocator needs to take bus limits into account for picking the zone that the memory is allocated from. Reported-by: Roger Quadros <rogerq@ti.com> Signed-off-by: Christoph Hellwig <hch@lst.de> Tested-by: Peter Ujfalusi <peter.ujfalusi@ti.com> Tested-by: Roger Quadros <rogerq@ti.com>
* ARM/dma-mapping: remove get_coherent_dma_maskChristoph Hellwig2020-03-201-37/+4
| | | | | | | | The core DMA code already checks for valid DMA masks earlier on, and doesn't allow NULL struct device pointers. Remove the now not required checks. Signed-off-by: Christoph Hellwig <hch@lst.de>
* ARM: dma-api: fix max_pfn off-by-one error in __dma_supported()Chen-Yu Tsai2020-02-041-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | max_pfn, as set in arch/arm/mm/init.c: static void __init find_limits(unsigned long *min, unsigned long *max_low, unsigned long *max_high) { *max_low = PFN_DOWN(memblock_get_current_limit()); *min = PFN_UP(memblock_start_of_DRAM()); *max_high = PFN_DOWN(memblock_end_of_DRAM()); } with memblock_end_of_DRAM() pointing to the next byte after DRAM. As such, max_pfn points to the PFN after the end of DRAM. Thus when using max_pfn to check DMA masks, we should subtract one when checking DMA ranges against it. Commit 8bf1268f48ad ("ARM: dma-api: fix off-by-one error in __dma_supported()") fixed the same issue, but missed this spot. This issue was found while working on the sun4i-csi v4l2 driver on the Allwinner R40 SoC. On Allwinner SoCs, DRAM is offset at 0x40000000, and we are starting to use of_dma_configure() with the "dma-ranges" property in the device tree to have the DMA API handle the offset. In this particular instance, dma-ranges was set to the same range as the actual available (2 GiB) DRAM. The following error appeared when the driver attempted to allocate a buffer: sun4i-csi 1c09000.csi: Coherent DMA mask 0x7fffffff (pfn 0x40000-0xc0000) covers a smaller range of system memory than the DMA zone pfn 0x0-0xc0001 sun4i-csi 1c09000.csi: dma_alloc_coherent of size 307200 failed Fixing the off-by-one error makes things work. Link: http://lkml.kernel.org/r/20191224030239.5656-1-wens@kernel.org Fixes: 11a5aa32562e ("ARM: dma-mapping: check DMA mask against available memory") Fixes: 9f28cde0bc64 ("ARM: another fix for the DMA mapping checks") Fixes: ab746573c405 ("ARM: dma-mapping: allow larger DMA mask than supported") Signed-off-by: Chen-Yu Tsai <wens@csie.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Cc: Russell King <linux@armlinux.org.uk> Cc: Robin Murphy <robin.murphy@arm.com> Cc: <stable@vger.kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
* lib/genalloc.c: rename addr_in_gen_pool to gen_pool_has_addrHuang Shijie2019-12-051-1/+1
| | | | | | | | | | | | | | | | | | | | Follow the kernel conventions, rename addr_in_gen_pool to gen_pool_has_addr. [sjhuang@iluvatar.ai: fix Documentation/ too] Link: http://lkml.kernel.org/r/20181229015914.5573-1-sjhuang@iluvatar.ai Link: http://lkml.kernel.org/r/20181228083950.20398-1-sjhuang@iluvatar.ai Signed-off-by: Huang Shijie <sjhuang@iluvatar.ai> Reviewed-by: Andrew Morton <akpm@linux-foundation.org> Cc: Russell King <linux@armlinux.org.uk> Cc: Arnd Bergmann <arnd@arndb.de> Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Cc: Christoph Hellwig <hch@lst.de> Cc: Marek Szyprowski <m.szyprowski@samsung.com> Cc: Robin Murphy <robin.murphy@arm.com> Cc: Stephen Rothwell <sfr@canb.auug.org.au> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
* Merge tag 'for-linus' of git://git.armlinux.org.uk/~rmk/linux-armLinus Torvalds2019-11-301-15/+19
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Pull ARM updates from Russell King: - improve ARM implementation of pfn_valid() - various sparse fixes - spelling fixes - add further ARMv8 debug architecture versions - clang fix for decompressor - update to generic vDSO - remove Brahma-B53 from spectre hardening - initialise broadcast hrtimer device - use correct nm executable in decompressor - remove old mcount et.al. * tag 'for-linus' of git://git.armlinux.org.uk/~rmk/linux-arm: (26 commits) ARM: 8940/1: ftrace: remove mcount(),ftrace_caller_old() and ftrace_call_old() ARM: 8939/1: kbuild: use correct nm executable ARM: 8938/1: kernel: initialize broadcast hrtimer based clock event device ARM: 8937/1: spectre-v2: remove Brahma-B53 from hardening ARM: 8933/1: replace Sun/Solaris style flag on section directive ARM: 8932/1: Add clock_gettime64 entry point ARM: 8931/1: Add clock_getres entry point ARM: 8930/1: Add support for generic vDSO ARM: 8929/1: use APSR_nzcv instead of r15 as mrc operand ARM: 8927/1: ARM/hw_breakpoint: add more ARMv8 debug architecture versions support ARM: 8918/2: only build return_address() if needed ARM: 8928/1: ARM_ERRATA_775420: Spelling s/date/data/ ARM: 8925/1: tcm: include <asm/tcm.h> for missing declarations ARM: 8924/1: tcm: make dtcm_end and itcm_end static ARM: 8923/1: mm: include <asm/vga.h> for vga_base ARM: 8922/1: parse_dt_topology() rate is pointer to __be32 ARM: 8920/1: share get_signal_page from signal.c to process.c ARM: 8919/1: make unexported functions static ARM: 8917/1: mm: include <asm/set_memory.h> ARM: 8916/1: mm: make set_section_perms() static ...
| * ARM: 8909/1: make unexported items staticBen Dooks2019-10-271-15/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Fixup the following sparse warnings by making the functions and structures static. arch/arm/mm/dma-mapping.c:1562:6: warning: symbol '__arm_iommu_free_attrs' was not declared. Should it be static? arch/arm/mm/dma-mapping.c:1586:6: warning: symbol 'arm_iommu_free_attrs' was not declared. Should it be static? arch/arm/mm/dma-mapping.c:1592:6: warning: symbol 'arm_coherent_iommu_free_attrs' was not declared. Should it be static? arch/arm/mm/dma-mapping.c:1716:5: warning: symbol 'arm_coherent_iommu_map_sg' was not declared. Should it be static? arch/arm/mm/dma-mapping.c:1734:5: warning: symbol 'arm_iommu_map_sg' was not declared. Should it be static? arch/arm/mm/dma-mapping.c:1767:6: warning: symbol 'arm_coherent_iommu_unmap_sg' was not declared. Should it be static? arch/arm/mm/dma-mapping.c:1784:6: warning: symbol 'arm_iommu_unmap_sg' was not declared. Should it be static? arch/arm/mm/dma-mapping.c:1798:6: warning: symbol 'arm_iommu_sync_sg_for_cpu' was not declared. Should it be static? arch/arm/mm/dma-mapping.c:1816:6: warning: symbol 'arm_iommu_sync_sg_for_device' was not declared. Should it be static? arch/arm/mm/dma-mapping.c:2018:26: warning: symbol 'iommu_ops' was not declared. Should it be static? arch/arm/mm/dma-mapping.c:2040:26: warning: symbol 'iommu_coherent_ops' was not declared. Should it be static? Signed-off-by: Ben Dooks <ben.dooks@codethink.co.uk> Signed-off-by: Russell King <rmk+kernel@armlinux.org.uk>
* | dma-mapping: drop the dev argument to arch_sync_dma_for_*Christoph Hellwig2019-11-201-4/+4
| | | | | | | | | | | | | | | | These are pure cache maintainance routines, so drop the unused struct device argument. Signed-off-by: Christoph Hellwig <hch@lst.de> Suggested-by: Daniel Vetter <daniel.vetter@ffwll.ch>
* | dma-direct: provide mmap and get_sgtable method overridesChristoph Hellwig2019-11-111-6/+0
|/ | | | | | | | | | | | | | For dma-direct we know that the DMA address is an encoding of the physical address that we can trivially decode. Use that fact to provide implementations that do not need the arch_dma_coherent_to_pfn architecture hook. Note that we still can only support mmap of non-coherent memory only if the architecture provides a way to set an uncached bit in the page tables. This must be true for architectures that use the generic remap helpers, but other architectures can also manually select it. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Max Filippov <jcmvbkbc@gmail.com>
* xen/arm: remove xen_dma_opsChristoph Hellwig2019-09-111-1/+2
| | | | | | | | | arm and arm64 can just use xen_swiotlb_dma_ops directly like x86, no need for a pointer indirection. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Julien Grall <julien.grall@arm.com> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
* xen/arm: use dma-noncoherent.h calls for xen-swiotlb cache maintainanceChristoph Hellwig2019-09-111-7/+1
| | | | | | | | Copy the arm64 code that uses the dma-direct/swiotlb helpers for DMA on-coherent devices. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Stefano Stabellini <sstabellini@kernel.org>
* arm: remove wrappers for the generic dma remap helpersChristoph Hellwig2019-09-041-27/+5
| | | | | | Remove a few tiny wrappers around the generic dma remap code. Signed-off-by: Christoph Hellwig <hch@lst.de>
* dma-mapping: introduce a dma_common_find_pages helperChristoph Hellwig2019-09-041-6/+1
| | | | | | | | A helper to find the backing page array based on a virtual address. This also ensures we do the same vm_flags check everywhere instead of slightly different or missing ones in a few places. Signed-off-by: Christoph Hellwig <hch@lst.de>
* dma-mapping: always use VM_DMA_COHERENT for generic DMA remapChristoph Hellwig2019-09-041-6/+4
| | | | | | | | | Currently the generic dma remap allocator gets a vm_flags passed by the caller that is a little confusing. We just introduced a generic vmalloc-level flag to identify the dma coherent allocations, so use that everywhere and remove the now pointless argument. Signed-off-by: Christoph Hellwig <hch@lst.de>
* vmalloc: lift the arm flag for coherent mappings to common codeChristoph Hellwig2019-09-041-15/+7
| | | | | | | | | | | The arm architecture had a VM_ARM_DMA_CONSISTENT flag to mark DMA coherent remapping for a while. Lift this flag to common code so that we can use it generically. We also check it in the only place VM_USERMAP is directly check so that we can entirely replace that flag as well (although I'm not even sure why we'd want to allow remapping DMA appings, but I'd rather not change behavior). Signed-off-by: Christoph Hellwig <hch@lst.de>
generated by cgit v1.2.3 (git 2.39.1) at 2025-01-19 14:09:50 +0100