summaryrefslogtreecommitdiffstats
path: root/drivers/net/ethernet/amazon (follow)
Commit message (Collapse)AuthorAgeFilesLines
* net: ena: introduce ndo_xdp_xmit() function for XDP_REDIRECTShay Agroskin2020-12-102-12/+72
| | | | | | | | | | | | | | This patch implements the ndo_xdp_xmit() net_device function which is called when a packet is redirected to this driver using an XDP_REDIRECT directive. The function receives an array of xdp frames that it needs to xmit. The TX queues that are used to xmit these frames are the XDP queues used by the XDP_TX flow. Therefore a lock is added to synchronize both flows (XDP_TX and XDP_REDIRECT). Signed-off-by: Shay Agroskin <shayagr@amazon.com> Signed-off-by: Jakub Kicinski <kuba@kernel.org>
* net: ena: use xdp_return_frame() to free xdp framesShay Agroskin2020-12-102-8/+1
| | | | | | | | | | | | | XDP subsystem has a function to free XDP frames and their associated pages. Using this function would help the driver's XDP implementation to adjust to new changes in the XDP subsystem in the kernel (e.g. introduction of XDP MB). Also, remove 'xdp_rx_page' field from ena_tx_buffer struct since it is no longer used. Signed-off-by: Shay Agroskin <shayagr@amazon.com> Signed-off-by: Jakub Kicinski <kuba@kernel.org>
* net: ena: introduce XDP redirect implementationShay Agroskin2020-12-103-32/+53
| | | | | | | | | | | | | | | | | | | | This patch adds a partial support for the XDP_REDIRECT directive which instructs the driver to pass the packet to an interface specified by the program. The directive is passed to the driver by calling bpf_redirect() or bpf_redirect_map() functions from the eBPF program. To lay the ground for integration with the existing XDP TX implementation the patch removes the redundant page ref count increase in ena_xdp_xmit_frame() and then decrease in ena_clean_rx_irq(). Instead it only DMA unmaps descriptors for which XDP TX or REDIRECT directive was received. The XDP Redirect support is still missing .ndo_xdp_xmit function implementation, which allows to redirect packet to an ENA interface, which would be added in a later patch. Signed-off-by: Shay Agroskin <shayagr@amazon.com> Signed-off-by: Jakub Kicinski <kuba@kernel.org>
* net: ena: use xdp_frame in XDP TX flowShay Agroskin2020-12-101-24/+29
| | | | | | | | | | Rename the ena_xdp_xmit_buff() function to ena_xdp_xmit_frame() and pass it an xdp_frame struct instead of xdp_buff. This change lays the ground for XDP redirect implementation which uses xdp_frames when 'xmit'ing packets. Signed-off-by: Shay Agroskin <shayagr@amazon.com> Signed-off-by: Jakub Kicinski <kuba@kernel.org>
* net: ena: aggregate stats increase into a functionShay Agroskin2020-12-101-99/+68
| | | | | | | | | | | | | | | | | | | Introduce ena_increase_stat() function to increase statistics by a certain number. The function includes the - lock aquire (on 32bit machines) - stat increase - lock release (on 32bit machines) line sequence that is ubiquitous across the driver. The function increases a single stat at a time and several stats which are increased together weren't put into a function to avoid calling the function several times for each stat which looks bad and might decrease performance. Signed-off-by: Shay Agroskin <shayagr@amazon.com> Signed-off-by: Jakub Kicinski <kuba@kernel.org>
* net: ena: fix coding style nitsShay Agroskin2020-12-103-5/+6
| | | | | | | | | | | | | | | | This commit fixes two nits, but it does not generate any change to binary because of the optimization of gcc. - use `count` instead of `channels->combined_count` - change return type from `int` to `bool` Also add spaces and change macro order in OR assignment to make the code easier to read. Signed-off-by: Sameeh Jubran <sameehj@amazon.com> Signed-off-by: Kuniyuki Iwashima <kuniyu@amazon.co.jp> Signed-off-by: Shay Agroskin <shayagr@amazon.com> Signed-off-by: Jakub Kicinski <kuba@kernel.org>
* net: ena: store values in their appropriate variables typesShay Agroskin2020-12-102-9/+8
| | | | | | | | | | | This patch changes some of the variables types to match the values they hold. These wrong types fail some of our static checkers that search for accidental conversions in our driver. Signed-off-by: Ido Segev <idose@amazon.com> Signed-off-by: Igor Chauskin <igorch@amazon.com> Signed-off-by: Shay Agroskin <shayagr@amazon.com> Signed-off-by: Jakub Kicinski <kuba@kernel.org>
* net: ena: add device distinct log prefix to filesShay Agroskin2020-12-105-184/+312
| | | | | | | | | | | | | | | | | | | | | ENA logs are adjusted to display the full ENA representation to distinct each ENA device in case of multiple interfaces. Using netdev_err/warn and dev_info functions for logging provides uniform printing with clear distinction of the device and interface. This patch changes all printing in ena_com files to use netev_* logging functions except for messages of info level. Log functions of that level would be printed with dev_info because of the early stage they are called in when net_device struct isn't yet registered. To allow using netdev_* functions in all ena_com functions, a pointer to the net_device was added to ena_com_dev struct. The patch also adds some log messages to make driver debugging easier. Signed-off-by: Amit Bernstein <amitbern@amazon.com> Signed-off-by: Shay Agroskin <shayagr@amazon.com> Signed-off-by: Jakub Kicinski <kuba@kernel.org>
* net: ena: use constant value for net_device allocationShay Agroskin2020-12-101-23/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | The patch changes the maximum number of RX/TX queues it advertises to the kernel (via alloc_etherdev_mq()) from a value received from the device to a constant value which is the minimum between 128 and the number of CPUs in the system. By allocating the net_device struct with a constant number of queues, the driver is able to allocate it at a much earlier stage, before calling any ena_com functions. This would allow to make all log prints in ena_com to use netdev_* log functions instead or current pr_* ones. Note: netdev_* prints in ena_com functions that are called before net_device registration in ena_probe() might print messages that are a bit ugly (with strings like "(unnamed net_device) (uninitialized)"). However we decided to use netdev_* prints in these functions anyway, for the sake of getting better messages later, when ena_com functions are called after ena_probe() form other parts of the driver. See discussion about this decision in [1]. [1] http://www.mail-archive.com/netdev@vger.kernel.org/msg353590.html Signed-off-by: Shay Agroskin <shayagr@amazon.com> Signed-off-by: Jakub Kicinski <kuba@kernel.org>
* Merge https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-nextJakub Kicinski2020-12-041-1/+1
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Alexei Starovoitov says: ==================== pull-request: bpf-next 2020-12-03 The main changes are: 1) Support BTF in kernel modules, from Andrii. 2) Introduce preferred busy-polling, from Björn. 3) bpf_ima_inode_hash() and bpf_bprm_opts_set() helpers, from KP Singh. 4) Memcg-based memory accounting for bpf objects, from Roman. 5) Allow bpf_{s,g}etsockopt from cgroup bind{4,6} hooks, from Stanislav. * https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next: (118 commits) selftests/bpf: Fix invalid use of strncat in test_sockmap libbpf: Use memcpy instead of strncpy to please GCC selftests/bpf: Add fentry/fexit/fmod_ret selftest for kernel module selftests/bpf: Add tp_btf CO-RE reloc test for modules libbpf: Support attachment of BPF tracing programs to kernel modules libbpf: Factor out low-level BPF program loading helper bpf: Allow to specify kernel module BTFs when attaching BPF programs bpf: Remove hard-coded btf_vmlinux assumption from BPF verifier selftests/bpf: Add CO-RE relocs selftest relying on kernel module BTF selftests/bpf: Add support for marking sub-tests as skipped selftests/bpf: Add bpf_testmod kernel module for testing libbpf: Add kernel module BTF support for CO-RE relocations libbpf: Refactor CO-RE relocs to not assume a single BTF object libbpf: Add internal helper to load BTF data by FD bpf: Keep module's btf_data_size intact after load bpf: Fix bpf_put_raw_tracepoint()'s use of __module_address() selftests/bpf: Add Userspace tests for TCP_WINDOW_CLAMP bpf: Adds support for setting window clamp samples/bpf: Fix spelling mistake "recieving" -> "receiving" bpf: Fix cold build of test_progs-no_alu32 ... ==================== Link: https://lore.kernel.org/r/20201204021936.85653-1-alexei.starovoitov@gmail.com Signed-off-by: Jakub Kicinski <kuba@kernel.org>
| * xsk: Propagate napi_id to XDP socket Rx pathBjörn Töpel2020-12-011-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | Add napi_id to the xdp_rxq_info structure, and make sure the XDP socket pick up the napi_id in the Rx path. The napi_id is used to find the corresponding NAPI structure for socket busy polling. Signed-off-by: Björn Töpel <bjorn.topel@intel.com> Signed-off-by: Daniel Borkmann <daniel@iogearbox.net> Acked-by: Ilias Apalodimas <ilias.apalodimas@linaro.org> Acked-by: Michael S. Tsirkin <mst@redhat.com> Acked-by: Tariq Toukan <tariqt@nvidia.com> Link: https://lore.kernel.org/bpf/20201130185205.196029-7-bjorn.topel@gmail.com
* | Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/netJakub Kicinski2020-11-282-50/+33
|\ \ | | | | | | | | | | | | | | | | | | | | | | | | Trivial conflict in CAN, keep the net-next + the byteswap wrapper. Conflicts: drivers/net/can/usb/gs_usb.c Signed-off-by: Jakub Kicinski <kuba@kernel.org>
| * | net: ena: fix packet's addresses for rx_offset featureShay Agroskin2020-11-251-9/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch fixes two lines in which the rx_offset received by the device wasn't taken into account: - prefetch function: In our driver the copied data would reside in rx_info->page + rx_headroom + rx_offset so the prefetch function is changed accordingly. - setting page_offset to zero for descriptors > 1: for every descriptor but the first, the rx_offset is zero. Hence the page_offset value should be set to rx_headroom. The previous implementation changed the value of rx_info after the descriptor was added to the SKB (essentially providing wrong page offset). Fixes: 68f236df93a9 ("net: ena: add support for the rx offset feature") Signed-off-by: Shay Agroskin <shayagr@amazon.com> Signed-off-by: Jakub Kicinski <kuba@kernel.org>
| * | net: ena: set initial DMA width to avoid intel iommu issueShay Agroskin2020-11-251-9/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The ENA driver uses the readless mechanism, which uses DMA, to find out what the DMA mask is supposed to be. If DMA is used without setting the dma_mask first, it causes the Intel IOMMU driver to think that ENA is a 32-bit device and therefore disables IOMMU passthrough permanently. This patch sets the dma_mask to be ENA_MAX_PHYS_ADDR_SIZE_BITS=48 before readless initialization in ena_device_init()->ena_com_mmio_reg_read_request_init(), which is large enough to workaround the intel_iommu issue. DMA mask is set again to the correct value after it's received from the device after readless is initialized. The patch also changes the driver to use dma_set_mask_and_coherent() function instead of the two pci_set_dma_mask() and pci_set_consistent_dma_mask() ones. Both methods achieve the same effect. Fixes: 1738cd3ed342 ("net: ena: Add a driver for Amazon Elastic Network Adapters (ENA)") Signed-off-by: Mike Cui <mikecui@amazon.com> Signed-off-by: Arthur Kiyanovski <akiyano@amazon.com> Signed-off-by: Shay Agroskin <shayagr@amazon.com> Signed-off-by: Jakub Kicinski <kuba@kernel.org>
| * | net: ena: handle bad request id in ena_netdevShay Agroskin2020-11-252-32/+14
| |/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | After request id is checked in validate_rx_req_id() its value is still used in the line rx_ring->free_ids[next_to_clean] = rx_ring->ena_bufs[i].req_id; even if it was found to be out-of-bound for the array free_ids. The patch moves the request id to an earlier stage in the napi routine and makes sure its value isn't used if it's found out-of-bounds. Fixes: 30623e1ed116 ("net: ena: avoid memory access violation by validating req_id properly") Signed-off-by: Ido Segev <idose@amazon.com> Signed-off-by: Shay Agroskin <shayagr@amazon.com> Signed-off-by: Jakub Kicinski <kuba@kernel.org>
* / net: don't include ethtool.h from netdevice.hJakub Kicinski2020-11-241-0/+1
|/ | | | | | | | | | | | | | | | linux/netdevice.h is included in very many places, touching any of its dependecies causes large incremental builds. Drop the linux/ethtool.h include, linux/netdevice.h just needs a forward declaration of struct ethtool_ops. Fix all the places which made use of this implicit include. Acked-by: Johannes Berg <johannes@sipsolutions.net> Acked-by: Shannon Nelson <snelson@pensando.io> Reviewed-by: Jesse Brandeburg <jesse.brandeburg@intel.com> Link: https://lore.kernel.org/r/20201120225052.1427503-1-kuba@kernel.org Signed-off-by: Jakub Kicinski <kuba@kernel.org>
* drivers/net/ethernet: remove incorrectly formatted docJesse Brandeburg2020-09-261-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | As part of the W=1 series for ethernet, these drivers were discovered to be using kdoc style comments but were not actually doing kdoc. The kernel uses kdoc style when documenting code, not doxygen or other styles. Fixed Warnings: drivers/net/ethernet/amazon/ena/ena_com.c:613: warning: Function parameter or member 'ena_dev' not described in 'ena_com_set_llq' drivers/net/ethernet/aquantia/atlantic/hw_atl/hw_atl_b0.c:1540: warning: Cannot understand * @brief Set VLAN filter table drivers/net/ethernet/xilinx/ll_temac_main.c:114: warning: Function parameter or member 'lp' not described in 'temac_indirect_busywait' drivers/net/ethernet/xilinx/ll_temac_main.c:129: warning: Function parameter or member 'lp' not described in 'temac_indirect_in32' drivers/net/ethernet/xilinx/ll_temac_main.c:129: warning: Function parameter or member 'reg' not described in 'temac_indirect_in32' drivers/net/ethernet/xilinx/ll_temac_main.c:147: warning: Function parameter or member 'lp' not described in 'temac_indirect_in32_locked' drivers/net/ethernet/xilinx/ll_temac_main.c:147: warning: Function parameter or member 'reg' not described in 'temac_indirect_in32_locked' drivers/net/ethernet/xilinx/ll_temac_main.c:172: warning: Function parameter or member 'lp' not described in 'temac_indirect_out32' drivers/net/ethernet/xilinx/ll_temac_main.c:172: warning: Function parameter or member 'reg' not described in 'temac_indirect_out32' drivers/net/ethernet/xilinx/ll_temac_main.c:172: warning: Function parameter or member 'value' not described in 'temac_indirect_out32' drivers/net/ethernet/xilinx/ll_temac_main.c:188: warning: Function parameter or member 'lp' not described in 'temac_indirect_out32_locked' drivers/net/ethernet/xilinx/ll_temac_main.c:188: warning: Function parameter or member 'reg' not described in 'temac_indirect_out32_locked' drivers/net/ethernet/xilinx/ll_temac_main.c:188: warning: Function parameter or member 'value' not described in 'temac_indirect_out32_locked' drivers/net/ethernet/xilinx/ll_temac_main.c:212: warning: Function parameter or member 'lp' not described in 'temac_dma_in32_be' drivers/net/ethernet/xilinx/ll_temac_main.c:212: warning: Function parameter or member 'reg' not described in 'temac_dma_in32_be' drivers/net/ethernet/xilinx/ll_temac_main.c:228: warning: Function parameter or member 'lp' not described in 'temac_dma_out32_be' drivers/net/ethernet/xilinx/ll_temac_main.c:228: warning: Function parameter or member 'reg' not described in 'temac_dma_out32_be' drivers/net/ethernet/xilinx/ll_temac_main.c:228: warning: Function parameter or member 'value' not described in 'temac_dma_out32_be' drivers/net/ethernet/xilinx/ll_temac_main.c:247: warning: Function parameter or member 'lp' not described in 'temac_dma_dcr_in' drivers/net/ethernet/xilinx/ll_temac_main.c:247: warning: Function parameter or member 'reg' not described in 'temac_dma_dcr_in' drivers/net/ethernet/xilinx/ll_temac_main.c:255: warning: Function parameter or member 'lp' not described in 'temac_dma_dcr_out' drivers/net/ethernet/xilinx/ll_temac_main.c:255: warning: Function parameter or member 'reg' not described in 'temac_dma_dcr_out' drivers/net/ethernet/xilinx/ll_temac_main.c:255: warning: Function parameter or member 'value' not described in 'temac_dma_dcr_out' drivers/net/ethernet/xilinx/ll_temac_main.c:265: warning: Function parameter or member 'lp' not described in 'temac_dcr_setup' drivers/net/ethernet/xilinx/ll_temac_main.c:265: warning: Function parameter or member 'op' not described in 'temac_dcr_setup' drivers/net/ethernet/xilinx/ll_temac_main.c:265: warning: Function parameter or member 'np' not described in 'temac_dcr_setup' drivers/net/ethernet/xilinx/ll_temac_main.c:300: warning: Function parameter or member 'ndev' not described in 'temac_dma_bd_release' drivers/net/ethernet/xilinx/ll_temac_main.c:330: warning: Function parameter or member 'ndev' not described in 'temac_dma_bd_init' drivers/net/ethernet/xilinx/ll_temac_main.c:600: warning: Function parameter or member 'ndev' not described in 'temac_setoptions' drivers/net/ethernet/xilinx/ll_temac_main.c:600: warning: Function parameter or member 'options' not described in 'temac_setoptions' Signed-off-by: Jesse Brandeburg <jesse.brandeburg@intel.com> Signed-off-by: David S. Miller <davem@davemloft.net>
* net: ena: Fix all static chekers' warningsShay Agroskin2020-09-213-8/+13
| | | | | | | | | | | | | | | | | | | | After running Sparse checker on the driver using make C=1 M=drivers/net/ethernet/amazon/ena the only error that is thrown is: sparse: sparse: Using plain integer as NULL pointer about the line struct ena_calc_queue_size_ctx calc_queue_ctx = { 0 }; This patch fixes this warning, thus making our driver free (for now) of Sparse errors/warnings. To make a more complete work, this patch also fixes all static warnings that were found using an internal static checker. Signed-off-by: Ido Segev <idose@amazon.com> Signed-off-by: Shay Agroskin <shayagr@amazon.com> Signed-off-by: David S. Miller <davem@davemloft.net>
* net: ena: Change RSS related macros and variables namesShay Agroskin2020-09-212-14/+15
| | | | | | | | | | | | | | | | The formal name changes to "ENA_ADMIN_RSS_INDIRECTION_TABLE_CONFIG". Indirection is the ability to reference "something" using "something else" instead of the value itself. Indirection table, as the name implies, is the ability to reference CPU/Queue value using hash-to-CPU table instead of CPU/Queue itself. This patch renames the variable keys_num, which describes the number of words in the RSS hash key, to key_parts which makes its purpose clearer in RSS context. Signed-off-by: Amit Bernstein <amitbern@amazon.com> Signed-off-by: Shay Agroskin <shayagr@amazon.com> Signed-off-by: David S. Miller <davem@davemloft.net>
* net: ena: Remove redundant print of placement policyShay Agroskin2020-09-211-8/+2
| | | | | | | | | The placement policy is printed in the process of queue creation in ena_up(). No need to print it in ena_probe(). Signed-off-by: Arthur Kiyanovski <akiyano@amazon.com> Signed-off-by: Shay Agroskin <shayagr@amazon.com> Signed-off-by: David S. Miller <davem@davemloft.net>
* net: ena: Capitalize all log strings and improve code readabilityShay Agroskin2020-09-217-155/+179
| | | | | | | | | | | | | | | Capitalize all log strings printed by the ena driver to make their format uniform across it. Also fix indentation, spelling mistakes and comments to improve code readability. This also includes adding comments to macros/enums whose purpose might be difficult to understand. Separate some code into functions to make it easier to understand the purpose of these lines. Signed-off-by: Amit Bernstein <amitbern@amazon.com> Signed-off-by: Shay Agroskin <shayagr@amazon.com> Signed-off-by: David S. Miller <davem@davemloft.net>
* net: ena: Change log message to netif/dev functionShay Agroskin2020-09-211-19/+23
| | | | | | | | | | | | | | | | | Make log prints in ena_netdev use the same log functions as the rest of the driver. For the sake of consistency, all prints in ena_netdev file were converted into netif_* format except where netdev struct isn't yet defined. For these places, dev_* log functions are used (similar to the patch for ena_com files). This commit leaves some corner cases which would be changed in a future patch. Signed-off-by: Amit Bernstein <amitbern@amazon.com> Signed-off-by: Shay Agroskin <shayagr@amazon.com> Signed-off-by: David S. Miller <davem@davemloft.net>
* net: ena: Change license into format to SPDX in all filesShay Agroskin2020-09-2112-348/+24
| | | | | | | | | | | All ena files should now use SPDX format in their license string. This doesn't change the license of the files, but rather states the same license in fewer words. Also update the license years in some of the files. Signed-off-by: Shay Agroskin <shayagr@amazon.com> Signed-off-by: David S. Miller <davem@davemloft.net>
* net: ena: xdp: add queue counters for xdp actionsSameeh Jubran2020-09-113-3/+28
| | | | | | | | | | | | | When using XDP every ingress packet is passed to an eBPF (xdp) program which returns an action for this packet. This patch adds counters for the number of times each such action was received. It also counts all the invalid actions received from the eBPF program. Signed-off-by: Shay Agroskin <shayagr@amazon.com> Signed-off-by: Sameeh Jubran <sameehj@amazon.com> Signed-off-by: David S. Miller <davem@davemloft.net>
* net: ena: ethtool: add stats printing to XDP queuesSameeh Jubran2020-09-111-17/+28
| | | | | | | | | | | | Added statistics for TX queues that are used for XDP TX. The statistics are the same as the ones printed for regular non-XDP TX queues. The XDP queue statistics can be queried using `ethtool -S <ifname>` Signed-off-by: Shay Agroskin <shayagr@amazon.com> Signed-off-by: Sameeh Jubran <sameehj@amazon.com> Signed-off-by: David S. Miller <davem@davemloft.net>
* net: ena: ethtool: Add new device statisticsSameeh Jubran2020-09-116-23/+170
| | | | | | | | | | | | The new metrics provide granular visibility along multiple network dimensions and enable troubleshooting and remediation of issues caused by instances exceeding network performance allowances. The new statistics can be queried using ethtool command. Signed-off-by: Guy Tzalik <gtzalik@amazon.com> Signed-off-by: Sameeh Jubran <sameehj@amazon.com> Signed-off-by: David S. Miller <davem@davemloft.net>
* net: ena: ethtool: convert stat_offset to 64 bit resolutionSameeh Jubran2020-09-111-10/+8
| | | | | | | | | | The type of all stat fields is u64, therefore when iterating over stat fields in a stats struct, it makes sense to use an offset in 64 bit resolution. Doing so allows us to drop some of the casting that is currently used when referencing stats. Signed-off-by: Sameeh Jubran <sameehj@amazon.com> Signed-off-by: David S. Miller <davem@davemloft.net>
* net: ena: Make missed_tx stat incrementalShay Agroskin2020-08-201-1/+4
| | | | | | | | | | | | | | | Most statistics in ena driver are incremented, meaning that a stat's value is a sum of all increases done to it since driver/queue initialization. This patch makes all statistics this way, effectively making missed_tx statistic incremental. Also added a comment regarding rx_drops and tx_drops to make it clearer how these counters are calculated. Fixes: 11095fdb712b ("net: ena: add statistics for missed tx packets") Signed-off-by: Shay Agroskin <shayagr@amazon.com> Signed-off-by: David S. Miller <davem@davemloft.net>
* net: ena: Change WARN_ON expression in ena_del_napi_in_range()Shay Agroskin2020-08-201-7/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | The ena_del_napi_in_range() function unregisters the napi handler for rings in a given range. This function had the following WARN_ON macro: WARN_ON(ENA_IS_XDP_INDEX(adapter, i) && adapter->ena_napi[i].xdp_ring); This macro prints the call stack if the expression inside of it is true [1], but the expression inside of it is the wanted situation. The expression checks whether the ring has an XDP queue and its index corresponds to a XDP one. This patch changes the expression to !ENA_IS_XDP_INDEX(adapter, i) && adapter->ena_napi[i].xdp_ring which indicates an unwanted situation. Also, change the structure of the function. The napi handler is unregistered for all rings, and so there's no need to check whether the index is an XDP index or not. By removing this check the code becomes much more readable. Fixes: 548c4940b9f1 ("net: ena: Implement XDP_TX action") Signed-off-by: Shay Agroskin <shayagr@amazon.com> Signed-off-by: David S. Miller <davem@davemloft.net>
* net: ena: Prevent reset after device destructionShay Agroskin2020-08-201-9/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The reset work is scheduled by the timer routine whenever it detects that a device reset is required (e.g. when a keep_alive signal is missing). When releasing device resources in ena_destroy_device() the driver cancels the scheduling of the timer routine without destroying the reset work explicitly. This creates the following bug: The driver is suspended and the ena_suspend() function is called -> This function calls ena_destroy_device() to free the net device resources -> The driver waits for the timer routine to finish its execution and then cancels it, thus preventing from it to be called again. If, in its final execution, the timer routine schedules a reset, the reset routine might be called afterwards,and a redundant call to ena_restore_device() would be made. By changing the reset routine we allow it to read the device's state accurately. This is achieved by checking whether ENA_FLAG_TRIGGER_RESET flag is set before resetting the device and making both the destruction function and the flag check are under rtnl lock. The ENA_FLAG_TRIGGER_RESET is cleared at the end of the destruction routine. Also surround the flag check with 'likely' because we expect that the reset routine would be called only when ENA_FLAG_TRIGGER_RESET flag is set. The destruction of the timer and reset services in __ena_shutoff() have to stay, even though the timer routine is destroyed in ena_destroy_device(). This is to avoid a case in which the reset routine is scheduled after free_netdev() in __ena_shutoff(), which would create an access to freed memory in adapter->flags. Fixes: 8c5c7abdeb2d ("net: ena: add power management ops to the ENA driver") Signed-off-by: Shay Agroskin <shayagr@amazon.com> Signed-off-by: David S. Miller <davem@davemloft.net>
* bpf, xdp: Remove XDP_QUERY_PROG and XDP_QUERY_PROG_HW XDP commandsAndrii Nakryiko2020-07-261-6/+0
| | | | | | | | | | | | | | | Now that BPF program/link management is centralized in generic net_device code, kernel code never queries program id from drivers, so XDP_QUERY_PROG/XDP_QUERY_PROG_HW commands are unnecessary. This patch removes all the implementations of those commands in kernel, along the xdp_attachment_query(). This patch was compile-tested on allyesconfig. Signed-off-by: Andrii Nakryiko <andriin@fb.com> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Link: https://lore.kernel.org/bpf/20200722064603.3350758-10-andriin@fb.com
* net: ena: support new LLQ acceleration modeArthur Kiyanovski2020-07-227-24/+109
| | | | | | | | | | | | | | | | | | | | | | | | | | New devices add a new hardware acceleration engine, which adds some restrictions to the driver. Metadata descriptor must be present for each packet and the maximum burst size between two doorbells is now limited to a number advertised by the device. This patch adds: 1. A handshake protocol between the driver and the device, so the device will enable the accelerated queues only when both sides support it. 2. The driver support for the new acceleration engine: 2.1. Send metadata descriptor for each Tx packet. 2.2. Limit the number of packets sent between doorbells.(*) (*) A previous driver implementation of this feature was comitted in commit 05d62ca218f8 ("net: ena: add handling of llq max tx burst size") however the design of the interface between the driver and device changed since then. This change is reflected in this commit. Signed-off-by: Netanel Belgazal <netanel@amazon.com> Signed-off-by: Arthur Kiyanovski <akiyano@amazon.com> Signed-off-by: David S. Miller <davem@davemloft.net>
* net: ena: move llq configuration from ena_probe to ena_device_init()Arthur Kiyanovski2020-07-221-63/+73
| | | | | | | | | | | | | | | | | | | | | | | | | | | When the ENA device resets to recover from some error state, all LLQ configuration values are reset to their defaults, because LLQ is initialized only once during ena_probe(). Changes in this commit: 1. Move the LLQ configuration process into ena_init_device() which is called from both ena_probe() and ena_restore_device(). This way, LLQ setup configurations that are different from the default values will survive resets. 2. Extract the LLQ bar mapping to ena_map_llq_bar(), and call once in the lifetime of the driver from ena_probe(), since there is no need to unmap and map the LLQ bar again every reset. 3. Map the LLQ bar if it exists, regardless if initialization of LLQ placement policy (ENA_ADMIN_PLACEMENT_POLICY_DEV) succeeded or not. Initialization might fail the first time, falling back to the ENA_ADMIN_PLACEMENT_POLICY_HOST placement policy, but later succeed after device reset, in which case the LLQ bar needs to be mapped already. Signed-off-by: Sameeh Jubran <sameehj@amazon.com> Signed-off-by: Arthur Kiyanovski <akiyano@amazon.com> Signed-off-by: David S. Miller <davem@davemloft.net>
* net: ena: enable support of rss hash key and function changesArthur Kiyanovski2020-07-222-2/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Add the rss_configurable_function_key bit to driver_supported_feature. This bit tells the device that the driver in question supports the retrieving and updating of RSS function and hash key, and therefore the device should allow RSS function and key manipulation. This commit turns on device support for hash key and RSS function management. Without this commit this feature is turned off at the device and appears to the user as unsupported. This commit concludes the following series of already merged commits: commit 0af3c4e2eab8 ("net: ena: changes to RSS hash key allocation") commit c1bd17e51c71 ("net: ena: change default RSS hash function to Toeplitz") commit f66c2ea3b18a ("net: ena: allow setting the hash function without changing the key") commit e9a1de378dd4 ("net: ena: fix error returning in ena_com_get_hash_function()") commit 80f8443fcdaa ("net: ena: avoid unnecessary admin command when RSS function set fails") commit 6a4f7dc82d1e ("net: ena: rss: do not allocate key when not supported") commit 0d1c3de7b8c7 ("net: ena: fix incorrect default RSS key") The above commits represent the last part of the implementation of this feature, and with them merged the feature can be enabled in the device. Signed-off-by: Arthur Kiyanovski <akiyano@amazon.com> Signed-off-by: David S. Miller <davem@davemloft.net>
* net: ena: add support for traffic mirroringArthur Kiyanovski2020-07-222-7/+13
| | | | | | | | | | | | | | | | | | Add support for traffic mirroring, where the hardware reads the buffer from the instance memory directly. Traffic Mirroring needs access to the rx buffers in the instance. To have this access, this patch: 1. Changes the code to map and unmap the rx buffers bidirectionally. 2. Enables the relevant bit in driver_supported_features to indicate to the FW that this driver supports traffic mirroring. Rx completion is not generated until mirroring is done to avoid the situation where the driver changes the buffer before it is mirrored. Signed-off-by: Arthur Kiyanovski <akiyano@amazon.com> Signed-off-by: David S. Miller <davem@davemloft.net>
* net: ena: cosmetic: change ena_com_stats_admin stats to u64Arthur Kiyanovski2020-07-222-7/+7
| | | | | | | | | | | | | The size of the admin statistics in ena_com_stats_admin is changed from 32bit to 64bit so to align with the sizes of the other statistics in the driver (i.e. rx_stats, tx_stats and ena_stats_dev). This is done as part of an effort to create a unified API to read statistics. Signed-off-by: Shay Agroskin <shayagr@amazon.com> Signed-off-by: Arthur Kiyanovski <akiyano@amazon.com> Signed-off-by: David S. Miller <davem@davemloft.net>
* net: ena: cosmetic: satisfy gcc warningArthur Kiyanovski2020-07-221-1/+1
| | | | | | | | | | gcc 4.8 reports a warning when initializing with = {0}. Dropping the "0" from the braces fixes the issue. This fix is not ANSI compatible but is allowed by gcc. Signed-off-by: Sameeh Jubran <sameehj@amazon.com> Signed-off-by: Arthur Kiyanovski <akiyano@amazon.com> Signed-off-by: David S. Miller <davem@davemloft.net>
* net: ena: add reserved PCI device IDArthur Kiyanovski2020-07-221-0/+5
| | | | | | | | Add a reserved PCI device ID to the driver's table Used for internal testing purposes. Signed-off-by: Arthur Kiyanovski <akiyano@amazon.com> Signed-off-by: David S. Miller <davem@davemloft.net>
* net: ena: avoid unnecessary rearming of interrupt vector when busy-pollingArthur Kiyanovski2020-07-222-1/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | For an overview of the race created by this patch goto synchronization label. In napi busy-poll mode, the kernel invokes the napi handler of the device repeatedly to poll the NIC's receive queues. This process repeats until a timeout, specific for each connection, is up. By polling packets in busy-poll mode the user may gain lower latency and higher throughput (since the kernel no longer waits for interrupts to poll the queues) in expense of CPU usage. Upon completing a napi routine, the driver checks whether the routine was called by an interrupt handler. If so, the driver re-enables interrupts for the device. This is needed since an interrupt routine invocation disables future invocations until explicitly re-enabled. The driver avoids re-enabling the interrupts if they were not disabled in the first place (e.g. if driver in busy mode). Originally, the driver checked whether interrupt re-enabling is needed by reading the 'ena_napi->unmask_interrupt' variable. This atomic variable was set upon interrupt and cleared after re-enabling it. In the 4.10 Linux version, the 'napi_complete_done' call was changed so that it returns 'false' when device should not re-enable interrupts, and 'true' otherwise. The change includes reading the "NAPIF_STATE_IN_BUSY_POLL" flag to check if the napi call is in busy-poll mode, and if so, return 'false'. The driver was changed to re-enable interrupts according to this routine's return value. The Linux community rejected the use of the 'ena_napi->unmaunmask_interrupt' variable to determine whether unmasking is needed, and urged to use napi_napi_complete_done() return value solely. See https://lore.kernel.org/patchwork/patch/741149/ for more details As explained, a busy-poll session exists for a specified timeout value, after which it exits the busy-poll mode and re-enters it later. This leads to many invocations of the napi handler where napi_complete_done() false indicates that interrupts should be re-enabled. This creates a bug in which the interrupts are re-enabled unnecessarily. To reproduce this bug: 1) echo 50 | sudo tee /proc/sys/net/core/busy_poll 2) echo 50 | sudo tee /proc/sys/net/core/busy_read 3) Add counters that check whether 'ena_unmask_interrupt(tx_ring, rx_ring);' is called without disabling the interrupts in the first place (i.e. with calling the interrupt routine ena_intr_msix_io()) Steps 1+2 enable busy-poll as the default mode for new connections. The busy poll routine rearms the interrupts after every session by design, and so we need to add an extra check that the interrupts were masked in the first place. synchronization: This patch introduces a race between the interrupt handler ena_intr_msix_io() and the napi routine ena_io_poll(). Some macros and instruction were added to prevent this race from leaving the interrupts masked. The following specifies the different race scenarios in this patch: 1) interrupt handler and napi routine run sequentially i) interrupt handler is called, sets 'interrupts_masked' flag and successfully schedules the napi handler via softirq. In this scenario the napi routine might not see the flag change for several reasons: a) The flag is stored in a register by the compiler. For this case the WRITE_ONCE macro which prevents this. b) The compiler might reorder the instruction. For this the smp_wmb() instruction was used which implies a compiler memory barrier. c) On archs with weak consistency model (like ARM64) the napi routine might be scheduled and start running before the flag STORE instruction is committed to cache/memory. To ensure this doesn't happen, the smp_wmb() instruction was added. It ensures that the flag set instruction is committed before scheduling napi. ii) compiler reorders the flag's value check in the 'if' with the flag set in the napi routine. This scenario is prevented by smp_rmb() call after the flag check. 2) interrupt handler and napi routine run in parallel (can happen when busy poll routine invokes the napi handler) i) interrupt handler sets the flag in one core, while the napi routine reads it in another core. This scenario also is divided into two cases: a) napi_complete_done() doesn't finish running, in which case napi_sched() would just set NAPIF_STATE_MISSED and the napi routine would reschedule itself without changing the flag's value. b) napi_complete_done() finishes running. In this case the napi routine might override the flag's value. This doesn't present any rise since it later unmasks the interrupt vector. Signed-off-by: Shay Agroskin <shayagr@amazon.com> Signed-off-by: Arthur Kiyanovski <akiyano@amazon.com> Cc: Eric Dumazet <eric.dumazet@gmail.com> Signed-off-by: David S. Miller <davem@davemloft.net>
* net: ena: Fix using plain integer as NULL pointer in ena_init_napi_in_rangeWang Hai2020-07-211-3/+2
| | | | | | | | | | | | | | Fix sparse build warning: drivers/net/ethernet/amazon/ena/ena_netdev.c:2193:34: warning: Using plain integer as NULL pointer Reported-by: Hulk Robot <hulkci@huawei.com> Signed-off-by: Wang Hai <wanghai38@huawei.com> Suggested-by: Joe Perches <joe@perches.com> Acked-by: Shay Agroskin <shayagr@amazon.com> Acked-by: Shay Agroskin <shayagr@amazon.com> Signed-off-by: David S. Miller <davem@davemloft.net>
* ena_netdev: use generic power managementVaibhav Gupta2020-07-011-13/+9
| | | | | | | | | | | | | With legacy PM, drivers themselves were responsible for managing the device's power states and takes care of register states. After upgrading to the generic structure, PCI core will take care of required tasks and drivers should do only device-specific operations. Compile-tested only. Signed-off-by: Vaibhav Gupta <vaibhavgupta40@gmail.com> Signed-off-by: David S. Miller <davem@davemloft.net>
* treewide: replace '---help---' in Kconfig files with 'help'Masahiro Yamada2020-06-131-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | Since commit 84af7a6194e4 ("checkpatch: kconfig: prefer 'help' over '---help---'"), the number of '---help---' has been gradually decreasing, but there are still more than 2400 instances. This commit finishes the conversion. While I touched the lines, I also fixed the indentation. There are a variety of indentation styles found. a) 4 spaces + '---help---' b) 7 spaces + '---help---' c) 8 spaces + '---help---' d) 1 space + 1 tab + '---help---' e) 1 tab + '---help---' (correct indentation) f) 1 tab + 1 space + '---help---' g) 1 tab + 2 spaces + '---help---' In order to convert all of them to 1 tab + 'help', I ran the following commend: $ find . -name 'Kconfig*' | xargs sed -i 's/^[[:space:]]*---help---/\thelp/' Signed-off-by: Masahiro Yamada <masahiroy@kernel.org>
* net: ena: xdp: update napi budget for DROP and ABORTEDSameeh Jubran2020-06-051-4/+4
| | | | | | | | | | | | | | | | | This patch fixes two issues with XDP: 1. If the XDP verdict is XDP_ABORTED we break the loop, which results in us handling one buffer per napi cycle instead of the total budget (usually 64). To overcome this simply change the xdp_verdict check to != XDP_PASS. When the verdict is XDP_PASS, the skb is not expected to be NULL. 2. Update the residual budget for XDP_DROP and XDP_ABORTED, since packets are handled in these cases. Fixes: 548c4940b9f1 ("net: ena: Implement XDP_TX action") Signed-off-by: Sameeh Jubran <sameehj@amazon.com> Signed-off-by: David S. Miller <davem@davemloft.net>
* net: ena: xdp: XDP_TX: fix memory leakSameeh Jubran2020-06-051-1/+1
| | | | | | | | | | | | | When sending very high packet rate, the XDP tx queues can get full and start dropping packets. In this case we don't free the pages which results in ena driver draining the system memory. Fix: Simply free the pages when necessary. Fixes: 548c4940b9f1 ("net: ena: Implement XDP_TX action") Signed-off-by: Sameeh Jubran <sameehj@amazon.com> Signed-off-by: David S. Miller <davem@davemloft.net>
* xdp: Rename convert_to_xdp_frame in xdp_convert_buff_to_frameLorenzo Bianconi2020-06-021-1/+1
| | | | | | | | | | | In order to use standard 'xdp' prefix, rename convert_to_xdp_frame utility routine in xdp_convert_buff_to_frame and replace all the occurrences Signed-off-by: Lorenzo Bianconi <lorenzo@kernel.org> Signed-off-by: Alexei Starovoitov <ast@kernel.org> Acked-by: Jesper Dangaard Brouer <brouer@redhat.com> Link: https://lore.kernel.org/bpf/6344f739be0d1a08ab2b9607584c4d5478c8c083.1590698295.git.lorenzo@kernel.org
* net: ena: reduce driver load timeArthur Kiyanovski2020-05-224-10/+33
| | | | | | | | | | | | | This commit reduces the driver load time by using usec resolution instead of msec when polling for hardware state change. Also add back-off mechanism to handle cases where minimal sleep time is not enough. Signed-off-by: Josh Triplett <josh@joshtriplett.org> Signed-off-by: Sameeh Jubran <sameehj@amazon.com> Signed-off-by: Arthur Kiyanovski <akiyano@amazon.com> Signed-off-by: David S. Miller <davem@davemloft.net>
* net: ena: cosmetic: minor code changesArthur Kiyanovski2020-05-224-7/+7
| | | | | | | | | | 1. Use BIT macro instead of shift operator for code clarity 2. Replace multiple flag assignments to a single assignment of multiple flags in ena_com_add_single_rx_desc() 3. Move ENA_HASH_KEY_SIZE from ena_netdev.h to ena_com.h Signed-off-by: Arthur Kiyanovski <akiyano@amazon.com> Signed-off-by: David S. Miller <davem@davemloft.net>
* net: ena: cosmetic: fix spacing issuesArthur Kiyanovski2020-05-224-6/+6
| | | | | | | | | 1. Add leading and trailing spaces to several comments for better readability 2. Make tabs and spaces uniform in enum defines in ena_admin_defs.h Signed-off-by: Arthur Kiyanovski <akiyano@amazon.com> Signed-off-by: David S. Miller <davem@davemloft.net>
* net: ena: cosmetic: code reorderingsArthur Kiyanovski2020-05-222-11/+11
| | | | | | | | | | 1. Reorder sanity checks in get_comp_ctxt() to make more sense 2. Reorder variables in ena_com_fill_hash_function() and ena_calc_io_queue_size() in reverse christmas tree. 3. Move around member initializations. Signed-off-by: Arthur Kiyanovski <akiyano@amazon.com> Signed-off-by: David S. Miller <davem@davemloft.net>
* net: ena: cosmetic: remove unnecessary codeArthur Kiyanovski2020-05-222-14/+3
| | | | | | | | | | 1. Remove unused definition of DRV_MODULE_VERSION 2. Remove {} from single line-of-code ifs 3. Remove unnecessary comments from ena_get/set_coalesce() 4. Remove unnecessary extra spaces and newlines Signed-off-by: Arthur Kiyanovski <akiyano@amazon.com> Signed-off-by: David S. Miller <davem@davemloft.net>
generated by cgit v1.2.3 (git 2.39.1) at 2024-12-25 10:47:10 +0100