diff options
| author | Dave Jiang <dave.jiang@intel.com> | 2021-11-29 18:19:38 +0100 |
|---|---|---|
| committer | Vinod Koul <vkoul@kernel.org> | 2021-12-17 17:09:28 +0100 |
| commit | 7930d85535751bc8b05c6731c6b79d874671f13c (patch) | |
| tree | 985eb1029d4206787a3f4b793ca5e5415d1b0c2c /drivers/dma/idxd/submit.c | |
| parent | dmaengine: idxd: set defaults for wq configs (diff) | |
| download | linux-7930d85535751bc8b05c6731c6b79d874671f13c.tar.xz linux-7930d85535751bc8b05c6731c6b79d874671f13c.zip | |
dmaengine: idxd: add knob for enqcmds retries
Add a sysfs knob to allow tuning of retries for the kernel ENQCMDS
descriptor submission. While on host, it is not as likely that ENQCMDS
return busy during normal operations due to the driver controlling the
number of descriptors allocated for submission. However, when the driver is
operating as a guest driver, the chance of retry goes up significantly due
to sharing a wq with multiple VMs. A default value is provided with the
system admin being able to tune the value on a per WQ basis.
Suggested-by: Sanjay Kumar <sanjay.k.kumar@intel.com>
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
Link: https://lore.kernel.org/r/163820629464.2702134.7577370098568297574.stgit@djiang5-desk3.ch.intel.com
Signed-off-by: Vinod Koul <vkoul@kernel.org>
Diffstat (limited to 'drivers/dma/idxd/submit.c')
| -rw-r--r-- | drivers/dma/idxd/submit.c | 31 |
1 files changed, 24 insertions, 7 deletions
diff --git a/drivers/dma/idxd/submit.c b/drivers/dma/idxd/submit.c index 776fa81db61d..569815a84e95 100644 --- a/drivers/dma/idxd/submit.c +++ b/drivers/dma/idxd/submit.c @@ -123,6 +123,29 @@ static void llist_abort_desc(struct idxd_wq *wq, struct idxd_irq_entry *ie, idxd_dma_complete_txd(found, IDXD_COMPLETE_ABORT, false); } +/* + * ENQCMDS typically fail when the WQ is inactive or busy. On host submission, the driver + * has better control of number of descriptors being submitted to a shared wq by limiting + * the number of driver allocated descriptors to the wq size. However, when the swq is + * exported to a guest kernel, it may be shared with multiple guest kernels. This means + * the likelihood of getting busy returned on the swq when submitting goes significantly up. + * Having a tunable retry mechanism allows the driver to keep trying for a bit before giving + * up. The sysfs knob can be tuned by the system administrator. + */ +int idxd_enqcmds(struct idxd_wq *wq, void __iomem *portal, const void *desc) +{ + int rc, retries = 0; + + do { + rc = enqcmds(portal, desc); + if (rc == 0) + break; + cpu_relax(); + } while (retries++ < wq->enqcmds_retries); + + return rc; +} + int idxd_submit_desc(struct idxd_wq *wq, struct idxd_desc *desc) { struct idxd_device *idxd = wq->idxd; @@ -166,13 +189,7 @@ int idxd_submit_desc(struct idxd_wq *wq, struct idxd_desc *desc) if (wq_dedicated(wq)) { iosubmit_cmds512(portal, desc->hw, 1); } else { - /* - * It's not likely that we would receive queue full rejection - * since the descriptor allocation gates at wq size. If we - * receive a -EAGAIN, that means something went wrong such as the - * device is not accepting descriptor at all. - */ - rc = enqcmds(portal, desc->hw); + rc = idxd_enqcmds(wq, portal, desc->hw); if (rc < 0) { percpu_ref_put(&wq->wq_active); /* abort operation frees the descriptor */ |