diff options
| author | Ming Lei <ming.lei@redhat.com> | 2021-01-22 03:33:17 +0100 |
|---|---|---|
| committer | Martin K. Petersen <martin.petersen@oracle.com> | 2021-03-04 23:37:00 +0100 |
| commit | 020b0f0a31920e5b7e7e120d4560453b67b70733 (patch) | |
| tree | f86878d9930a3c58f62eda4008628fc2fa7dbcaa /drivers/scsi/scsi_priv.h | |
| parent | scsi: core: Make sure sdev->queue_depth is <= max(shost->can_queue, 1024) (diff) | |
| download | linux-020b0f0a31920e5b7e7e120d4560453b67b70733.tar.xz linux-020b0f0a31920e5b7e7e120d4560453b67b70733.zip | |
scsi: core: Replace sdev->device_busy with sbitmap
SCSI currently uses an atomic variable to track queue depth for each
attached device. The queue depth depends on many factors such as transport
type and device implementation. In addition, the SCSI device queue depth is
not a static entity but changes over time as a result of congestion
management.
While blk-mq currently tracks queue depth for each hctx, it can't easily be
changed to accommodate the SCSI per-device requirement.
The current approach of using an atomic variable doesn't scale well when
there are lots of CPU cores and the disk is very fast. IOPS can be
substantially impacted by the atomic in the hot path.
Replace the atomic variable sdev->device_busy with an sbitmap for tracking
the SCSI device queue depth.
It has been observed that IOPS is improved ~30% by this patchset in the
following test:
1) test machine(32 logical CPU cores)
Thread(s) per core: 2
Core(s) per socket: 8
Socket(s): 2
NUMA node(s): 2
Model name: Intel(R) Xeon(R) Silver 4110 CPU @ 2.10GHz
2) setup scsi_debug:
modprobe scsi_debug virtual_gb=128 max_luns=1 submit_queues=32 delay=0 max_queue=256
3) fio script:
fio --rw=randread --size=128G --direct=1 --ioengine=libaio --iodepth=2048 \
--numjobs=32 --bs=4k --group_reporting=1 --group_reporting=1 --runtime=60 \
--loops=10000 --name=job1 --filename=/dev/sdN
[mkp: fix device_busy reference in mpt3sas]
Link: https://lore.kernel.org/r/20210122023317.687987-14-ming.lei@redhat.com
Link: https://lore.kernel.org/linux-block/20200119071432.18558-6-ming.lei@redhat.com/
Cc: Omar Sandoval <osandov@fb.com>
Cc: Kashyap Desai <kashyap.desai@broadcom.com>
Cc: Sumanesh Samanta <sumanesh.samanta@broadcom.com>
Cc: Ewan D. Milne <emilne@redhat.com>
Cc: Hannes Reinecke <hare@suse.de>
Tested-by: Sumanesh Samanta <sumanesh.samanta@broadcom.com>
Reviewed-by: Hannes Reinecke <hare@suse.de>
Signed-off-by: Ming Lei <ming.lei@redhat.com>
Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
Diffstat (limited to 'drivers/scsi/scsi_priv.h')
| -rw-r--r-- | drivers/scsi/scsi_priv.h | 3 |
1 files changed, 3 insertions, 0 deletions
diff --git a/drivers/scsi/scsi_priv.h b/drivers/scsi/scsi_priv.h index 180636d54982..30b35002d2f8 100644 --- a/drivers/scsi/scsi_priv.h +++ b/drivers/scsi/scsi_priv.h @@ -5,6 +5,7 @@ #include <linux/device.h> #include <linux/async.h> #include <scsi/scsi_device.h> +#include <linux/sbitmap.h> struct request_queue; struct request; @@ -182,6 +183,8 @@ static inline void scsi_dh_add_device(struct scsi_device *sdev) { } static inline void scsi_dh_release_device(struct scsi_device *sdev) { } #endif +extern int scsi_device_max_queue_depth(struct scsi_device *sdev); + /* * internal scsi timeout functions: for use by mid-layer and transport * classes. |