s390/pci: Documentation for zPCI

There are changes in the usage of PCI for the user:
 - new kernel parameter
 - modification of the way functions are enumerated

Let's document these.

Signed-off-by: Pierre Morel <pmorel@linux.ibm.com>
Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

3 files changed, 129 insertions, 0 deletions

diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index f2a93c8679e8..e6d4fb2e32d0 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -3669,6 +3669,8 @@
 				may put more devices in an IOMMU group.
 		force_floating	[S390] Force usage of floating interrupts.
 		nomio		[S390] Do not use MIO instructions.
+		norid		[S390] ignore the RID field and force use of
+				one PCI domain per PCI function
 
 	pcie_aspm=	[PCIE] Forcibly enable or disable PCIe Active State Power
 			Management.
diff --git a/Documentation/s390/index.rst b/Documentation/s390/index.rst
index f7af2061e406..cf71df5776b4 100644
--- a/Documentation/s390/index.rst
+++ b/Documentation/s390/index.rst
@@ -15,6 +15,7 @@ s390 Architecture
     vfio-ccw
     zfcpdump
     common_io
+    pci
 
     text_files
 
diff --git a/Documentation/s390/pci.rst b/Documentation/s390/pci.rst
new file mode 100644
index 000000000000..75e043d4da85
--- /dev/null
+++ b/Documentation/s390/pci.rst
@@ -0,0 +1,126 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+=========
+S/390 PCI
+=========
+
+Authors:
+        - Pierre Morel
+
+Copyright, IBM Corp. 2020
+
+
+command line parameters and debugfs entries
+===========================================
+
+Command line parameters
+-----------------------
+
+* nomio
+
+  Do not use MIO instructions.
+
+* norid
+
+  Ignore the RID field and force use of one PCI domain per PCI function.
+
+debugfs entries
+---------------
+
+* /sys/kernel/debug/s390dbf/pci_*/ (S/390 debug feature)
+
+  Some views generated by the debug feature to hold various debug outputs.
+
+  - /sys/kernel/debug/s390dbf/pci_msg/sprintf
+    Messages from the processing of PCI events like machine check handling
+    and setting of global functionality like UID checking.
+
+  The level of logging can be changed to be more or less verbose by piping to
+  /sys/kernel/debug/s390dbf/pci_*/level a number between 0 and 6; see the
+  documentation on the S/390 debug feature (Documentation/s390/s390dbf.rst)
+  for details.
+
+Sysfs entries
+=============
+
+Specific entries, or entries specificities for zPCI functions.
+
+* /sys/bus/pci/slots/XXXXXXXX
+
+  The slot entries are setup using the FID (Function Identifier) of the
+  PCI function.
+
+  - /sys/bus/pci/slots/XXXXXXXX/power
+
+  A physical function currently supporting virtual function can not be
+  powered-off until all virtual-function have been removed with
+  echo 0 > /sys/bus/pci/devices/XXXX:XX:XX.X/sriov_numvf
+
+* /sys/bus/pci/devices/XXXX:XX:XX.X/
+
+  - function_id
+    zPCI function identifier unique for the complete Z System.
+    It define uniquely a function in the system.
+
+  - function_handle
+    Low level identifier used for a configured PCI function.
+    It may be useful for debuging.
+
+  - pchid
+    Model dependent location of the I/O adapter.
+
+  - pfgid
+    PCI Function Group ID, functions sharing identical functionality
+    are using a common identifier.
+    A PCI group defines interrupts, IOMMU, IOTLB and DMA specifics.
+
+  - vfn
+    The Virtual Function Number, from 1 to N for virtual functions.
+    0 for physical functions.
+
+  - pft
+    PCI function type specifies the type of the PCI function.
+
+  - port
+    The port correspond to the physical port the function is attached to.
+    It also gives an indication on the physical function a virtual function
+    is attached to.
+
+  - uid
+    The UID, Unique Identifier is defined when configuring a LPAR and is
+    unique inside an LPAR.
+
+  - pfip/segmentX
+    The segments are used to determine the isolation of a function.
+    They corresponds to the physical path to the function.
+    The more the segment are different the more the functions are isolated.
+
+Enumeration and hotplug
+=======================
+
+The PCI address is made of 4 parts: domain, bus, device and function,
+like in DDDD:BB:dd.f
+
+* When not using multi-functions (norid is set or firmware does not support
+  multi-functions)
+
+  - There is only one function per domain.
+
+  - the domain is set from the zPCI function's UID as defined during the
+    LPAR creation.
+
+  - Addresses look like DDDD:00:00.0
+
+* When using multi-functions (norid parameter is not set), there are some
+  change in the way zPCI functions are addressed:
+
+  - There is still only one bus per domain.
+
+  - There can be up to 256 functions per bus.
+
+  - The domain part of the address of all functions of all functions for
+    a multi-Function device is set from the zPCI function's UID as defined
+    in the LPAR creation for the function zero.
+
+  - New functions will only be ready to be used after the function zero
+    (the function with devfn 0) has been enumerated.