summaryrefslogtreecommitdiffstats
path: root/drivers/lightnvm (follow)
Commit message (Collapse)AuthorAgeFilesLines
* lightnvm: allow to force mm initializationMatias Bjørling2016-02-041-9/+16
| | | | | | | | | | | | | | | System block allows the device to initialize with its configured media manager. The system blocks is written to disk, and read again when media manager is determined. For this to work, the backend must store the data. Device drivers, such as null_blk, does not have any backend storage. This patch allows the media manager to be initialized without a storage backend. It also fix incorrect configuration of capabilities in null_blk, as it does not support get/set bad block interface. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: fix request intersection locking in rrpcJavier González2016-02-041-2/+1
| | | | | | | | | | | | This patch fixes an error on the calculation of intersecting logical addresses; it contemplates the case where a new request including several addresses intersects with a single locked address. This case is typical when multiple pages are sent in a new request, while GC - which at the moment sends one address at the time - is running. Signed-off-by: Javier González <javier@cnexlabs.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: warn if irqs are disabled in lock laddrJavier González2016-02-041-0/+2
| | | | | | | | | | Add a warning if irqs are disabled when locking a new address in rrpc. The typical path to a new request does not disable irqs, but this is not guaranteed in the future. Signed-off-by: Javier González <javier@cnexlabs.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: put bio before returnWenwei Tao2016-02-041-1/+3
| | | | | | | | The bio is not returned if the data page cannot be allocated. Signed-off-by: Wenwei Tao <ww.tao0320@gmail.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: introduce factory resetMatias Bjørling2016-01-122-0/+213
| | | | | | | | | | | | | | | | | | | | Now that a device can be managed using the system blocks, a method to reset the device is necessary as well. This patch introduces logic to reset the device easily to factory state and exposes it through an ioctl. The ioctl takes the following flags: NVM_FACTORY_ERASE_ONLY_USER By default all blocks, except host-reserved blocks are erased upon factory reset. Instead of this, only erase host-reserved blocks. NVM_FACTORY_RESET_HOST_BLKS Mark host-reserved blocks to be erased and set their type to free. NVM_FACTORY_RESET_GRWN_BBLKS Mark "grown bad blocks" to be erased and set their type to free. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: use system block for mm initializationMatias Bjørling2016-01-121-2/+23
| | | | | | | | | | | | | | Use system block information to register the appropriate media manager. This enables the LightNVM subsystem to instantiate a media manager selected by the user, instead of relying on automatic detection by each media manager loaded in the kernel. A device must now be initialized before it can proceed to initialize its media manager. Upon initialization, the configured media manager is automatically initialized as well. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: introduce ioctl to initialize deviceMatias Bjørling2016-01-121-0/+50
| | | | | | | | | | Based on the previous patch, we now introduce an ioctl to initialize the device using nvm_init_sysblock and create the necessary system blocks. The user may specify the media manager that they wish to instantiate on top. Default from user-space will be "gennvm". Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: core on-disk initializationMatias Bjørling2016-01-123-1/+564
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | An Open-Channel SSD shall be initialized before use. To initialize, we define an on-disk format, that keeps a small set of metadata to bring up the media manager on top of the device. The initial step is introduced to allow a user to format the disks for a given media manager. During format, a system block is stored on one to three separate luns on the device. Each lun has the system block duplicated. During initialization, the system block can be retrieved and the appropriate media manager can initialized. The on-disk format currently covers (struct nvm_system_block): - Magic value "NVMS". - Monotonic increasing sequence number. - The physical block erase count. - Version of the system block format. - Media manager type. - Media manager superblock physical address. The interface provides three functions to manage the system block: int nvm_init_sysblock(struct nvm_dev *, struct nvm_sb_info *) int nvm_get_sysblock(struct nvm *dev, struct nvm_sb_info *) int nvm_update_sysblock(struct nvm *dev, struct nvm_sb_info *) Each implement a part of the logic to manage the system block. The initialization creates the first system blocks and mark them on the device. Get retrieves the latest system block by scanning all pages in the associated system blocks. The update sysblock writes new metadata and allocates new block if necessary. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: introduce mlc lower page table mappingsMatias Bjørling2016-01-121-1/+60
| | | | | | | | | | | | | | | NAND MLC memories have both lower and upper pages. When programming, both of these must be written, before data can be read. However, these lower and upper pages might not placed at even and odd flash pages, but can be skipped. Therefore each flash memory has its lower pages defined, which can then be used when programming and to know when padding are necessary. This patch implements the lower page definition in the specification, and exposes it through a simple lookup table at dev->lptbl. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: add mccap supportMatias Bjørling2016-01-121-0/+1
| | | | | | | | | | Some flash media has extended capabilities, such as programming SLC pages on MLC/TLC flash, erase/program suspend, scramble and encryption. MCCAP is introduced to detect support for these capabilities in the command set. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: manage open and closed blocks separatelyJavier González2016-01-124-43/+120
| | | | | | | | | | | | | | | | | | | | | | LightNVM targets need to know the state of the flash block when doing flash optimizations. An example is implementing a write buffer to respect the flash page size. Currently, block state is not accounted for; the media manager only differentiates among free, bad and in-use blocks. This patch adds the logic in the generic media manager to enable targets manage blocks into open and close separately, and it implements such management in rrpc. It also adds a set of flags to describe the state of the block (open, closed, free, bad). In order to avoid taking two locks (nvm_lun and rrpc_lun) consecutively, we introduce lockless get_/put_block primitives so that the open and close list locks and future common logic is handled within the nvm_lun lock. Signed-off-by: Javier González <javier@cnexlabs.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: reference rrpc lun in rrpc blockJavier González2016-01-122-0/+2
| | | | | | | | | | | | | | Currently, a rrpc block only points to its nvm_lun. If a user wants to find the associated rrpc lun, it will have to calculate the index and look it up manually. By referencing the rrpc lun directly, this step can be omitted, at the cost of a larger memory footprint. This is important for upcoming patches that implement write buffering in rrpc. Signed-off-by: Javier González <javier@cnexlabs.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: introduce nvm_submit_ppaMatias Bjørling2016-01-121-1/+44
| | | | | | | | | | | | | Internal logic for both core and media managers, does not have a backing bio for issuing I/Os. Introduce nvm_submit_ppa to allow raw I/Os to be submitted to the underlying device driver. The function request the device, ppa, data buffer and its length and will submit the I/O synchronously to the device. The return value may therefore be used to detect any errors regarding the issued I/O. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: move rq->error to nvm_rq->errorMatias Bjørling2016-01-123-5/+6
| | | | | | | | Instead of passing request error into the LightNVM modules, incorporate it into the nvm_rq. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: support multiple ppas in nvm_erase_ppaMatias Bjørling2016-01-122-3/+3
| | | | | | | | Sometimes a user want to erase multiple PPAs at the same time. Extend nvm_erase_ppa to take multiple ppas and number of ppas to be erased. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: move the pages per block check out of the loopWenwei Tao2016-01-121-6/+5
| | | | | | | | | | There is no need to check whether dev's pages per block is beyond rrpc support every time we init a lun, we only need to check it once before enter the lun init loop. Signed-off-by: Wenwei Tao <ww.tao0320@gmail.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: sectors first in ppa listMatias Bjørling2016-01-121-3/+3
| | | | | | | | | The Westlake controller requires that the PPA list has sectors defined sequentially. Currently, the PPA list is created with planes first, then sectors. Change this to sectors first, then planes. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: fix locking and mempool in rrpc_lun_gcWenwei Tao2016-01-121-6/+6
| | | | | | | | | | | | | | | | This patch fix two issues in rrpc_lun_gc 1. prio_list is protected by rrpc_lun's lock not nvm_lun's, so acquire rlun's lock instead of lun's before operate on the list. 2. we delete block from prio_list before allocating gcb, but gcb allocation may fail, we end without putting it back to the list, this makes the block won't get reclaimed in the future. To solve this issue, delete block after gcb allocation. Signed-off-by: Wenwei Tao <ww.tao0320@gmail.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: put block back to gc list on its reclaim failWenwei Tao2016-01-121-4/+14
| | | | | | | | | | | We delete a block from the gc list before reclaim it, so put it back to the list on its reclaim fail, otherwise this block will not get reclaimed and be programmable in the future. Signed-off-by: Wenwei Tao <ww.tao0320@gmail.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: check bi_error in gcWenwei Tao2016-01-121-0/+6
| | | | | | | | | We should check last io completion status before starting another one. Signed-off-by: Wenwei Tao <ww.tao0320@gmail.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: refactor end_io functions for syncMatias Bjørling2016-01-123-24/+32
| | | | | | | | | | | | | | | To implement sync I/O support within the LightNVM core, the end_io functions are refactored to take an end_io function pointer instead of testing for initialized media manager, followed by calling its end_io function. Sync I/O can then be implemented using a callback that signal I/O completion. This is similar to the logic found in blk_to_execute_io(). By implementing it this way, the underlying device I/Os submission logic is abstracted away from core, targets, and media managers. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: refactor rqd ppa list into set/freeMatias Bjørling2016-01-121-21/+50
| | | | | | | | | | A device may be driven in single, double or quad plane mode. In that case, the rqd must have either one, two, or four PPAs set for a single PPA sent to the device. Refactor this logic into their own functions to be shared by program/erase/read in the core. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: move ppa erase logic to coreMatias Bjørling2016-01-122-64/+71
| | | | | | | | | | | A device may function in single, dual or quad plane mode. The gennvm media manager manages this with explicit helpers. They convert a single ppa to 1, 2 or 4 separate ppas in a ppa list. To aid implementation of recovery and system blocks, this functionality can be moved directly into the core. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: unlock rq and free ppa_list on submission failWenwei Tao2016-01-121-0/+6
| | | | | | | | | | | When rrpc_write_ppalist_rq and rrpc_read_ppalist_rq succeed, we setup rq correctly, but nvm_submit_io may afterward fail since it cannot allocate request or nvme_nvm_command, we return error but forget to cleanup the previous work. Signed-off-by: Wenwei Tao <ww.tao0320@gmail.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: add check after mempool allocationJavier Gonzalez2016-01-121-0/+2
| | | | | | | | | The mempool allocation might fail. Make sure to return error when it does, instead of causing a kernel panic. Signed-off-by: Javier Gonzalez <javier@cnexlabs.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: fix incorrect nr_free_blocks statChao Yu2016-01-121-0/+1
| | | | | | | | | | | When initing bad block list in gennvm_block_bb, once we move bad block from free_list to bb_list, we should maintain both stat info nr_free_blocks and nr_bad_blocks. So this patch fixes to add missing operation related to nr_free_blocks. Signed-off-by: Chao Yu <chao2.yu@samsung.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: fix bio submission issueWenwei Tao2016-01-122-2/+4
| | | | | | | | | | | Put bio when submission fails, since we get it before submission. And return error when backend device driver doesn't provide a submit_io method, thus we can end IO properly. Signed-off-by: Wenwei Tao <ww.tao0320@gmail.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: wrong offset in bad blk lun calculationMatias Bjørling2015-12-291-1/+1
| | | | | | | | | | | | | dev->nr_luns reports the total number of luns available in a device while dev->luns_per_chnl is the number of luns per channel. When multiple channels are available, the offset is calculated from a channel and lun id into a linear array. As it multiplies with the total number of luns, we go out of bound when channel id > 0 and causes the kernel to panic when we read a protected kernel memory area. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: do not compile in debugging by defaultMatias Bjørling2015-12-071-0/+1
| | | | | | | | | | The LightNVM module exposes a debug interface when CONFIG_NVM_DEBUG is set. This interfaces takes a string to configure media managers and targets. Make sure this interface is only exposed when chosen deliberately. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: prevent gennvm module unload on useMatias Bjørling2015-12-071-0/+5
| | | | | | | | | After the gennvm module has been initialized. It might be attached to one or several devices. In that case, the module is in use. Make sure that it can not be unloaded. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: fix media mgr registrationMatias Bjørling2015-12-071-39/+39
| | | | | | | | | | | | | | | This patch fixes two issues during media manager registration. 1. The ppa pool can be used at media manager registration. Allocate the ppa pool before that. 2. If a media manager can't be found, this should not lead to the device being unallocated. A media manager can be registered later, that can manage the device. Only warn if a media manager fails initialization. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: replace req queue with nvmdev for lldMatias Bjørling2015-12-073-9/+8
| | | | | | | | | | | In the case where a request queue is passed to the low lever lightnvm device drive integration, the device driver might pass its admin commands through another queue. Instead pass nvm_dev, and let the low level drive the appropriate queue. Reported-by: Christoph Hellwig <hch@infradead.org> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: refactor spin_unlock in gennvm_get_blkWenwei Tao2015-12-071-5/+2
| | | | | | | | | The spin_unlock is duplicated multiple times. Jump to a single unlock to improve the code flow. Signed-off-by: Wenwei Tao <ww.tao0320@gmail.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: put blks when luns configure failedWenwei Tao2015-12-071-2/+19
| | | | | | | | | | Put the allocated blocks back to the free list when the luns configure failed, to make these blocks useable to others. Signed-off-by: Wenwei Tao <ww.tao0320@gmail.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: use flags in rrpc_get_blkWenwei Tao2015-12-071-1/+1
| | | | | | | | | | rrpc_get_blk use constant 0 as the input parameter of nvm_get_blk, this may result in getting gc block failed unexpectedly. Signed-off-by: Wenwei Tao <ww.tao0320@gmail.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: missing nvm_lock acquireWenwei Tao2015-11-291-33/+42
| | | | | | | | | | To avoid race conditions, traverse dev, media manager, and target lists and also register, unregister entries to/from them, should be always under the nvm_lock control. Signed-off-by: Wenwei Tao <ww.tao0320@gmail.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: unconverted ppa returned in get_bb_tblMatias Bjørling2015-11-291-2/+1
| | | | | | | | | | | | The get_bb_tbl function takes ppa as a generic address, which is converted to the ppa device address within the device driver. When the update_bbtbl callback is called from get_bb_tbl, the device specific ppa is used, instead of the generic ppa. Make sure to pass the generic ppa. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: do device max sectors boundary check firstWenwei Tao2015-11-291-4/+6
| | | | | | | | | | do device max_phys_sect boundary check first, otherwise we will allocate dma_pools for devices whose max sectors are beyond lightnvm support and register them. Signed-off-by: Wenwei Tao <ww.tao0320@gmail.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: fix ioctl memory leaksSudip Mukherjee2015-11-291-2/+7
| | | | | | | | | If copy_to_user() fails we returned error but we missed releasing devices. Signed-off-by: Sudip Mukherjee <sudip@vectorindia.org> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: free memory when gennvm register failsWenwei Tao2015-11-291-5/+10
| | | | | | | | free allocated nvm block and gennvm lun structures when gennvm register fails, otherwise it will cause memory leak. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: add free and bad lun info to show lunsJavier Gonzalez2015-11-202-6/+15
| | | | | | | | | | | Add free block, used block, and bad block information to the show debug interface. This information is used to debug how targets track blocks. Also, change debug function name to make it more generic. Signed-off-by: Javier Gonzalez <javier@cnexlabs.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: keep track of block countsJavier Gonzalez2015-11-201-1/+13
| | | | | | | | | | | | | | Maintain number of in use blocks, free blocks, and bad blocks in a per lun basis. This allows the upper layers to get information about the state of each lun. Also, account for blocks reserved to the device on the free block count. nr_free_blocks matches now the actual number of blocks on the free list when the device is booted. Signed-off-by: Javier Gonzalez <javier@cnexlabs.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: missing free on init errorMatias Bjørling2015-11-201-2/+4
| | | | | | | | If either max_phys_sect is out of bound, the nvm_dev structure is not freed. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: wrong return value and redundant freeWenwei Tao2015-11-201-4/+2
| | | | | | | | | The return value should be non-zero under error conditions. Remove nvme_free(dev) to avoid free dev more than once. Signed-off-by: Wenwei Tao <ww.tao0320@gmail.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: cleanup queue before target removalJavier González2015-11-161-2/+2
| | | | | | | | | | This prevents outstanding IOs to be sent for completion to target after the target has been removed. The flow is now: stop new IOs > cleanup queue > remove target. Signed-off-by: Javier Gonzalez <javier@cnexlabs.com> Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: remove linear and device addr modesMatias Bjørling2015-11-163-9/+38
| | | | | | | | | | | | The linear and device specific address modes can be replaced with a simple offset and bit length conversion that is generic across all devices. This both simplifies the specification and removes the special case for qemu nvme, that previously relied on the linear address mapping. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: prevent double free on init errorMatias Bjørling2015-11-161-9/+3
| | | | | | | | Both the nvm_register and nvm_init does a kfree(dev) on error. Make sure to only free it once. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: prematurely activate nvm_devMatias Bjørling2015-11-161-4/+4
| | | | | | | | | We register with nvm_devices when there registration can still fail. Move the final registration at the end of the nvm_register function to make sure we are fully registered when added to the nvm_devices list. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: check for NAND flash and its typeMatias Bjørling2015-11-161-0/+10
| | | | | | | | Only NAND flash with SLC and MLC is supported. Make sure to not try to initialize TLC memory or other non-volatile memory types. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
* lightnvm: update bad block table formatMatias Bjørling2015-11-162-10/+24
| | | | | | | | | | | | | | | | | | | | | | The specification was changed to reflect a multi-value bad block table. Instead of bit-based bad block table, the bad block table now allows eight bad block categories. Currently four are defined: * Factory bad blocks * Grown bad blocks * Device-side reserved blocks * Host-side reserved blocks The factory and grown bad blocks are the regular bad blocks. The reserved blocks are either for internal use or external use. In particular, the device-side reserved blocks allows the host to bootstrap from a limited number of flash blocks. Reducing the flash blocks to scan upon super block initialization. Support for both get bad block table and set bad block table is added. Signed-off-by: Matias Bjørling <m@bjorling.me> Signed-off-by: Jens Axboe <axboe@fb.com>
generated by cgit v1.2.3 (git 2.39.1) at 2024-12-04 00:36:27 +0100