diff options
| author | Brian Coca <bcoca@users.noreply.github.com> | 2020-06-05 03:01:46 +0200 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2020-06-05 03:01:46 +0200 |
| commit | 062e780a68f9acd2ee6f824f252458b8a0351f24 (patch) | |
| tree | d30bdf5dc0586214c2078a43c7f7e1a77b672cfc | |
| parent | lineinfile - Add alternative backrefs example to docs (#69844) (diff) | |
| download | ansible-062e780a68f9acd2ee6f824f252458b8a0351f24.tar.xz ansible-062e780a68f9acd2ee6f824f252458b8a0351f24.zip | |
starting metadata sunset (#69454)
* starting metadata sunset
- purged metadata from any requirements
- fix indent in generic handler for yaml content (whey metadata display was off)
- make more resilient against bad formed docs
- removed all metadata from docs template
- remove metadata from schemas
- removed mdata tests and from unrelated tests
Co-authored-by: Felix Fontein <felix@fontein.de>
Co-authored-by: Rick Elrod <rick@elrod.me>
91 files changed, 140 insertions, 1761 deletions
diff --git a/docs/docsite/rst/dev_guide/developing_modules_documenting.rst b/docs/docsite/rst/dev_guide/developing_modules_documenting.rst index 1e73007ed3..5cba41377f 100644 --- a/docs/docsite/rst/dev_guide/developing_modules_documenting.rst +++ b/docs/docsite/rst/dev_guide/developing_modules_documenting.rst @@ -15,11 +15,11 @@ Every Ansible module written in Python must begin with seven standard sections i .. note:: Why don't the imports go first? - Keen Python programmers may notice that contrary to PEP 8's advice we don't put ``imports`` at the top of the file. This is because the ``ANSIBLE_METADATA`` through ``RETURN`` sections are not used by the module code itself; they are essentially extra docstrings for the file. The imports are placed after these special variables for the same reason as PEP 8 puts the imports after the introductory comments and docstrings. This keeps the active parts of the code together and the pieces which are purely informational apart. The decision to exclude E402 is based on readability (which is what PEP 8 is about). Documentation strings in a module are much more similar to module level docstrings, than code, and are never utilized by the module itself. Placing the imports below this documentation and closer to the code, consolidates and groups all related code in a congruent manner to improve readability, debugging and understanding. + Keen Python programmers may notice that contrary to PEP 8's advice we don't put ``imports`` at the top of the file. This is because the ``DOCUMENTATION`` through ``RETURN`` sections are not used by the module code itself; they are essentially extra docstrings for the file. The imports are placed after these special variables for the same reason as PEP 8 puts the imports after the introductory comments and docstrings. This keeps the active parts of the code together and the pieces which are purely informational apart. The decision to exclude E402 is based on readability (which is what PEP 8 is about). Documentation strings in a module are much more similar to module level docstrings, than code, and are never utilized by the module itself. Placing the imports below this documentation and closer to the code, consolidates and groups all related code in a congruent manner to improve readability, debugging and understanding. .. warning:: **Copy old modules with care!** - Some older modules in Ansible Core have ``imports`` at the bottom of the file, ``Copyright`` notices with the full GPL prefix, and/or ``ANSIBLE_METADATA`` fields in the wrong order. These are legacy files that need updating - do not copy them into new modules. Over time we're updating and correcting older modules. Please follow the guidelines on this page! + Some older modules in Ansible Core have ``imports`` at the bottom of the file, ``Copyright`` notices with the full GPL prefix, and/or ``DOCUMENTATION`` fields in the wrong order. These are legacy files that need updating - do not copy them into new modules. Over time we're updating and correcting older modules. Please follow the guidelines on this page! .. _shebang: @@ -60,61 +60,15 @@ Major additions to the module (for instance, rewrites) may add additional copyri ANSIBLE_METADATA block ====================== -After the shebang, the UTF-8 coding, the copyright, and the license, your module file should contain an ``ANSIBLE_METADATA`` section. This section provides information about the module for use by other tools. For new modules, the following block can be simply added into your module: +Since we moved to collections we have deprecated the METADATA functionality, it is no longer required for modules, but it will not break anything if present. -.. code-block:: python - - ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['preview'], - 'supported_by': 'community'} - -.. warning:: - - * ``metadata_version`` is the version of the ``ANSIBLE_METADATA`` schema, *not* the version of the module. - * Promoting a module's ``status`` or ``supported_by`` status should only be done by members of the Ansible Core Team. - -Ansible metadata fields ------------------------ - -:metadata_version: An "X.Y" formatted string. X and Y are integers which - define the metadata format version. Modules shipped with Ansible are - tied to an Ansible release, so we will only ship with a single version - of the metadata. We'll increment Y if we add fields or legal values - to an existing field. We'll increment X if we remove fields or values - or change the type or meaning of a field. - Current metadata_version is "1.1" - -:supported_by: Who supports the module. - Default value is ``community``. For information on what the support level values entail, please see - :ref:`Modules Support <modules_support>`. Values are: - - * core - * network - * certified - * community - * curated (*deprecated value - modules in this category should be core or - certified instead*) - -:status: List of strings describing how stable the module is likely to be. See also :ref:`module_lifecycle`. - The default value is a single element list ["preview"]. The following strings are valid - statuses and have the following meanings: - - :stableinterface: The module's options (the parameters or arguments it accepts) are stable. Every effort will be made not to remove options or change - their meaning. **Not** a rating of the module's code quality. - :preview: The module is in tech preview. It may be - unstable, the options may change, or it may require libraries or - web services that are themselves subject to incompatible changes. - :deprecated: The module is deprecated and will be removed in a future release. - :removed: The module is not present in the release. A stub is - kept so that documentation can be built. The documentation helps - users port from the removed module to new modules. .. _documentation_block: DOCUMENTATION block =================== -After the shebang, the UTF-8 coding, the copyright line, the license, and the ``ANSIBLE_METADATA`` section comes the ``DOCUMENTATION`` block. Ansible's online module documentation is generated from the ``DOCUMENTATION`` blocks in each module's source code. The ``DOCUMENTATION`` block must be valid YAML. You may find it easier to start writing your ``DOCUMENTATION`` string in an :ref:`editor with YAML syntax highlighting <other_tools_and_programs>` before you include it in your Python file. You can start by copying our `example documentation string <https://github.com/ansible/ansible/blob/devel/examples/DOCUMENTATION.yml>`_ into your module file and modifying it. If you run into syntax issues in your YAML, you can validate it on the `YAML Lint <http://www.yamllint.com/>`_ website. +After the shebang, the UTF-8 coding, the copyright line, and the license section comes the ``DOCUMENTATION`` block. Ansible's online module documentation is generated from the ``DOCUMENTATION`` blocks in each module's source code. The ``DOCUMENTATION`` block must be valid YAML. You may find it easier to start writing your ``DOCUMENTATION`` string in an :ref:`editor with YAML syntax highlighting <other_tools_and_programs>` before you include it in your Python file. You can start by copying our `example documentation string <https://github.com/ansible/ansible/blob/devel/examples/DOCUMENTATION.yml>`_ into your module file and modifying it. If you run into syntax issues in your YAML, you can validate it on the `YAML Lint <http://www.yamllint.com/>`_ website. Module documentation should briefly and accurately define what each module and option does, and how it works with others in the underlying system. Documentation should be written for broad audience--readable both by experts and non-experts. * Descriptions should always start with a capital letter and end with a full stop. Consistency always helps. @@ -353,7 +307,7 @@ For example, all AWS modules should include: EXAMPLES block ============== -After the shebang, the UTF-8 coding, the copyright line, the license, the ``ANSIBLE_METADATA`` section, and the ``DOCUMENTATION`` block comes the ``EXAMPLES`` block. Here you show users how your module works with real-world examples in multi-line plain-text YAML format. The best examples are ready for the user to copy and paste into a playbook. Review and update your examples with every change to your module. +After the shebang, the UTF-8 coding, the copyright line, the license section, and the ``DOCUMENTATION`` block comes the ``EXAMPLES`` block. Here you show users how your module works with real-world examples in multi-line plain-text YAML format. The best examples are ready for the user to copy and paste into a playbook. Review and update your examples with every change to your module. Per playbook best practices, each example should include a ``name:`` line:: @@ -375,7 +329,7 @@ If your module returns facts that are often needed, an example of how to use the RETURN block ============ -After the shebang, the UTF-8 coding, the copyright line, the license, the ``ANSIBLE_METADATA`` section, ``DOCUMENTATION`` and ``EXAMPLES`` blocks comes the ``RETURN`` block. This section documents the information the module returns for use by other modules. +After the shebang, the UTF-8 coding, the copyright line, the license section, ``DOCUMENTATION`` and ``EXAMPLES`` blocks comes the ``RETURN`` block. This section documents the information the module returns for use by other modules. If your module doesn't return anything (apart from the standard returns), this section of your module should read: ``RETURN = r''' # '''`` Otherwise, for each value returned, provide the following fields. All fields are required unless specified otherwise. @@ -447,7 +401,7 @@ Here are two example ``RETURN`` sections, one with three simple fields and one w Python imports ============== -After the shebang, the UTF-8 coding, the copyright line, the license, and the sections for ``ANSIBLE_METADATA``, ``DOCUMENTATION``, ``EXAMPLES``, and ``RETURN``, you can finally add the python imports. All modules must use Python imports in the form: +After the shebang, the UTF-8 coding, the copyright line, the license, and the sections for ``DOCUMENTATION``, ``EXAMPLES``, and ``RETURN``, you can finally add the python imports. All modules must use Python imports in the form: .. code-block:: python diff --git a/docs/docsite/rst/dev_guide/developing_modules_general.rst b/docs/docsite/rst/dev_guide/developing_modules_general.rst index 346881ea18..10e015550f 100644 --- a/docs/docsite/rst/dev_guide/developing_modules_general.rst +++ b/docs/docsite/rst/dev_guide/developing_modules_general.rst @@ -63,12 +63,6 @@ To create a new module: # Copyright: (c) 2018, Terry Jones <terry.jones@example.org> # GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt) - ANSIBLE_METADATA = { - 'metadata_version': '1.1', - 'status': ['preview'], - 'supported_by': 'community' - } - DOCUMENTATION = ''' --- module: my_test diff --git a/docs/docsite/rst/dev_guide/module_lifecycle.rst b/docs/docsite/rst/dev_guide/module_lifecycle.rst index 86acd732be..785466c427 100644 --- a/docs/docsite/rst/dev_guide/module_lifecycle.rst +++ b/docs/docsite/rst/dev_guide/module_lifecycle.rst @@ -4,15 +4,16 @@ The lifecycle of an Ansible module ********************************** -Modules in the main Ansible repo have a defined life cycle, from first introduction to final removal. The module life cycle is tied to the `Ansible release cycle <release_cycle>` and reflected in the :ref:`ansible_metadata_block`. A module may move through these four states: +Modules in the main Ansible repo have a defined life cycle, from first introduction to final removal. The module life cycle is tied to the `Ansible release cycle <release_cycle>`. +A module may move through these four states: -1. When a module is first accepted into Ansible, we consider it in tech preview and mark it ``preview``. Modules in ``preview`` are not stable. You may change the parameters or dependencies, expand or reduce the functionality of ``preview`` modules. Many modules remain ``preview`` for years. +1. When a module is first accepted into Ansible, we consider it in tech preview and will mark it as such in the documentation. -2. If a module matures, we may mark it ``stableinterface`` and commit to maintaining its parameters, dependencies, and functionality. We support (though we cannot guarantee) backwards compatibility for ``stableinterface`` modules, which means their parameters should be maintained with stable meanings. +2. If a module matures, we will remove the 'preview' mark in the documentation. We support (though we cannot guarantee) backwards compatibility for these modules, which means their parameters should be maintained with stable meanings. -3. If a module's target API changes radically, or if someone creates a better implementation of its functionality, we may mark it ``deprecated``. Modules that are ``deprecated`` are still available but they are reaching the end of their life cycle. We retain deprecated modules for 4 release cycles with deprecation warnings to help users update playbooks and roles that use them. +3. If a module's target API changes radically, or if someone creates a better implementation of its functionality, we may mark it deprecated. Modules that are deprecated are still available but they are reaching the end of their life cycle. We retain deprecated modules for 4 release cycles with deprecation warnings to help users update playbooks and roles that use them. -4. When a module has been deprecated for four release cycles, we remove the code and mark the stub file ``removed``. Modules that are ``removed`` are no longer shipped with Ansible. The stub file helps users find alternative modules. +4. When a module has been deprecated for four release cycles, we remove the code and mark the stub file removed. Modules that are removed are no longer shipped with Ansible. The stub file helps users find alternative modules. .. _deprecating_modules: @@ -24,14 +25,16 @@ To deprecate a module, you must: 1. Rename the file so it starts with an ``_``, for example, rename ``old_cloud.py`` to ``_old_cloud.py``. This keeps the module available and marks it as deprecated on the module index pages. 2. Mention the deprecation in the relevant ``CHANGELOG``. 3. Reference the deprecation in the relevant ``porting_guide_x.y.rst``. -4. Update ``ANSIBLE_METADATA`` to contain ``status: ['deprecated']``. -5. Add ``deprecated:`` to the documentation with the following sub-values: +4. Add ``deprecated:`` to the documentation with the following sub-values: - :removed_in: A ``string``, such as ``"2.9"``; the version of Ansible where the module will be replaced with a docs-only module stub. Usually current release +4. + :removed_in: A ``string``, such as ``"2.10"``; the version of Ansible where the module will be replaced with a docs-only module stub. Usually current release +4. :why: Optional string that used to detail why this has been removed. :alternative: Inform users they should do instead, i.e. ``Use M(whatmoduletouseinstead) instead.``. +* note: with the advent of collections and ``routing.yml`` we might soon require another entry in this file to mark the deprecation. + * For an example of documenting deprecation, see this `PR that deprecates multiple modules <https://github.com/ansible/ansible/pull/43781/files>`_. + Some of the elements in the PR might now be out of date. Changing a module name ====================== diff --git a/docs/docsite/rst/dev_guide/testing.rst b/docs/docsite/rst/dev_guide/testing.rst index 480d2c3c56..fbc716f4ed 100644 --- a/docs/docsite/rst/dev_guide/testing.rst +++ b/docs/docsite/rst/dev_guide/testing.rst @@ -66,11 +66,10 @@ When Shippable detects an error and it can be linked back to a file that has bee lib/ansible/modules/network/foo/bar.py:509:17: E265 block comment should start with '# ' - The test `ansible-test sanity --test validate-modules` failed with the following errors: + The test `ansible-test sanity --test validate-modules` failed with the following error: lib/ansible/modules/network/foo/bar.py:0:0: E307 version_added should be 2.4. Currently 2.3 - lib/ansible/modules/network/foo/bar.py:0:0: E316 ANSIBLE_METADATA.metadata_version: required key not provided @ data['metadata_version']. Got None -From the above example we can see that ``--test pep8`` and ``--test validate-modules`` have identified issues. The commands given allow you to run the same tests locally to ensure you've fixed the issues without having to push your changed to GitHub and wait for Shippable, for example: +From the above example we can see that ``--test pep8`` and ``--test validate-modules`` have identified an issue. The commands given allow you to run the same tests locally to ensure you've fixed all issues without having to push your changes to GitHub and wait for Shippable, for example: If you haven't already got Ansible available, use the local checkout by running:: diff --git a/docs/docsite/rst/dev_guide/testing_validate-modules.rst b/docs/docsite/rst/dev_guide/testing_validate-modules.rst index 850b3bd71d..a876d7df2f 100644 --- a/docs/docsite/rst/dev_guide/testing_validate-modules.rst +++ b/docs/docsite/rst/dev_guide/testing_validate-modules.rst @@ -80,10 +80,10 @@ Codes documentation-error Documentation Error Unknown ``DOCUMENTATION`` error documentation-syntax-error Documentation Error Invalid ``DOCUMENTATION`` schema illegal-future-imports Imports Error Only the following ``from __future__`` imports are allowed: ``absolute_import``, ``division``, and ``print_function``. - import-before-documentation Imports Error Import found before documentation variables. All imports must appear below ``DOCUMENTATION``/``EXAMPLES``/``RETURN``/``ANSIBLE_METADATA`` + import-before-documentation Imports Error Import found before documentation variables. All imports must appear below ``DOCUMENTATION``/``EXAMPLES``/``RETURN`` import-error Documentation Error ``Exception`` attempting to import module for ``argument_spec`` introspection - import-placement Locations Warning Imports should be directly below ``DOCUMENTATION``/``EXAMPLES``/``RETURN``/``ANSIBLE_METADATA`` for legacy modules - imports-improper-location Imports Error Imports should be directly below ``DOCUMENTATION``/``EXAMPLES``/``RETURN``/``ANSIBLE_METADATA`` + import-placement Locations Warning Imports should be directly below ``DOCUMENTATION``/``EXAMPLES``/``RETURN`` + imports-improper-location Imports Error Imports should be directly below ``DOCUMENTATION``/``EXAMPLES``/``RETURN`` incompatible-choices Documentation Error Choices value from the argument_spec is not compatible with type defined in the argument_spec incompatible-default-type Documentation Error Default value from the argument_spec is not compatible with type defined in the argument_spec invalid-argument-name Documentation Error Argument in argument_spec must not be one of 'message', 'syslog_facility' as it is used internally by Ansible Core Engine @@ -93,14 +93,10 @@ Codes invalid-documentation-options Documentation Error ``DOCUMENTATION.options`` must be a dictionary/hash when used invalid-examples Documentation Error ``EXAMPLES`` is not valid YAML invalid-extension Naming Error Official Ansible modules must have a ``.py`` extension for python modules or a ``.ps1`` for powershell modules - invalid-metadata-status Documentation Error ``ANSIBLE_METADATA.status`` of deprecated or removed can't include other statuses - invalid-metadata-type Documentation Error ``ANSIBLE_METADATA`` was not provided as a dict, YAML not supported, Invalid ``ANSIBLE_METADATA`` schema - invalid-module-deprecation-source Documentation Error The deprecated version for the module must not be from a documentation fragment from another collection or Ansible-base invalid-module-schema Documentation Error ``AnsibleModule`` schema validation error invalid-requires-extension Naming Error Module ``#AnsibleRequires -CSharpUtil`` should not end in .cs, Module ``#Requires`` should not end in .psm1 invalid-tagged-version Documentation Error All version numbers specified in code have to be explicitly tagged with the collection name, i.e. ``community.general:1.2.3`` or ``ansible.builtin:2.10`` last-line-main-call Syntax Error Call to ``main()`` not the last line (or ``removed_module()`` in the case of deprecated & docs only modules) - metadata-changed Documentation Error ``ANSIBLE_METADATA`` cannot be changed in a point release for a stable branch missing-doc-fragment Documentation Error ``DOCUMENTATION`` fragment missing missing-existing-doc-fragment Documentation Warning Pre-existing ``DOCUMENTATION`` fragment missing missing-documentation Documentation Error No ``DOCUMENTATION`` provided @@ -108,7 +104,6 @@ Codes missing-gplv3-license Documentation Error GPLv3 license header not found missing-if-name-main Syntax Error Next to last line is not ``if __name__ == "__main__":`` missing-main-call Syntax Error Did not find a call to ``main()`` (or ``removed_module()`` in the case of deprecated & docs only modules) - missing-metadata Documentation Error No ``ANSIBLE_METADATA`` provided missing-module-utils-basic-import Imports Warning Did not find ``ansible.module_utils.basic`` import missing-module-utils-import-csharp-requirements Imports Error No ``Ansible.ModuleUtils`` or C# Ansible util requirements/imports found missing-powershell-interpreter Syntax Error Interpreter line is not ``#!powershell`` diff --git a/docs/docsite/rst/network/dev_guide/developing_resource_modules_network.rst b/docs/docsite/rst/network/dev_guide/developing_resource_modules_network.rst index cd22181235..5ef5291020 100644 --- a/docs/docsite/rst/network/dev_guide/developing_resource_modules_network.rst +++ b/docs/docsite/rst/network/dev_guide/developing_resource_modules_network.rst @@ -53,12 +53,6 @@ For example, the resource model builder includes the ``myos_interfaces.yml`` sam --- GENERATOR_VERSION: '1.0' - ANSIBLE_METADATA: | - { - 'metadata_version': '1.1', - 'status': ['preview'], - 'supported_by': '<support_group>' - } NETWORK_OS: myos RESOURCE: interfaces COPYRIGHT: Copyright 2019 Red Hat diff --git a/docs/templates/plugin.rst.j2 b/docs/templates/plugin.rst.j2 index b829dd42a5..f61d841b9a 100644 --- a/docs/templates/plugin.rst.j2 +++ b/docs/templates/plugin.rst.j2 @@ -418,35 +418,6 @@ Status - This @{ plugin_type }@ will be removed in version @{ deprecated['removed_in'] | default('') | string | rst_ify }@. *[deprecated]* - For more information see `DEPRECATED`_. -{% else %} - -{% set support = { 'core': 'the Ansible Core Team', 'network': 'the Ansible Network Team', 'certified': 'an Ansible Partner', 'community': 'the Ansible Community', 'curated': 'a Third Party'} %} -{% set module_states = { 'preview': 'not guaranteed to have a backwards compatible interface', 'stableinterface': 'guaranteed to have backward compatible interface changes going forward'} %} - -{% if metadata %} -{% if metadata.status %} - -{% for cur_state in metadata.status %} -- This @{ plugin_type }@ is @{ module_states[cur_state] }@. *[@{ cur_state }@]* -{% endfor %} - -{% endif %} - -{% if metadata.supported_by %} -{% set supported_by = support[metadata.supported_by] %} -- This @{ plugin_type }@ is :ref:`maintained by @{ supported_by }@ <modules_support>`. *[@{ metadata.supported_by }@]* - -{% if metadata.supported_by in ('core', 'network') %} -Red Hat Support -~~~~~~~~~~~~~~~ - -More information about Red Hat's support of this @{ plugin_type }@ is available from this `Red Hat Knowledge Base article <https://access.redhat.com/articles/3166901>`_. -{% endif %} - -{% endif %} - -{% endif %} - {% endif %} {% if author is defined -%} diff --git a/hacking/metadata-tool.py b/hacking/metadata-tool.py deleted file mode 100755 index 0cd0adc9a2..0000000000 --- a/hacking/metadata-tool.py +++ /dev/null @@ -1,537 +0,0 @@ -#!/usr/bin/env python -# (c) 2016-2017, Toshio Kuratomi <tkuratomi@ansible.com> -# -# This file is part of Ansible -# -# Ansible is free software: you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation, either version 3 of the License, or -# (at your option) any later version. -# -# Ansible is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with Ansible. If not, see <http://www.gnu.org/licenses/>. - -# Make coding more python3-ish -from __future__ import (absolute_import, division, print_function) -__metaclass__ = type - -import ast -import csv -import os -import sys -from collections import defaultdict -from distutils.version import StrictVersion -from pprint import pformat, pprint - -from ansible.parsing.metadata import DEFAULT_METADATA, ParseError, extract_metadata -from ansible.plugins.loader import module_loader - - -# There's a few files that are not new-style modules. Have to blacklist them -NONMODULE_PY_FILES = frozenset(('async_wrapper.py',)) -NONMODULE_MODULE_NAMES = frozenset(os.path.splitext(p)[0] for p in NONMODULE_PY_FILES) - - -class MissingModuleError(Exception): - """Thrown when unable to find a plugin""" - pass - - -def usage(): - print("""Usage: - metadata-tool.py report [--version X] - metadata-tool.py add [--version X] [--overwrite] CSVFILE - metadata-tool.py add-default [--version X] [--overwrite] - medatada-tool.py upgrade [--version X]""") - sys.exit(1) - - -def parse_args(arg_string): - if len(arg_string) < 1: - usage() - - action = arg_string[0] - - version = None - if '--version' in arg_string: - version_location = arg_string.index('--version') - arg_string.pop(version_location) - version = arg_string.pop(version_location) - - overwrite = False - if '--overwrite' in arg_string: - overwrite = True - arg_string.remove('--overwrite') - - csvfile = None - if len(arg_string) == 2: - csvfile = arg_string[1] - elif len(arg_string) > 2: - usage() - - return action, {'version': version, 'overwrite': overwrite, 'csvfile': csvfile} - - -def find_documentation(module_data): - """Find the DOCUMENTATION metadata for a module file""" - start_line = -1 - mod_ast_tree = ast.parse(module_data) - for child in mod_ast_tree.body: - if isinstance(child, ast.Assign): - for target in child.targets: - if target.id == 'DOCUMENTATION': - start_line = child.lineno - 1 - break - - return start_line - - -def remove_metadata(module_data, start_line, start_col, end_line, end_col): - """Remove a section of a module file""" - lines = module_data.split('\n') - new_lines = lines[:start_line] - if start_col != 0: - new_lines.append(lines[start_line][:start_col]) - - next_line = lines[end_line] - if len(next_line) - 1 != end_col: - new_lines.append(next_line[end_col:]) - - if len(lines) > end_line: - new_lines.extend(lines[end_line + 1:]) - return '\n'.join(new_lines) - - -def insert_metadata(module_data, new_metadata, insertion_line, targets=('ANSIBLE_METADATA',)): - """Insert a new set of metadata at a specified line""" - assignments = ' = '.join(targets) - pretty_metadata = pformat(new_metadata, width=1).split('\n') - - new_lines = [] - new_lines.append('{0} = {1}'.format(assignments, pretty_metadata[0])) - - if len(pretty_metadata) > 1: - for line in pretty_metadata[1:]: - new_lines.append('{0}{1}'.format(' ' * (len(assignments) - 1 + len(' = {')), line)) - - old_lines = module_data.split('\n') - lines = old_lines[:insertion_line] + new_lines + old_lines[insertion_line:] - return '\n'.join(lines) - - -def parse_assigned_metadata_initial(csvfile): - """ - Fields: - :0: Module name - :1: Core (x if so) - :2: Extras (x if so) - :3: Category - :4: Supported/SLA - :5: Curated - :6: Stable - :7: Deprecated - :8: Notes - :9: Team Notes - :10: Notes 2 - :11: final supported_by field - """ - with open(csvfile, 'rb') as f: - for record in csv.reader(f): - module = record[0] - - if record[12] == 'core': - supported_by = 'core' - elif record[12] == 'curated': - supported_by = 'curated' - elif record[12] == 'community': - supported_by = 'community' - else: - print('Module %s has no supported_by field. Using community' % record[0]) - supported_by = 'community' - supported_by = DEFAULT_METADATA['supported_by'] - - status = [] - if record[6]: - status.append('stableinterface') - if record[7]: - status.append('deprecated') - if not status: - status.extend(DEFAULT_METADATA['status']) - - yield (module, {'version': DEFAULT_METADATA['metadata_version'], 'supported_by': supported_by, 'status': status}) - - -def parse_assigned_metadata(csvfile): - """ - Fields: - :0: Module name - :1: supported_by string. One of the valid support fields - core, community, certified, network - :2: stableinterface - :3: preview - :4: deprecated - :5: removed - - https://docs.ansible.com/ansible/latest/dev_guide/developing_modules_documenting.html#ansible-metadata-block - """ - with open(csvfile, 'rb') as f: - for record in csv.reader(f): - module = record[0] - supported_by = record[1] - - status = [] - if record[2]: - status.append('stableinterface') - if record[4]: - status.append('deprecated') - if record[5]: - status.append('removed') - if not status or record[3]: - status.append('preview') - - yield (module, {'metadata_version': '1.1', 'supported_by': supported_by, 'status': status}) - - -def write_metadata(filename, new_metadata, version=None, overwrite=False): - with open(filename, 'rb') as f: - module_data = f.read() - - try: - current_metadata, start_line, start_col, end_line, end_col, targets = \ - extract_metadata(module_data=module_data, offsets=True) - except SyntaxError: - if filename.endswith('.py'): - raise - # Probably non-python modules. These should all have python - # documentation files where we can place the data - raise ParseError('Could not add metadata to {0}'.format(filename)) - - if current_metadata is None: - # No current metadata so we can just add it - start_line = find_documentation(module_data) - if start_line < 0: - if os.path.basename(filename) in NONMODULE_PY_FILES: - # These aren't new-style modules - return - - raise Exception('Module file {0} had no ANSIBLE_METADATA or DOCUMENTATION'.format(filename)) - - module_data = insert_metadata(module_data, new_metadata, start_line, targets=('ANSIBLE_METADATA',)) - - elif overwrite or (version is not None and ('metadata_version' not in current_metadata or - StrictVersion(current_metadata['metadata_version']) < StrictVersion(version))): - # Current metadata that we do not want. Remove the current - # metadata and put the new version in its place - module_data = remove_metadata(module_data, start_line, start_col, end_line, end_col) - module_data = insert_metadata(module_data, new_metadata, start_line, targets=targets) - - else: - # Current metadata and we don't want to overwrite it - return - - # Save the new version of the module - with open(filename, 'wb') as f: - f.write(module_data) - - -def return_metadata(plugins): - """Get the metadata for all modules - - Handle duplicate module names - - :arg plugins: List of plugins to look for - :returns: Mapping of plugin name to metadata dictionary - """ - metadata = {} - for name, filename in plugins: - # There may be several files for a module (if it is written in another - # language, for instance) but only one of them (the .py file) should - # contain the metadata. - if name not in metadata or metadata[name] is not None: - with open(filename, 'rb') as f: - module_data = f.read() - metadata[name] = extract_metadata(module_data=module_data, offsets=True)[0] - return metadata - - -def metadata_summary(plugins, version=None): - """Compile information about the metadata status for a list of modules - - :arg plugins: List of plugins to look for. Each entry in the list is - a tuple of (module name, full path to module) - :kwarg version: If given, make sure the modules have this version of - metadata or higher. - :returns: A tuple consisting of a list of modules with no metadata at the - required version and a list of files that have metadata at the - required version. - """ - no_metadata = {} - has_metadata = {} - supported_by = defaultdict(set) - status = defaultdict(set) - requested_version = StrictVersion(version) - - all_mods_metadata = return_metadata(plugins) - for name, filename in plugins: - # Does the module have metadata? - if name not in no_metadata and name not in has_metadata: - metadata = all_mods_metadata[name] - if metadata is None: - no_metadata[name] = filename - elif version is not None and ('metadata_version' not in metadata or StrictVersion(metadata['metadata_version']) < requested_version): - no_metadata[name] = filename - else: - has_metadata[name] = filename - - # What categories does the plugin belong in? - if all_mods_metadata[name] is None: - # No metadata for this module. Use the default metadata - supported_by[DEFAULT_METADATA['supported_by']].add(filename) - status[DEFAULT_METADATA['status'][0]].add(filename) - else: - supported_by[all_mods_metadata[name]['supported_by']].add(filename) - for one_status in all_mods_metadata[name]['status']: - status[one_status].add(filename) - - return list(no_metadata.values()), list(has_metadata.values()), supported_by, status - -# Filters to convert between metadata versions - - -def convert_metadata_pre_1_0_to_1_0(metadata): - """ - Convert pre-1.0 to 1.0 metadata format - - :arg metadata: The old metadata - :returns: The new metadata - - Changes from pre-1.0 to 1.0: - * ``version`` field renamed to ``metadata_version`` - * ``supported_by`` field value ``unmaintained`` has been removed (change to - ``community`` and let an external list track whether a module is unmaintained) - * ``supported_by`` field value ``committer`` has been renamed to ``curated`` - """ - new_metadata = {'metadata_version': '1.0', - 'supported_by': metadata['supported_by'], - 'status': metadata['status'] - } - if new_metadata['supported_by'] == 'unmaintained': - new_metadata['supported_by'] = 'community' - elif new_metadata['supported_by'] == 'committer': - new_metadata['supported_by'] = 'curated' - - return new_metadata - - -def convert_metadata_1_0_to_1_1(metadata): - """ - Convert 1.0 to 1.1 metadata format - - :arg metadata: The old metadata - :returns: The new metadata - - Changes from 1.0 to 1.1: - - * ``supported_by`` field value ``curated`` has been removed - * ``supported_by`` field value ``certified`` has been added - * ``supported_by`` field value ``network`` has been added - """ - new_metadata = {'metadata_version': '1.1', - 'supported_by': metadata['supported_by'], - 'status': metadata['status'] - } - if new_metadata['supported_by'] == 'unmaintained': - new_metadata['supported_by'] = 'community' - elif new_metadata['supported_by'] == 'curated': - new_metadata['supported_by'] = 'certified' - - return new_metadata - -# Subcommands - - -def add_from_csv(csv_file, version=None, overwrite=False): - """Implement the subcommand to add metadata from a csv file - """ - # Add metadata for everything from the CSV file - diagnostic_messages = [] - for module_name, new_metadata in parse_assigned_metadata(csv_file): - filename = module_loader.find_plugin(module_name, mod_type='.py') - if filename is None: - diagnostic_messages.append('Unable to find the module file for {0}'.format(module_name)) - continue - - try: - write_metadata(filename, new_metadata, version, overwrite) - except ParseError as e: - diagnostic_messages.append(e.args[0]) - continue - - if diagnostic_messages: - pprint(diagnostic_messages) - - return 0 - - -def add_default(version=None, overwrite=False): - """Implement the subcommand to add default metadata to modules - - Add the default metadata to any plugin which lacks it. - :kwarg version: If given, the metadata must be at least this version. - Otherwise, treat the module as not having existing metadata. - :kwarg overwrite: If True, overwrite any existing metadata. Otherwise, - do not modify files which have metadata at an appropriate version - """ - # List of all plugins - plugins = module_loader.all(path_only=True) - plugins = ((os.path.splitext((os.path.basename(p)))[0], p) for p in plugins) - plugins = (p for p in plugins if p[0] not in NONMODULE_MODULE_NAMES) - - # Iterate through each plugin - processed = set() - diagnostic_messages = [] - for name, filename in (info for info in plugins if info[0] not in processed): - try: - write_metadata(filename, DEFAULT_METADATA, version, overwrite) - except ParseError as e: - diagnostic_messages.append(e.args[0]) - continue - processed.add(name) - - if diagnostic_messages: - pprint(diagnostic_messages) - - return 0 - - -def upgrade_metadata(version=None): - """Implement the subcommand to upgrade the default metadata in modules. - - :kwarg version: If given, the version of the metadata to upgrade to. If - not given, upgrade to the latest format version. - """ - if version is None: - # Number larger than any of the defined metadata formats. - version = 9999999 - requested_version = StrictVersion(version) - - # List all plugins - plugins = module_loader.all(path_only=True) - plugins = ((os.path.splitext((os.path.basename(p)))[0], p) for p in plugins) - plugins = (p for p in plugins if p[0] not in NONMODULE_MODULE_NAMES) - - processed = set() - diagnostic_messages = [] - for name, filename in (info for info in plugins if info[0] not in processed): - # For each plugin, read the existing metadata - with open(filename, 'rb') as f: - module_data = f.read() - metadata = extract_metadata(module_data=module_data, offsets=True)[0] - - # If the metadata isn't the requested version, convert it to the new - # version - if 'metadata_version' not in metadata or metadata['metadata_version'] != version: - # - # With each iteration of metadata, add a new conditional to - # upgrade from the previous version - # - - if 'metadata_version' not in metadata: - # First version, pre-1.0 final metadata - metadata = convert_metadata_pre_1_0_to_1_0(metadata) - - if metadata['metadata_version'] == '1.0' and StrictVersion('1.0') < requested_version: - metadata = convert_metadata_1_0_to_1_1(metadata) - - if metadata['metadata_version'] == '1.1' and StrictVersion('1.1') < requested_version: - # 1.1 version => XXX. We don't yet have anything beyond 1.1 - # so there's nothing here - pass - - # Replace the existing metadata with the new format - try: - write_metadata(filename, metadata, version, overwrite=True) - except ParseError as e: - diagnostic_messages.append(e.args[0]) - continue - - processed.add(name) - - if diagnostic_messages: - pprint(diagnostic_messages) - - return 0 - - -def report(version=None): - """Implement the report subcommand - - Print out all the modules that have metadata and all the ones that do not. - - :kwarg version: If given, the metadata must be at least this version. - Otherwise return it as not having metadata - """ - # List of all plugins - plugins = module_loader.all(path_only=True) - plugins = ((os.path.splitext((os.path.basename(p)))[0], p) for p in plugins) - plugins = (p for p in plugins if p[0] not in NONMODULE_MODULE_NAMES) - plugins = list(plugins) - - no_metadata, has_metadata, support, status = metadata_summary(plugins, version=version) - - print('== Has metadata ==') - pprint(sorted(has_metadata)) - print('') - - print('== Has no metadata ==') - pprint(sorted(no_metadata)) - print('') - - print('== Supported by core ==') - pprint(sorted(support['core'])) - print('== Supported by value certified ==') - pprint(sorted(support['certified'])) - print('== Supported by value network ==') - pprint(sorted(support['network'])) - print('== Supported by community ==') - pprint(sorted(support['community'])) - print('') - - print('== Status: stableinterface ==') - pprint(sorted(status['stableinterface'])) - print('== Status: preview ==') - pprint(sorted(status['preview'])) - print('== Status: deprecated ==') - pprint(sorted(status['deprecated'])) - print('== Status: removed ==') - pprint(sorted(status['removed'])) - print('') - - print('== Summary ==') - print('No Metadata: {0} Has Metadata: {1}'.format(len(no_metadata), len(has_metadata))) - print('Support level: core: {0} community: {1} certified: {2} network: {3}'.format(len(support['core']), - len(support['community']), len(support['certified']), len(support['network']))) - print('Status StableInterface: {0} Status Preview: {1} Status Deprecated: {2} Status Removed: {3}'.format(len(status['stableinterface']), - len(status['preview']), len(status['deprecated']), len(status['removed']))) - - return 0 - - -if __name__ == '__main__': - action, args = parse_args(sys.argv[1:]) - - if action == 'report': - rc = report(version=args['version']) - elif action == 'add': - rc = add_from_csv(args['csvfile'], version=args['version'], overwrite=args['overwrite']) - elif action == 'add-default': - rc = add_default(version=args['version'], overwrite=args['overwrite']) - elif action == 'upgrade': - rc = upgrade_metadata(version=args['version']) - - sys.exit(rc) diff --git a/hacking/report.py b/hacking/report.py index 9c5d52d6dd..580d87dd00 100755 --- a/hacking/report.py +++ b/hacking/report.py @@ -9,7 +9,6 @@ import json import os import sqlite3 import sys -import yaml DATABASE_PATH = os.path.expanduser('~/.ansible/report.db') BASE_PATH = os.path.abspath(os.path.join(os.path.dirname(os.path.abspath(__file__)), '..')) + '/' @@ -81,7 +80,6 @@ def populate_modules(): module_dir = os.path.join(BASE_PATH, 'lib/ansible/modules/') modules_rows = [] - module_statuses_rows = [] for root, dir_names, file_names in os.walk(module_dir): for file_name in file_names: @@ -99,29 +97,15 @@ def populate_modules(): result = read_docstring(path) - metadata = result['metadata'] doc = result['doc'] - if not metadata: - if module == 'async_wrapper': - continue - - raise Exception('no metadata for: %s' % path) - modules_rows.append(dict( module=module, namespace=namespace, path=path.replace(BASE_PATH, ''), - supported_by=metadata['supported_by'], version_added=str(doc.get('version_added', '')) if doc else '', )) - for status in metadata['status']: - module_statuses_rows.append(dict( - module=module, - status=status, - )) - populate_data(dict( modules=dict( rows=modules_rows, @@ -129,15 +113,8 @@ def populate_modules(): ('module', 'TEXT'), ('namespace', 'TEXT'), ('path', 'TEXT'), - ('supported_by', 'TEXT'), ('version_added', 'TEXT'), )), - module_statuses=dict( - rows=module_statuses_rows, - schema=( - ('module', 'TEXT'), - ('status', 'TEXT'), - )), )) diff --git a/lib/ansible/cli/doc.py b/lib/ansible/cli/doc.py index 94992a6628..dec582fb98 100644 --- a/lib/ansible/cli/doc.py +++ b/lib/ansible/cli/doc.py @@ -24,7 +24,6 @@ from ansible.module_utils._text import to_native, to_text from ansible.module_utils.common._collections_compat import Container, Sequence from ansible.module_utils.common.json import AnsibleJSONEncoder from ansible.module_utils.six import string_types -from ansible.parsing.metadata import extract_metadata from ansible.parsing.plugin_docs import read_docstub from ansible.parsing.yaml.dumper import AnsibleDumper from ansible.plugins.loader import action_loader, fragment_loader @@ -54,10 +53,6 @@ def add_collection_plugins(plugin_list, plugin_type, coll_filter=None): plugin_list.update(DocCLI.find_plugins(os.path.join(path, 'plugins', ptype), plugin_type, collection=collname)) -class RemovedPlugin(Exception): - pass - - class PluginNotFound(Exception): pass @@ -112,12 +107,45 @@ class DocCLI(CLI): return options + def display_plugin_list(self, results): + + # format for user + displace = max(len(x) for x in self.plugin_list) + linelimit = display.columns - displace - 5 + text = [] + + # format display per option + if context.CLIARGS['list_files']: + # list plugin file names + for plugin in results.keys(): + filename = results[plugin] + text.append("%-*s %-*.*s" % (displace, plugin, linelimit, len(filename), filename)) + else: + # list plugin names and short desc + deprecated = [] + for plugin in results.keys(): + desc = DocCLI.tty_ify(results[plugin]) + + if len(desc) > linelimit: + desc = desc[:linelimit] + '...' + + if plugin.startswith('_'): # Handle deprecated # TODO: add mark for deprecated collection plugins + deprecated.append("%-*s %-*.*s" % (displace, plugin[1:], linelimit, len(desc), desc)) + else: + text.append("%-*s %-*.*s" % (displace, plugin, linelimit, len(desc), desc)) + + if len(deprecated) > 0: + text.append("\nDEPRECATED:") + text.extend(deprecated) + + # display results + DocCLI.pager("\n".join(text)) + def run(self): super(DocCLI, self).run() plugin_type = context.CLIARGS['type'] - do_json = context.CLIARGS['json_format'] if plugin_type in C.DOCUMENTABLE_PLUGINS: @@ -130,6 +158,7 @@ class DocCLI(CLI): if basedir: AnsibleCollectionConfig.playbook_paths = basedir loader.add_directory(basedir, with_subdir=True) + if context.CLIARGS['module_path']: for path in context.CLIARGS['module_path']: if path: @@ -162,43 +191,10 @@ class DocCLI(CLI): if do_json: jdump(results) elif self.plugin_list: - # format for user - displace = max(len(x) for x in self.plugin_list) - linelimit = display.columns - displace - 5 - text = [] - - # format display per option - if context.CLIARGS['list_files']: - # list files - - for plugin in results.keys(): - - filename = results[plugin] - text.append("%-*s %-*.*s" % (displace, plugin, linelimit, len(filename), filename)) - else: - # list plugins - deprecated = [] - for plugin in results.keys(): - desc = DocCLI.tty_ify(results[plugin]) - - if len(desc) > linelimit: - desc = desc[:linelimit] + '...' - - if plugin.startswith('_'): # Handle deprecated # TODO: add mark for deprecated collection plugins - deprecated.append("%-*s %-*.*s" % (displace, plugin[1:], linelimit, len(desc), desc)) - else: - text.append("%-*s %-*.*s" % (displace, plugin, linelimit, len(desc), desc)) - - if len(deprecated) > 0: - text.append("\nDEPRECATED:") - text.extend(deprecated) - - # display results - DocCLI.pager("\n".join(text)) + self.display_plugin_list(results) else: display.warning("No plugins found.") - - # dump plugin desc/metadata as JSON + # dump plugin desc/data as JSON elif context.CLIARGS['dump']: plugin_data = {} plugin_names = DocCLI.get_all_plugins_of_type(plugin_type) @@ -208,7 +204,6 @@ class DocCLI(CLI): plugin_data[plugin_name] = plugin_info jdump(plugin_data) - else: # display specific plugin docs if len(context.CLIARGS['args']) == 0: @@ -222,9 +217,6 @@ class DocCLI(CLI): except PluginNotFound: display.warning("%s %s not found in:\n%s\n" % (plugin_type, plugin, search_paths)) continue - except RemovedPlugin: - display.warning("%s %s has been removed\n" % (plugin_type, plugin)) - continue except Exception as e: display.vvv(traceback.format_exc()) raise AnsibleError("%s %s missing documentation (or could not parse" @@ -235,8 +227,7 @@ class DocCLI(CLI): # The doc section existed but was empty continue - plugin_docs[plugin] = {'doc': doc, 'examples': plainexamples, - 'return': returndocs, 'metadata': metadata} + plugin_docs[plugin] = {'doc': doc, 'examples': plainexamples, 'return': returndocs, 'metadata': metadata} if do_json: # Some changes to how json docs are formatted @@ -257,6 +248,8 @@ class DocCLI(CLI): doc_data['return'], doc_data['metadata']) if textret: text.append(textret) + else: + display.warning("No valid documentation was retrieved from '%s'" % plugin) if text: DocCLI.pager(''.join(text)) @@ -286,20 +279,13 @@ class DocCLI(CLI): collection_name = '.'.join(plugin_name.split('.')[1:3]) try: - doc, __, __, metadata = get_docstring(filename, fragment_loader, verbose=(context.CLIARGS['verbosity'] > 0), - collection_name=collection_name, is_module=(plugin_type == 'module')) + doc, __, __, __ = get_docstring(filename, fragment_loader, verbose=(context.CLIARGS['verbosity'] > 0), + collection_name=collection_name, is_module=(plugin_type == 'module')) except Exception: display.vvv(traceback.format_exc()) - raise AnsibleError( - "%s %s at %s has a documentation error formatting or is missing documentation." % - (plugin_type, plugin_name, filename)) + raise AnsibleError("%s %s at %s has a documentation formatting error or is missing documentation." % (plugin_type, plugin_name, filename)) if doc is None: - if 'removed' not in metadata.get('status', []): - raise AnsibleError( - "%s %s at %s has a documentation error formatting or is missing documentation." % - (plugin_type, plugin_name, filename)) - # Removed plugins don't have any documentation return None @@ -339,22 +325,8 @@ class DocCLI(CLI): filename, fragment_loader, verbose=(context.CLIARGS['verbosity'] > 0), collection_name=collection_name, is_module=(plugin_type == 'module')) - # If the plugin existed but did not have a DOCUMENTATION element and was not removed, it's - # an error + # If the plugin existed but did not have a DOCUMENTATION element and was not removed, it's an error if doc is None: - # doc may be None when the module has been removed. Calling code may choose to - # handle that but we can't. - if 'status' in metadata and isinstance(metadata['status'], Container): - if 'removed' in metadata['status']: - raise RemovedPlugin('%s has been removed' % plugin) - - # Backwards compat: no documentation but valid metadata (or no metadata, which results in using the default metadata). - # Probably should make this an error in 2.10 - return {}, {}, {}, metadata - else: - # If metadata is invalid, warn but don't error - display.warning(u'%s has an invalid ANSIBLE_METADATA field' % plugin) - raise ValueError('%s did not contain a DOCUMENTATION attribute' % plugin) doc['filename'] = filename @@ -383,7 +355,10 @@ class DocCLI(CLI): if context.CLIARGS['show_snippet'] and plugin_type == 'module': text = DocCLI.get_snippet_text(doc) else: - text = DocCLI.get_man_text(doc) + try: + text = DocCLI.get_man_text(doc) + except Exception as e: + raise AnsibleError("Unable to retrieve documentation from '%s' due to: %s" % (plugin, to_native(e))) return text @@ -450,13 +425,6 @@ class DocCLI(CLI): continue if not doc or not isinstance(doc, dict): - with open(filename) as f: - metadata = extract_metadata(module_data=f.read()) - if metadata[0]: - if 'removed' not in metadata[0].get('status', []): - display.warning("%s parsing did not produce documentation." % plugin) - else: - continue desc = 'UNDOCUMENTED' else: desc = doc.get('short_description', 'INVALID SHORT DESCRIPTION').strip() @@ -564,16 +532,16 @@ class DocCLI(CLI): aliases = '' if 'aliases' in opt: if len(opt['aliases']) > 0: - aliases = "(Aliases: " + ", ".join(str(i) for i in opt['aliases']) + ")" + aliases = "(Aliases: " + ", ".join(to_text(i) for i in opt['aliases']) + ")" del opt['aliases'] choices = '' if 'choices' in opt: if len(opt['choices']) > 0: - choices = "(Choices: " + ", ".join(str(i) for i in opt['choices']) + ")" + choices = "(Choices: " + ", ".join(to_text(i) for i in opt['choices']) + ")" del opt['choices'] default = '' if 'default' in opt or not required: - default = "[Default: %s" % str(opt.pop('default', '(null)')) + "]" + default = "[Default: %s" % to_text(opt.pop('default', '(null)')) + "]" text.append(textwrap.fill(DocCLI.tty_ify(aliases + choices + default), limit, initial_indent=opt_indent, subsequent_indent=opt_indent)) @@ -613,31 +581,6 @@ class DocCLI(CLI): text.append('') @staticmethod - def get_support_block(doc): - # Note: 'curated' is deprecated and not used in any of the modules we ship - support_level_msg = {'core': 'The Ansible Core Team', - 'network': 'The Ansible Network Team', - 'certified': 'an Ansible Partner', - 'community': 'The Ansible Community', - 'curated': 'A Third Party', - } - return [" * This module is maintained by %s" % support_level_msg[doc['metadata']['supported_by']]] - - @staticmethod - def get_metadata_block(doc): - text = [] - - text.append("METADATA:") - text.append('\tSUPPORT LEVEL: %s' % doc['metadata']['supported_by']) - - for k in (m for m in doc['metadata'] if m != 'supported_by'): - if isinstance(k, list): - text.append("\t%s: %s" % (k.capitalize(), ", ".join(doc['metadata'][k]))) - else: - text.append("\t%s: %s" % (k.capitalize(), doc['metadata'][k])) - return text - - @staticmethod def get_man_text(doc): DocCLI.IGNORE = DocCLI.IGNORE + (context.CLIARGS['type'],) @@ -656,7 +599,7 @@ class DocCLI(CLI): text.append("%s\n" % textwrap.fill(DocCLI.tty_ify(desc), limit, initial_indent=opt_indent, subsequent_indent=opt_indent)) - if 'deprecated' in doc and doc['deprecated'] is not None and len(doc['deprecated']) > 0: + if doc.get('deprecated', False): text.append("DEPRECATED: \n") if isinstance(doc['deprecated'], dict): if 'version' in doc['deprecated'] and 'removed_in' not in doc['deprecated']: @@ -666,22 +609,15 @@ class DocCLI(CLI): text.append("%s" % doc.pop('deprecated')) text.append("\n") - try: - support_block = DocCLI.get_support_block(doc) - if support_block: - text.extend(support_block) - except Exception: - pass # FIXME: not suported by plugins - if doc.pop('action', False): text.append(" * note: %s\n" % "This module has a corresponding action plugin.") - if 'options' in doc and doc['options']: + if doc.get('options', False): text.append("OPTIONS (= is mandatory):\n") DocCLI.add_fields(text, doc.pop('options'), limit, opt_indent) text.append('') - if 'notes' in doc and doc['notes'] and len(doc['notes']) > 0: + if doc.get('notes', False): text.append("NOTES:") for note in doc['notes']: text.append(textwrap.fill(DocCLI.tty_ify(note), limit - 6, @@ -690,7 +626,7 @@ class DocCLI(CLI): text.append('') del doc['notes'] - if 'seealso' in doc and doc['seealso']: + if doc.get('seealso', False): text.append("SEE ALSO:") for item in doc['seealso']: if 'module' in item: @@ -719,7 +655,7 @@ class DocCLI(CLI): text.append('') del doc['seealso'] - if 'requirements' in doc and doc['requirements'] is not None and len(doc['requirements']) > 0: + if doc.get('requirements', False): req = ", ".join(doc.pop('requirements')) text.append("REQUIREMENTS:%s\n" % textwrap.fill(DocCLI.tty_ify(req), limit - 16, initial_indent=" ", subsequent_indent=opt_indent)) @@ -732,11 +668,12 @@ class DocCLI(CLI): elif isinstance(doc[k], (list, tuple)): text.append('%s: %s' % (k.upper(), ', '.join(doc[k]))) else: - text.append(DocCLI._dump_yaml({k.upper(): doc[k]}, opt_indent)) + # use empty indent since this affects the start of the yaml doc, not it's keys + text.append(DocCLI._dump_yaml({k.upper(): doc[k]}, '')) del doc[k] - text.append('') + text.append('') - if 'plainexamples' in doc and doc['plainexamples'] is not None: + if doc.get('plainexamples', False): text.append("EXAMPLES:") text.append('') if isinstance(doc['plainexamples'], string_types): @@ -746,20 +683,11 @@ class DocCLI(CLI): text.append('') text.append('') - if 'returndocs' in doc and doc['returndocs'] is not None: + if doc.get('returndocs', False): text.append("RETURN VALUES:") if isinstance(doc['returndocs'], string_types): text.append(doc.pop('returndocs')) else: text.append(yaml.dump(doc.pop('returndocs'), indent=2, default_flow_style=False)) - text.append('') - - try: - metadata_block = DocCLI.get_metadata_block(doc) - if metadata_block: - text.extend(metadata_block) - text.append('') - except Exception: - pass # metadata is optional return "\n".join(text) diff --git a/lib/ansible/galaxy/data/network/library/example_command.py.j2 b/lib/ansible/galaxy/data/network/library/example_command.py.j2 index 8c8594bd25..0f3dac2d98 100644 --- a/lib/ansible/galaxy/data/network/library/example_command.py.j2 +++ b/lib/ansible/galaxy/data/network/library/example_command.py.j2 @@ -22,11 +22,6 @@ __metaclass__ = type ### Documentation -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['preview'], - 'supported_by': 'community'} - - DOCUMENTATION = """ Examples: https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/network/iosxr/iosxr_command.py diff --git a/lib/ansible/galaxy/data/network/library/example_config.py.j2 b/lib/ansible/galaxy/data/network/library/example_config.py.j2 index 15c29aff2e..2c2c72be9c 100644 --- a/lib/ansible/galaxy/data/network/library/example_config.py.j2 +++ b/lib/ansible/galaxy/data/network/library/example_config.py.j2 @@ -22,11 +22,6 @@ __metaclass__ = type ### Documentation -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['preview'], - 'supported_by': 'community'} - - DOCUMENTATION = """ Examples: https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/network/iosxr/iosxr_config.py diff --git a/lib/ansible/galaxy/data/network/library/example_facts.py.j2 b/lib/ansible/galaxy/data/network/library/example_facts.py.j2 index ecff927f39..9f7608c39a 100644 --- a/lib/ansible/galaxy/data/network/library/example_facts.py.j2 +++ b/lib/ansible/galaxy/data/network/library/example_facts.py.j2 @@ -22,11 +22,6 @@ __metaclass__ = type ### Documentation -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['preview'], - 'supported_by': 'community'} - - DOCUMENTATION = """ Examples: https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/network/iosxr/iosxr_facts.py diff --git a/lib/ansible/modules/add_host.py b/lib/ansible/modules/add_host.py index 5b125ad3cb..99e62464d4 100644 --- a/lib/ansible/modules/add_host.py +++ b/lib/ansible/modules/add_host.py @@ -7,9 +7,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['stableinterface'], - 'supported_by': 'core'} DOCUMENTATION = r''' --- diff --git a/lib/ansible/modules/apt.py b/lib/ansible/modules/apt.py index bee314e601..d90d12dca6 100644 --- a/lib/ansible/modules/apt.py +++ b/lib/ansible/modules/apt.py @@ -10,9 +10,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['stableinterface'], - 'supported_by': 'core'} DOCUMENTATION = ''' --- diff --git a/lib/ansible/modules/apt_key.py b/lib/ansible/modules/apt_key.py index 425193a5a5..d8bb6e1543 100644 --- a/lib/ansible/modules/apt_key.py +++ b/lib/ansible/modules/apt_key.py @@ -9,9 +9,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['preview'], - 'supported_by': 'core'} DOCUMENTATION = ''' --- diff --git a/lib/ansible/modules/apt_repository.py b/lib/ansible/modules/apt_repository.py index db14521665..834bdec1dd 100644 --- a/lib/ansible/modules/apt_repository.py +++ b/lib/ansible/modules/apt_repository.py @@ -10,9 +10,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['preview'], - 'supported_by': 'core'} DOCUMENTATION = ''' --- diff --git a/lib/ansible/modules/assemble.py b/lib/ansible/modules/assemble.py index 078262608f..1940ac6101 100644 --- a/lib/ansible/modules/assemble.py +++ b/lib/ansible/modules/assemble.py @@ -9,9 +9,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['stableinterface'], - 'supported_by': 'core'} DOCUMENTATION = r''' --- diff --git a/lib/ansible/modules/assert.py b/lib/ansible/modules/assert.py index 9d251262d0..eb6b7a76f8 100644 --- a/lib/ansible/modules/assert.py +++ b/lib/ansible/modules/assert.py @@ -7,9 +7,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['stableinterface'], - 'supported_by': 'core'} DOCUMENTATION = r''' --- diff --git a/lib/ansible/modules/async_status.py b/lib/ansible/modules/async_status.py index db42869957..158e771d25 100644 --- a/lib/ansible/modules/async_status.py +++ b/lib/ansible/modules/async_status.py @@ -7,9 +7,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['stableinterface'], - 'supported_by': 'core'} DOCUMENTATION = r''' --- diff --git a/lib/ansible/modules/blockinfile.py b/lib/ansible/modules/blockinfile.py index f5b422c6bb..ecb06d45c3 100644 --- a/lib/ansible/modules/blockinfile.py +++ b/lib/ansible/modules/blockinfile.py @@ -8,9 +8,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['preview'], - 'supported_by': 'core'} DOCUMENTATION = r''' --- diff --git a/lib/ansible/modules/command.py b/lib/ansible/modules/command.py index 2ebf7f743a..5844c2d849 100644 --- a/lib/ansible/modules/command.py +++ b/lib/ansible/modules/command.py @@ -9,10 +9,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['stableinterface'], - 'supported_by': 'core'} - DOCUMENTATION = r''' --- module: command diff --git a/lib/ansible/modules/copy.py b/lib/ansible/modules/copy.py index 382ac376fd..b6cd8c681f 100644 --- a/lib/ansible/modules/copy.py +++ b/lib/ansible/modules/copy.py @@ -8,9 +8,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['stableinterface'], - 'supported_by': 'core'} DOCUMENTATION = r''' --- diff --git a/lib/ansible/modules/cron.py b/lib/ansible/modules/cron.py index 0e102b0387..6a59acb6f7 100644 --- a/lib/ansible/modules/cron.py +++ b/lib/ansible/modules/cron.py @@ -11,9 +11,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['preview'], - 'supported_by': 'core'} DOCUMENTATION = r''' --- diff --git a/lib/ansible/modules/debconf.py b/lib/ansible/modules/debconf.py index eaeae1a39d..559b4f2b9c 100644 --- a/lib/ansible/modules/debconf.py +++ b/lib/ansible/modules/debconf.py @@ -7,9 +7,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['stableinterface'], - 'supported_by': 'core'} DOCUMENTATION = r''' --- diff --git a/lib/ansible/modules/debug.py b/lib/ansible/modules/debug.py index 8a3f0caa52..1a140eb2bf 100644 --- a/lib/ansible/modules/debug.py +++ b/lib/ansible/modules/debug.py @@ -7,9 +7,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['stableinterface'], - 'supported_by': 'core'} DOCUMENTATION = r''' --- diff --git a/lib/ansible/modules/dnf.py b/lib/ansible/modules/dnf.py index aaf404ed73..226703a7f4 100644 --- a/lib/ansible/modules/dnf.py +++ b/lib/ansible/modules/dnf.py @@ -11,11 +11,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['stableinterface'], - 'supported_by': 'core'} - - DOCUMENTATION = ''' --- module: dnf diff --git a/lib/ansible/modules/dpkg_selections.py b/lib/ansible/modules/dpkg_selections.py index 2e35878ba3..9043786ece 100644 --- a/lib/ansible/modules/dpkg_selections.py +++ b/lib/ansible/modules/dpkg_selections.py @@ -8,11 +8,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['preview'], - 'supported_by': 'core'} - - DOCUMENTATION = ''' --- module: dpkg_selections diff --git a/lib/ansible/modules/expect.py b/lib/ansible/modules/expect.py index 6a0d6b7749..f1b2ec6ff9 100644 --- a/lib/ansible/modules/expect.py +++ b/lib/ansible/modules/expect.py @@ -8,11 +8,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['preview'], - 'supported_by': 'core'} - - DOCUMENTATION = r''' --- module: expect diff --git a/lib/ansible/modules/fail.py b/lib/ansible/modules/fail.py index 149049df8f..ed0a013a3f 100644 --- a/lib/ansible/modules/fail.py +++ b/lib/ansible/modules/fail.py @@ -7,9 +7,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['stableinterface'], - 'supported_by': 'core'} DOCUMENTATION = r''' --- diff --git a/lib/ansible/modules/fetch.py b/lib/ansible/modules/fetch.py index d87ec1274f..959b992ddc 100644 --- a/lib/ansible/modules/fetch.py +++ b/lib/ansible/modules/fetch.py @@ -9,9 +9,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['stableinterface'], - 'supported_by': 'core'} DOCUMENTATION = r''' --- diff --git a/lib/ansible/modules/file.py b/lib/ansible/modules/file.py index ee96657234..ba5d4bd7fa 100644 --- a/lib/ansible/modules/file.py +++ b/lib/ansible/modules/file.py @@ -8,9 +8,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['stableinterface'], - 'supported_by': 'core'} DOCUMENTATION = r''' --- diff --git a/lib/ansible/modules/find.py b/lib/ansible/modules/find.py index e9ac43edfe..2288df7845 100644 --- a/lib/ansible/modules/find.py +++ b/lib/ansible/modules/find.py @@ -10,9 +10,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['stableinterface'], - 'supported_by': 'core'} DOCUMENTATION = r''' --- diff --git a/lib/ansible/modules/gather_facts.py b/lib/ansible/modules/gather_facts.py index 711c840a98..756093597c 100644 --- a/lib/ansible/modules/gather_facts.py +++ b/lib/ansible/modules/gather_facts.py @@ -7,11 +7,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['preview'], - 'supported_by': 'core'} - - DOCUMENTATION = ''' --- module: gather_facts diff --git a/lib/ansible/modules/get_url.py b/lib/ansible/modules/get_url.py index d9f02217b4..7a48067c3c 100644 --- a/lib/ansible/modules/get_url.py +++ b/lib/ansible/modules/get_url.py @@ -7,9 +7,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['stableinterface'], - 'supported_by': 'core'} DOCUMENTATION = r''' --- diff --git a/lib/ansible/modules/getent.py b/lib/ansible/modules/getent.py index 29337b0cfb..1997aa0d70 100644 --- a/lib/ansible/modules/getent.py +++ b/lib/ansible/modules/getent.py @@ -7,9 +7,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['stableinterface'], - 'supported_by': 'core'} DOCUMENTATION = ''' --- diff --git a/lib/ansible/modules/git.py b/lib/ansible/modules/git.py index 4b4d740d7f..b47e94517b 100644 --- a/lib/ansible/modules/git.py +++ b/lib/ansible/modules/git.py @@ -8,11 +8,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['preview'], - 'supported_by': 'core'} - - DOCUMENTATION = ''' --- module: git diff --git a/lib/ansible/modules/group.py b/lib/ansible/modules/group.py index 4c6d0b99a3..006e38185b 100644 --- a/lib/ansible/modules/group.py +++ b/lib/ansible/modules/group.py @@ -7,9 +7,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['stableinterface'], - 'supported_by': 'core'} DOCUMENTATION = ''' --- diff --git a/lib/ansible/modules/group_by.py b/lib/ansible/modules/group_by.py index 5f43eea32d..3ef3b51114 100644 --- a/lib/ansible/modules/group_by.py +++ b/lib/ansible/modules/group_by.py @@ -7,9 +7,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['stableinterface'], - 'supported_by': 'core'} DOCUMENTATION = r''' --- diff --git a/lib/ansible/modules/hostname.py b/lib/ansible/modules/hostname.py index 1669a64357..c0ffe37adf 100644 --- a/lib/ansible/modules/hostname.py +++ b/lib/ansible/modules/hostname.py @@ -7,9 +7,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['preview'], - 'supported_by': 'core'} DOCUMENTATION = ''' --- diff --git a/lib/ansible/modules/import_playbook.py b/lib/ansible/modules/import_playbook.py index 17ce2c9161..25ba1a7354 100644 --- a/lib/ansible/modules/import_playbook.py +++ b/lib/ansible/modules/import_playbook.py @@ -7,11 +7,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = { - 'metadata_version': '1.1', - 'status': ['preview'], - 'supported_by': 'core' -} DOCUMENTATION = r''' --- diff --git a/lib/ansible/modules/import_role.py b/lib/ansible/modules/import_role.py index baca26f20b..eb4107047a 100644 --- a/lib/ansible/modules/import_role.py +++ b/lib/ansible/modules/import_role.py @@ -6,11 +6,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = { - 'metadata_version': '1.1', - 'status': ['stableinterface'], - 'supported_by': 'core' -} DOCUMENTATION = r''' --- diff --git a/lib/ansible/modules/import_tasks.py b/lib/ansible/modules/import_tasks.py index 3d54a7c549..e3884c75b1 100644 --- a/lib/ansible/modules/import_tasks.py +++ b/lib/ansible/modules/import_tasks.py @@ -7,11 +7,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = { - 'metadata_version': '1.1', - 'status': ['stableinterface'], - 'supported_by': 'core' -} DOCUMENTATION = r''' --- diff --git a/lib/ansible/modules/include.py b/lib/ansible/modules/include.py index cde31150fe..75b816057d 100644 --- a/lib/ansible/modules/include.py +++ b/lib/ansible/modules/include.py @@ -7,11 +7,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = { - 'metadata_version': '1.1', - 'status': ['preview'], - 'supported_by': 'core' -} DOCUMENTATION = r''' --- diff --git a/lib/ansible/modules/include_role.py b/lib/ansible/modules/include_role.py index 722251cb8e..10c1d83c25 100644 --- a/lib/ansible/modules/include_role.py +++ b/lib/ansible/modules/include_role.py @@ -7,11 +7,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = { - 'metadata_version': '1.1', - 'status': ['stableinterface'], - 'supported_by': 'core' -} DOCUMENTATION = r''' --- diff --git a/lib/ansible/modules/include_tasks.py b/lib/ansible/modules/include_tasks.py index f55351b399..fad798d311 100644 --- a/lib/ansible/modules/include_tasks.py +++ b/lib/ansible/modules/include_tasks.py @@ -7,11 +7,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = { - 'metadata_version': '1.1', - 'status': ['stableinterface'], - 'supported_by': 'core' -} DOCUMENTATION = r''' --- diff --git a/lib/ansible/modules/include_vars.py b/lib/ansible/modules/include_vars.py index 58d1f00bf2..24f4359e94 100644 --- a/lib/ansible/modules/include_vars.py +++ b/lib/ansible/modules/include_vars.py @@ -6,11 +6,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = { - 'metadata_version': '1.1', - 'status': ['stableinterface'], - 'supported_by': 'core' -} DOCUMENTATION = r''' --- diff --git a/lib/ansible/modules/iptables.py b/lib/ansible/modules/iptables.py index e6973689ed..7ec22cfd88 100644 --- a/lib/ansible/modules/iptables.py +++ b/lib/ansible/modules/iptables.py @@ -8,9 +8,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['preview'], - 'supported_by': 'core'} DOCUMENTATION = r''' --- diff --git a/lib/ansible/modules/known_hosts.py b/lib/ansible/modules/known_hosts.py index 235019b018..858c6ba8d0 100644 --- a/lib/ansible/modules/known_hosts.py +++ b/lib/ansible/modules/known_hosts.py @@ -7,11 +7,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['preview'], - 'supported_by': 'core'} - - DOCUMENTATION = ''' --- module: known_hosts diff --git a/lib/ansible/modules/lineinfile.py b/lib/ansible/modules/lineinfile.py index 5860f4d6e7..1fa50c881d 100644 --- a/lib/ansible/modules/lineinfile.py +++ b/lib/ansible/modules/lineinfile.py @@ -9,10 +9,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['preview'], - 'supported_by': 'core'} - DOCUMENTATION = r''' --- diff --git a/lib/ansible/modules/meta.py b/lib/ansible/modules/meta.py index 2d683f0217..706ab9c4b9 100644 --- a/lib/ansible/modules/meta.py +++ b/lib/ansible/modules/meta.py @@ -7,9 +7,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['preview'], - 'supported_by': 'core'} DOCUMENTATION = r''' module: meta diff --git a/lib/ansible/modules/package.py b/lib/ansible/modules/package.py index 24ac099f20..f709d3e890 100644 --- a/lib/ansible/modules/package.py +++ b/lib/ansible/modules/package.py @@ -9,11 +9,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['stableinterface'], - 'supported_by': 'core'} - - DOCUMENTATION = ''' --- module: package diff --git a/lib/ansible/modules/package_facts.py b/lib/ansible/modules/package_facts.py index 60fb566988..e957e8300b 100644 --- a/lib/ansible/modules/package_facts.py +++ b/lib/ansible/modules/package_facts.py @@ -7,9 +7,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['preview'], - 'supported_by': 'core'} DOCUMENTATION = ''' module: package_facts diff --git a/lib/ansible/modules/pause.py b/lib/ansible/modules/pause.py index 3b2b0e3799..ac9d01fc34 100644 --- a/lib/ansible/modules/pause.py +++ b/lib/ansible/modules/pause.py @@ -6,11 +6,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['stableinterface'], - 'supported_by': 'core'} - - DOCUMENTATION = ''' --- module: pause diff --git a/lib/ansible/modules/ping.py b/lib/ansible/modules/ping.py index 460f2a7aee..a2472cbfe7 100644 --- a/lib/ansible/modules/ping.py +++ b/lib/ansible/modules/ping.py @@ -9,10 +9,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['stableinterface'], - 'supported_by': 'core'} - DOCUMENTATION = ''' --- module: ping diff --git a/lib/ansible/modules/pip.py b/lib/ansible/modules/pip.py index 80b02ef010..97ae7d901f 100644 --- a/lib/ansible/modules/pip.py +++ b/lib/ansible/modules/pip.py @@ -7,9 +7,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['preview'], - 'supported_by': 'core'} DOCUMENTATION = ''' --- diff --git a/lib/ansible/modules/raw.py b/lib/ansible/modules/raw.py index 928350dbc6..d9c9cbbc40 100644 --- a/lib/ansible/modules/raw.py +++ b/lib/ansible/modules/raw.py @@ -7,11 +7,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['stableinterface'], - 'supported_by': 'core'} - - DOCUMENTATION = r''' --- module: raw diff --git a/lib/ansible/modules/reboot.py b/lib/ansible/modules/reboot.py index 0fab23a609..dea7823ed2 100644 --- a/lib/ansible/modules/reboot.py +++ b/lib/ansible/modules/reboot.py @@ -6,9 +6,6 @@ from __future__ import (absolute_import, division, print_function) __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['preview'], - 'supported_by': 'core'} DOCUMENTATION = r''' module: reboot diff --git a/lib/ansible/modules/replace.py b/lib/ansible/modules/replace.py index d10da8049e..560a5447d0 100644 --- a/lib/ansible/modules/replace.py +++ b/lib/ansible/modules/replace.py @@ -8,10 +8,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['stableinterface'], - 'supported_by': 'core'} - DOCUMENTATION = r''' --- diff --git a/lib/ansible/modules/rpm_key.py b/lib/ansible/modules/rpm_key.py index 74a9d50079..350cf4e3d1 100644 --- a/lib/ansible/modules/rpm_key.py +++ b/lib/ansible/modules/rpm_key.py @@ -9,9 +9,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['preview'], - 'supported_by': 'core'} DOCUMENTATION = ''' --- diff --git a/lib/ansible/modules/script.py b/lib/ansible/modules/script.py index ed66947c4b..11399b9c03 100644 --- a/lib/ansible/modules/script.py +++ b/lib/ansible/modules/script.py @@ -4,9 +4,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['stableinterface'], - 'supported_by': 'core'} DOCUMENTATION = r''' --- diff --git a/lib/ansible/modules/service.py b/lib/ansible/modules/service.py index 8846a36c65..7067b780e0 100644 --- a/lib/ansible/modules/service.py +++ b/lib/ansible/modules/service.py @@ -7,9 +7,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['stableinterface'], - 'supported_by': 'core'} DOCUMENTATION = r''' --- diff --git a/lib/ansible/modules/service_facts.py b/lib/ansible/modules/service_facts.py index c890163139..407be92128 100644 --- a/lib/ansible/modules/service_facts.py +++ b/lib/ansible/modules/service_facts.py @@ -9,10 +9,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['preview'], - 'supported_by': 'core'} - DOCUMENTATION = r''' --- module: service_facts diff --git a/lib/ansible/modules/set_fact.py b/lib/ansible/modules/set_fact.py index e3394fbe04..f87ab3cc5b 100644 --- a/lib/ansible/modules/set_fact.py +++ b/lib/ansible/modules/set_fact.py @@ -7,9 +7,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['stableinterface'], - 'supported_by': 'core'} DOCUMENTATION = r''' --- diff --git a/lib/ansible/modules/set_stats.py b/lib/ansible/modules/set_stats.py index 95109e96a6..65ae54a721 100644 --- a/lib/ansible/modules/set_stats.py +++ b/lib/ansible/modules/set_stats.py @@ -7,9 +7,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['preview'], - 'supported_by': 'core'} DOCUMENTATION = r''' --- diff --git a/lib/ansible/modules/setup.py b/lib/ansible/modules/setup.py index 70756568be..fa0de69b5f 100644 --- a/lib/ansible/modules/setup.py +++ b/lib/ansible/modules/setup.py @@ -8,11 +8,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['stableinterface'], - 'supported_by': 'core'} - - DOCUMENTATION = ''' --- module: setup diff --git a/lib/ansible/modules/shell.py b/lib/ansible/modules/shell.py index 8066d7ad52..e8bb7e031b 100644 --- a/lib/ansible/modules/shell.py +++ b/lib/ansible/modules/shell.py @@ -12,11 +12,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['stableinterface'], - 'supported_by': 'core'} - - DOCUMENTATION = r''' --- module: shell diff --git a/lib/ansible/modules/slurp.py b/lib/ansible/modules/slurp.py index c1891eac04..e1b5067bad 100644 --- a/lib/ansible/modules/slurp.py +++ b/lib/ansible/modules/slurp.py @@ -8,10 +8,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['stableinterface'], - 'supported_by': 'core'} - DOCUMENTATION = r''' --- module: slurp diff --git a/lib/ansible/modules/stat.py b/lib/ansible/modules/stat.py index 663d2c54a5..8868eb2812 100644 --- a/lib/ansible/modules/stat.py +++ b/lib/ansible/modules/stat.py @@ -6,9 +6,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['stableinterface'], - 'supported_by': 'core'} DOCUMENTATION = r''' --- diff --git a/lib/ansible/modules/subversion.py b/lib/ansible/modules/subversion.py index 1e60529a06..43b02cb20e 100644 --- a/lib/ansible/modules/subversion.py +++ b/lib/ansible/modules/subversion.py @@ -7,9 +7,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['preview'], - 'supported_by': 'core'} DOCUMENTATION = ''' --- diff --git a/lib/ansible/modules/systemd.py b/lib/ansible/modules/systemd.py index 0ecd8ea588..0f0263e199 100644 --- a/lib/ansible/modules/systemd.py +++ b/lib/ansible/modules/systemd.py @@ -7,9 +7,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['stableinterface'], - 'supported_by': 'core'} DOCUMENTATION = ''' module: systemd diff --git a/lib/ansible/modules/sysvinit.py b/lib/ansible/modules/sysvinit.py index 779024865a..309bb0541f 100644 --- a/lib/ansible/modules/sysvinit.py +++ b/lib/ansible/modules/sysvinit.py @@ -8,11 +8,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = { - 'metadata_version': '1.1', - 'status': ['preview'], - 'supported_by': 'core' -} DOCUMENTATION = ''' module: sysvinit diff --git a/lib/ansible/modules/tempfile.py b/lib/ansible/modules/tempfile.py index 08a56fbf7b..f77373f35f 100644 --- a/lib/ansible/modules/tempfile.py +++ b/lib/ansible/modules/tempfile.py @@ -8,9 +8,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['preview'], - 'supported_by': 'core'} DOCUMENTATION = ''' --- diff --git a/lib/ansible/modules/template.py b/lib/ansible/modules/template.py index c24deb72af..e01a18bb93 100644 --- a/lib/ansible/modules/template.py +++ b/lib/ansible/modules/template.py @@ -9,9 +9,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['stableinterface'], - 'supported_by': 'core'} DOCUMENTATION = r''' --- diff --git a/lib/ansible/modules/unarchive.py b/lib/ansible/modules/unarchive.py index fa38cec190..f69ab1c6cb 100644 --- a/lib/ansible/modules/unarchive.py +++ b/lib/ansible/modules/unarchive.py @@ -11,9 +11,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['preview'], - 'supported_by': 'core'} DOCUMENTATION = r''' --- diff --git a/lib/ansible/modules/uri.py b/lib/ansible/modules/uri.py index 21d5bec868..3cc4273230 100644 --- a/lib/ansible/modules/uri.py +++ b/lib/ansible/modules/uri.py @@ -8,10 +8,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['stableinterface'], - 'supported_by': 'core'} - DOCUMENTATION = r''' --- module: uri diff --git a/lib/ansible/modules/user.py b/lib/ansible/modules/user.py index 254e07f552..2a59ef0bc5 100644 --- a/lib/ansible/modules/user.py +++ b/lib/ansible/modules/user.py @@ -7,9 +7,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['stableinterface'], - 'supported_by': 'core'} DOCUMENTATION = r''' module: user diff --git a/lib/ansible/modules/wait_for.py b/lib/ansible/modules/wait_for.py index e11dfcb166..9c7614df97 100644 --- a/lib/ansible/modules/wait_for.py +++ b/lib/ansible/modules/wait_for.py @@ -7,9 +7,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['stableinterface'], - 'supported_by': 'core'} DOCUMENTATION = r''' --- diff --git a/lib/ansible/modules/wait_for_connection.py b/lib/ansible/modules/wait_for_connection.py index e3c65f4399..7b27ac4fd6 100644 --- a/lib/ansible/modules/wait_for_connection.py +++ b/lib/ansible/modules/wait_for_connection.py @@ -7,9 +7,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['stableinterface'], - 'supported_by': 'core'} DOCUMENTATION = r''' --- diff --git a/lib/ansible/modules/yum.py b/lib/ansible/modules/yum.py index fe1d0e8bfc..c95ea758e7 100644 --- a/lib/ansible/modules/yum.py +++ b/lib/ansible/modules/yum.py @@ -10,9 +10,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['stableinterface'], - 'supported_by': 'core'} DOCUMENTATION = ''' --- diff --git a/lib/ansible/modules/yum_repository.py b/lib/ansible/modules/yum_repository.py index a709156aab..a875c4cd98 100644 --- a/lib/ansible/modules/yum_repository.py +++ b/lib/ansible/modules/yum_repository.py @@ -9,12 +9,6 @@ from __future__ import absolute_import, division, print_function __metaclass__ = type -ANSIBLE_METADATA = { - 'metadata_version': '1.1', - 'status': ['stableinterface'], - 'supported_by': 'core' -} - DOCUMENTATION = ''' --- module: yum_repository diff --git a/lib/ansible/parsing/metadata.py b/lib/ansible/parsing/metadata.py deleted file mode 100644 index 70ed4491ce..0000000000 --- a/lib/ansible/parsing/metadata.py +++ /dev/null @@ -1,245 +0,0 @@ -# (c) 2017, Toshio Kuratomi <tkuratomi@ansible.com> -# -# This file is part of Ansible -# -# Ansible is free software: you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation, either version 3 of the License, or -# (at your option) any later version. -# -# Ansible is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with Ansible. If not, see <http://www.gnu.org/licenses/>. - -# Make coding more python3-ish -from __future__ import (absolute_import, division, print_function) -__metaclass__ = type - -import ast -import sys - -import yaml - -from ansible.module_utils._text import to_text - - -# There are currently defaults for all metadata fields so we can add it -# automatically if a file doesn't specify it -DEFAULT_METADATA = {'metadata_version': '1.1', 'status': ['preview'], 'supported_by': 'community'} - - -class ParseError(Exception): - """Thrown when parsing a file fails""" - pass - - -def _seek_end_of_dict(module_data, start_line, start_col, next_node_line, next_node_col): - """Look for the end of a dict in a set of lines - - We know the starting position of the dict and we know the start of the - next code node but in between there may be multiple newlines and comments. - There may also be multiple python statements on the same line (separated - by semicolons) - - Examples:: - ANSIBLE_METADATA = {[..]} - DOCUMENTATION = [..] - - ANSIBLE_METADATA = {[..]} # Optional comments with confusing junk => {} - # Optional comments {} - DOCUMENTATION = [..] - - ANSIBLE_METADATA = { - [..] - } - # Optional comments {} - DOCUMENTATION = [..] - - ANSIBLE_METADATA = {[..]} ; DOCUMENTATION = [..] - - ANSIBLE_METADATA = {}EOF - """ - if next_node_line is None: - # The dict is the last statement in the file - snippet = module_data.splitlines()[start_line:] - next_node_col = 0 - # Include the last line in the file - last_line_offset = 0 - else: - # It's somewhere in the middle so we need to separate it from the rest - snippet = module_data.splitlines()[start_line:next_node_line] - # Do not include the last line because that's where the next node - # starts - last_line_offset = 1 - - if next_node_col == 0: - # This handles all variants where there are only comments and blank - # lines between the dict and the next code node - - # Step backwards through all the lines in the snippet - for line_idx, line in tuple(reversed(tuple(enumerate(snippet))))[last_line_offset:]: - end_col = None - # Step backwards through all the characters in the line - for col_idx, char in reversed(tuple(enumerate(c for c in line))): - if not isinstance(char, bytes): - # Python3 wart. slicing a byte string yields integers - char = bytes((char,)) - if char == b'}' and end_col is None: - # Potentially found the end of the dict - end_col = col_idx - - elif char == b'#' and end_col is not None: - # The previous '}' was part of a comment. Keep trying - end_col = None - - if end_col is not None: - # Found the end! - end_line = start_line + line_idx - break - else: - raise ParseError('Unable to find the end of dictionary') - else: - # Harder cases involving multiple statements on one line - # Good Ansible Module style doesn't do this so we're just going to - # treat this as an error for now: - raise ParseError('Multiple statements per line confuses the module metadata parser.') - - return end_line, end_col - - -def _seek_end_of_string(module_data, start_line, start_col, next_node_line, next_node_col): - """ - This is much trickier than finding the end of a dict. A dict has only one - ending character, "}". Strings have four potential ending characters. We - have to parse the beginning of the string to determine what the ending - character will be. - - Examples: - ANSIBLE_METADATA = '''[..]''' # Optional comment with confusing chars ''' - # Optional comment with confusing chars ''' - DOCUMENTATION = [..] - - ANSIBLE_METADATA = ''' - [..] - ''' - DOCUMENTATIONS = [..] - - ANSIBLE_METADATA = '''[..]''' ; DOCUMENTATION = [..] - - SHORT_NAME = ANSIBLE_METADATA = '''[..]''' ; DOCUMENTATION = [..] - - String marker variants: - * '[..]' - * "[..]" - * '''[..]''' - * \"\"\"[..]\"\"\" - - Each of these come in u, r, and b variants: - * '[..]' - * u'[..]' - * b'[..]' - * r'[..]' - * ur'[..]' - * ru'[..]' - * br'[..]' - * b'[..]' - * rb'[..]' - """ - raise NotImplementedError('Finding end of string not yet implemented') - - -def extract_metadata(module_ast=None, module_data=None, offsets=False): - """Extract the metadata from a module - - :kwarg module_ast: ast representation of the module. At least one of this - or ``module_data`` must be given. If the code calling - :func:`extract_metadata` has already parsed the module_data into an ast, - giving the ast here will save reparsing it. - :kwarg module_data: Byte string containing a module's code. At least one - of this or ``module_ast`` must be given. - :kwarg offsets: If set to True, offests into the source code will be - returned. This requires that ``module_data`` be set. - :returns: a tuple of metadata (a dict), line the metadata starts on, - column the metadata starts on, line the metadata ends on, column the - metadata ends on, and the names the metadata is assigned to. One of - the names the metadata is assigned to will be ANSIBLE_METADATA. If no - metadata is found, the tuple will be (None, -1, -1, -1, -1, None). - If ``offsets`` is False then the tuple will consist of - (metadata, -1, -1, -1, -1, None). - :raises ansible.parsing.metadata.ParseError: if ``module_data`` does not parse - :raises SyntaxError: if ``module_data`` is needed but does not parse correctly - """ - if offsets and module_data is None: - raise TypeError('If offsets is True then module_data must also be given') - - if module_ast is None and module_data is None: - raise TypeError('One of module_ast or module_data must be given') - - metadata = None - start_line = -1 - start_col = -1 - end_line = -1 - end_col = -1 - targets = None - if module_ast is None: - module_ast = ast.parse(module_data) - - for root_idx, child in reversed(list(enumerate(module_ast.body))): - if isinstance(child, ast.Assign): - for target in child.targets: - if isinstance(target, ast.Name) and target.id == 'ANSIBLE_METADATA': - metadata = ast.literal_eval(child.value) - if not offsets: - continue - - try: - # Determine where the next node starts - next_node = module_ast.body[root_idx + 1] - next_lineno = next_node.lineno - next_col_offset = next_node.col_offset - except IndexError: - # Metadata is defined in the last node of the file - next_lineno = None - next_col_offset = None - - if isinstance(child.value, ast.Dict): - # Determine where the current metadata ends - end_line, end_col = _seek_end_of_dict(module_data, - child.lineno - 1, - child.col_offset, - next_lineno, - next_col_offset) - - elif isinstance(child.value, ast.Str): - metadata = yaml.safe_load(child.value.s) - end_line, end_col = _seek_end_of_string(module_data, - child.lineno - 1, - child.col_offset, - next_lineno, - next_col_offset) - elif isinstance(child.value, ast.Bytes): - metadata = yaml.safe_load(to_text(child.value.s, errors='surrogate_or_strict')) - end_line, end_col = _seek_end_of_string(module_data, - child.lineno - 1, - child.col_offset, - next_lineno, - next_col_offset) - else: - raise ParseError('Ansible plugin metadata must be a dict') - - # Do these after the if-else so we don't pollute them in - # case this was a false positive - start_line = child.lineno - 1 - start_col = child.col_offset - targets = [t.id for t in child.targets] - break - - if metadata is not None: - # Once we've found the metadata we're done - break - - return metadata, start_line, start_col, end_line, end_col, targets diff --git a/lib/ansible/parsing/plugin_docs.py b/lib/ansible/parsing/plugin_docs.py index 4f036127e0..468fed1407 100644 --- a/lib/ansible/parsing/plugin_docs.py +++ b/lib/ansible/parsing/plugin_docs.py @@ -5,17 +5,18 @@ from __future__ import (absolute_import, division, print_function) __metaclass__ = type import ast -import yaml from ansible.module_utils._text import to_text -from ansible.parsing.metadata import extract_metadata from ansible.parsing.yaml.loader import AnsibleLoader from ansible.utils.display import Display display = Display() +# NOTE: should move to just reading the variable as we do in plugin_loader since we already load as a 'module' +# which is much faster than ast parsing ourselves. def read_docstring(filename, verbose=True, ignore_errors=True): + """ Search for assignment of the DOCUMENTATION and EXAMPLES variables in the given file. Parse DOCUMENTATION from YAML and return the YAML doc or None together with EXAMPLES, as plain text. @@ -25,7 +26,7 @@ def read_docstring(filename, verbose=True, ignore_errors=True): 'doc': None, 'plainexamples': None, 'returndocs': None, - 'metadata': None, + 'metadata': None, # NOTE: not used anymore, kept for compat 'seealso': None, } @@ -33,6 +34,7 @@ def read_docstring(filename, verbose=True, ignore_errors=True): 'DOCUMENTATION': 'doc', 'EXAMPLES': 'plainexamples', 'RETURN': 'returndocs', + 'ANSIBLE_METADATA': 'metadata', # NOTE: now unused, but kept for backwards compat } try: @@ -54,33 +56,16 @@ def read_docstring(filename, verbose=True, ignore_errors=True): if isinstance(child.value, ast.Dict): data[varkey] = ast.literal_eval(child.value) else: - if theid == 'DOCUMENTATION': - # string should be yaml - data[varkey] = AnsibleLoader(child.value.s, file_name=filename).get_single_data() - else: - # not yaml, should be a simple string + if theid in ['EXAMPLES', 'RETURN']: + # examples 'can' be yaml, return must be, but even if so, we dont want to parse as such here + # as it can create undesired 'objects' that don't display well as docs. data[varkey] = to_text(child.value.s) - display.debug('assigned :%s' % varkey) - - # Metadata is per-file and a dict rather than per-plugin/function and yaml - data['metadata'] = extract_metadata(module_ast=M)[0] - - if data['metadata']: - # remove version - for field in ('version', 'metadata_version'): - if field in data['metadata']: - del data['metadata'][field] - - if 'supported_by' not in data['metadata']: - data['metadata']['supported_by'] = 'community' + else: + # string should be yaml if already not a dict + data[varkey] = AnsibleLoader(child.value.s, file_name=filename).get_single_data() - if 'status' not in data['metadata']: - data['metadata']['status'] = ['preview'] + display.debug('assigned: %s' % varkey) - else: - # Add default metadata - data['metadata'] = {'supported_by': 'community', - 'status': ['preview']} except Exception: if verbose: display.error("unable to parse %s" % filename) diff --git a/lib/ansible/plugins/filter/urlsplit.py b/lib/ansible/plugins/filter/urlsplit.py index 19ad5f26f8..84e460aad1 100644 --- a/lib/ansible/plugins/filter/urlsplit.py +++ b/lib/ansible/plugins/filter/urlsplit.py @@ -6,13 +6,6 @@ from __future__ import (absolute_import, division, print_function) __metaclass__ = type -ANSIBLE_METADATA = { - 'metadata_version': '1.1', - 'status': ['preview'], - 'supported_by': 'core' -} - - from ansible.errors import AnsibleFilterError from ansible.module_utils.six.moves.urllib.parse import urlsplit from ansible.utils import helpers diff --git a/lib/ansible/plugins/inventory/toml.py b/lib/ansible/plugins/inventory/toml.py index 31fe95fb56..efbec37216 100644 --- a/lib/ansible/plugins/inventory/toml.py +++ b/lib/ansible/plugins/inventory/toml.py @@ -4,10 +4,6 @@ from __future__ import (absolute_import, division, print_function) __metaclass__ = type -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['preview'], - 'supported_by': 'core'} - DOCUMENTATION = ''' inventory: toml version_added: "2.8" diff --git a/test/integration/targets/ansible-doc/fakemodule.output b/test/integration/targets/ansible-doc/fakemodule.output index 3c20a0c4a6..a7abc241d4 100644 --- a/test/integration/targets/ansible-doc/fakemodule.output +++ b/test/integration/targets/ansible-doc/fakemodule.output @@ -2,7 +2,6 @@ this is a fake module - * This module is maintained by The Ansible Community OPTIONS (= is mandatory): - _notreal @@ -11,10 +10,6 @@ OPTIONS (= is mandatory): AUTHOR: me - METADATA: - status: - - preview - supported_by: community - + SHORT_DESCIPTOIN: fake module diff --git a/test/integration/targets/ansible-doc/test.yml b/test/integration/targets/ansible-doc/test.yml index 1781caad46..93a64777a5 100644 --- a/test/integration/targets/ansible-doc/test.yml +++ b/test/integration/targets/ansible-doc/test.yml @@ -49,6 +49,7 @@ - name: documented module with removed status command: ansible-doc test_docs_removed_status register: result + - assert: that: - '"WARNING" not in result.stderr' @@ -58,41 +59,17 @@ - name: empty module command: ansible-doc test_empty register: result + ignore_errors: true + - assert: that: - - 'result.stdout == ""' - - 'result.stderr == ""' + - result is failed - name: module with no documentation command: ansible-doc test_no_docs register: result - - assert: - that: - - 'result.stdout == ""' - - 'result.stderr == ""' + ignore_errors: true - - name: module with no documentation and no metadata - command: ansible-doc test_no_docs_no_metadata - register: result - - assert: - that: - - 'result.stdout == ""' - - 'result.stderr == ""' - - - name: module with no documentation and no status in metadata - command: ansible-doc test_no_docs_no_status - ignore_errors: yes - register: result - - assert: - that: - - 'result.stdout == ""' - - 'result.stderr == ""' - - - name: module with no documentation and non-iterable status in metadata - command: ansible-doc test_no_docs_non_iterable_status - ignore_errors: yes - register: result - assert: that: - - 'result is failed' - - '"ERROR! module test_no_docs_non_iterable_status missing documentation (or could not parse documentation): test_no_docs_non_iterable_status did not contain a DOCUMENTATION attribute" in result.stderr' + - result is failed diff --git a/test/integration/targets/collections/inventory_cache/.keep b/test/integration/targets/collections/inventory_cache/.keep deleted file mode 100644 index e69de29bb2..0000000000 --- a/test/integration/targets/collections/inventory_cache/.keep +++ /dev/null diff --git a/test/lib/ansible_test/_data/sanity/validate-modules/validate_modules/main.py b/test/lib/ansible_test/_data/sanity/validate-modules/validate_modules/main.py index 0765da7670..b409974b11 100644 --- a/test/lib/ansible_test/_data/sanity/validate-modules/validate_modules/main.py +++ b/test/lib/ansible_test/_data/sanity/validate-modules/validate_modules/main.py @@ -41,7 +41,7 @@ import yaml from ansible import __version__ as ansible_version from ansible.executor.module_common import REPLACER_WINDOWS from ansible.module_utils.common._collections_compat import Mapping -from ansible.module_utils._text import to_bytes, to_native +from ansible.module_utils._text import to_native from ansible.plugins.loader import fragment_loader from ansible.utils.collection_loader._collection_finder import _AnsibleCollectionFinder from ansible.utils.plugin_docs import BLACKLIST, tag_versions_and_dates, add_fragments, get_docstring @@ -49,7 +49,7 @@ from ansible.utils.version import SemanticVersion from .module_args import AnsibleModuleImportError, AnsibleModuleNotInitialized, get_argument_spec -from .schema import ansible_module_kwargs_schema, doc_schema, metadata_1_1_schema, return_schema +from .schema import ansible_module_kwargs_schema, doc_schema, return_schema from .utils import CaptureStd, NoArgsAnsibleModule, compare_unordered_lists, is_empty, parse_yaml, parse_isodate from voluptuous.humanize import humanize_error @@ -696,7 +696,7 @@ class ModuleValidator(Validator): code='import-before-documentation', msg=('Import found before documentation variables. ' 'All imports must appear below ' - 'DOCUMENTATION/EXAMPLES/RETURN/ANSIBLE_METADATA.'), + 'DOCUMENTATION/EXAMPLES/RETURN.'), line=child.lineno ) break @@ -713,8 +713,7 @@ class ModuleValidator(Validator): code='import-before-documentation', msg=('Import found before documentation ' 'variables. All imports must appear below ' - 'DOCUMENTATION/EXAMPLES/RETURN/' - 'ANSIBLE_METADATA.'), + 'DOCUMENTATION/EXAMPLES/RETURN.'), line=child.lineno ) break @@ -724,7 +723,7 @@ class ModuleValidator(Validator): msg = ( 'import-placement', ('Imports should be directly below DOCUMENTATION/EXAMPLES/' - 'RETURN/ANSIBLE_METADATA.') + 'RETURN.') ) if self._is_new_module(): self.reporter.error( @@ -829,11 +828,6 @@ class ModuleValidator(Validator): 'lineno': 0, 'end_lineno': 0, }, - 'ANSIBLE_METADATA': { - 'value': None, - 'lineno': 0, - 'end_lineno': 0, - } } for child in self.ast.body: if isinstance(child, ast.Assign): @@ -859,17 +853,6 @@ class ModuleValidator(Validator): docs['RETURN']['end_lineno'] = ( child.lineno + len(child.value.s.splitlines()) ) - elif grandchild.id == 'ANSIBLE_METADATA': - docs['ANSIBLE_METADATA']['value'] = child.value - docs['ANSIBLE_METADATA']['lineno'] = child.lineno - try: - docs['ANSIBLE_METADATA']['end_lineno'] = ( - child.lineno + len(child.value.s.splitlines()) - ) - except AttributeError: - docs['ANSIBLE_METADATA']['end_lineno'] = ( - child.value.values[-1].lineno - ) return docs @@ -939,51 +922,13 @@ class ModuleValidator(Validator): if self.object_name.startswith('_') and not os.path.islink(self.object_path): filename_deprecated_or_removed = True - # Have to check the metadata first so that we know if the module is removed or deprecated - metadata = None - if not self.collection: - if not bool(doc_info['ANSIBLE_METADATA']['value']): - self.reporter.error( - path=self.object_path, - code='missing-metadata', - msg='No ANSIBLE_METADATA provided' - ) - else: - if isinstance(doc_info['ANSIBLE_METADATA']['value'], ast.Dict): - metadata = ast.literal_eval( - doc_info['ANSIBLE_METADATA']['value'] - ) - else: - self.reporter.error( - path=self.object_path, - code='missing-metadata-format', - msg='ANSIBLE_METADATA was not provided as a dict, YAML not supported' - ) - - if metadata: - self._validate_docs_schema(metadata, metadata_1_1_schema(), - 'ANSIBLE_METADATA', 'invalid-metadata-type') - # We could validate these via the schema if we knew what the values are ahead of - # time. We can figure that out for deprecated but we can't for removed. Only the - # metadata has that information. - if 'removed' in metadata['status']: - removed = True - if 'deprecated' in metadata['status']: - deprecated = True - if (deprecated or removed) and len(metadata['status']) > 1: - self.reporter.error( - path=self.object_path, - code='missing-metadata-status', - msg='ANSIBLE_METADATA.status must be exactly one of "deprecated" or "removed"' - ) - else: - # We are testing a collection - if self.routing: - routing_deprecation = self.routing.get('plugin_routing', {}).get('modules', {}).get(self.name, {}).get('deprecation', {}) - if routing_deprecation: - # meta/runtime.yml says this is deprecated - routing_says_deprecated = True - deprecated = True + # We are testing a collection + if self.routing: + routing_deprecation = self.routing.get('plugin_routing', {}).get('modules', {}).get(self.name, {}).get('deprecation', {}) + if routing_deprecation: + # meta/runtime.yml says this is deprecated + routing_says_deprecated = True + deprecated = True if not removed: if not bool(doc_info['DOCUMENTATION']['value']): @@ -1080,7 +1025,7 @@ class ModuleValidator(Validator): ) if not self.collection: - existing_doc = self._check_for_new_args(doc, metadata) + existing_doc = self._check_for_new_args(doc) self._check_version_added(doc, existing_doc) if not bool(doc_info['EXAMPLES']['value']): @@ -1152,7 +1097,7 @@ class ModuleValidator(Validator): self.reporter.error( path=self.object_path, code='deprecation-mismatch', - msg='Module deprecation/removed must agree in Metadata, by prepending filename with' + msg='Module deprecation/removed must agree in documentaiton, by prepending filename with' ' "_", and setting DOCUMENTATION.deprecated for deprecation or by removing all' ' documentation for removed' ) @@ -2003,7 +1948,7 @@ class ModuleValidator(Validator): msg=msg ) - def _check_for_new_args(self, doc, metadata): + def _check_for_new_args(self, doc): if not self.base_branch or self._is_new_module(): return @@ -2038,16 +1983,6 @@ class ModuleValidator(Validator): except ValueError: mod_version_added = self._create_strict_version('0.0') - if self.base_branch and 'stable-' in self.base_branch: - metadata.pop('metadata_version', None) - metadata.pop('version', None) - if metadata != existing_metadata: - self.reporter.error( - path=self.object_path, - code='metadata-changed', - msg=('ANSIBLE_METADATA cannot be changed in a point release for a stable branch') - ) - options = doc.get('options', {}) or {} should_be = '.'.join(ansible_version.split('.')[:2]) @@ -2146,24 +2081,10 @@ class ModuleValidator(Validator): doc_info, docs = self._validate_docs() # See if current version => deprecated.removed_in, ie, should be docs only - if isinstance(doc_info['ANSIBLE_METADATA']['value'], ast.Dict) and 'removed' in ast.literal_eval(doc_info['ANSIBLE_METADATA']['value'])['status']: - end_of_deprecation_should_be_removed_only = True - elif docs and 'deprecated' in docs and docs['deprecated'] is not None: - end_of_deprecation_should_be_removed_only = False - if 'removed_at_date' in docs['deprecated']: - try: - removed_at_date = docs['deprecated']['removed_at_date'] - if parse_isodate(removed_at_date) < datetime.date.today(): - msg = "Module's deprecated.removed_at_date date '%s' is before today" % removed_at_date - self.reporter.error( - path=self.object_path, - code='deprecated-date', - msg=msg, - ) - except ValueError: - # Already checked during schema validation - pass + if docs and docs.get('deprecated', False): + if 'removed_in' in docs['deprecated']: + removed_in = None try: collection_name, version = self._split_tagged_version(docs['deprecated']['removed_in']) if collection_name != self.collection_name: @@ -2172,20 +2093,31 @@ class ModuleValidator(Validator): code='invalid-module-deprecation-source', msg=('The deprecation version for a module must be added in this collection') ) - # Treat the module as not to be removed: - raise ValueError('') - removed_in = self._create_strict_version(str(version)) + else: + removed_in = self.StrictVersion(str(version)) + except ValueError: - end_of_deprecation_should_be_removed_only = False - else: + # ignore and hope we previouslly reported + pass + + if removed_in: if not self.collection: - strict_ansible_version = self._create_strict_version('.'.join(ansible_version.split('.')[:2])) + strict_ansible_version = self.StrictVersion('.'.join(ansible_version.split('.')[:2])) end_of_deprecation_should_be_removed_only = strict_ansible_version >= removed_in elif self.collection_version: strict_ansible_version = self.collection_version end_of_deprecation_should_be_removed_only = strict_ansible_version >= removed_in - else: - end_of_deprecation_should_be_removed_only = False + + # handle deprecation by date + if 'removed_at_date' in docs['deprecated']: + try: + removed_at_date = docs['deprecated']['removed_at_date'] + if parse_isodate(removed_at_date) < datetime.date.today(): + msg = "Module's deprecated.removed_at_date date '%s' is before today" % removed_at_date + self.reporter.error(path=self.object_path, code='deprecated-date', msg=msg) + except ValueError: + # ignore and hope we previouslly reported + pass if self._python_module() and not self._just_docs() and not end_of_deprecation_should_be_removed_only: self._validate_ansible_module_call(docs) diff --git a/test/lib/ansible_test/_data/sanity/validate-modules/validate_modules/schema.py b/test/lib/ansible_test/_data/sanity/validate-modules/validate_modules/schema.py index 9eea2e94ad..1dcb081bfd 100644 --- a/test/lib/ansible_test/_data/sanity/validate-modules/validate_modules/schema.py +++ b/test/lib/ansible_test/_data/sanity/validate-modules/validate_modules/schema.py @@ -442,32 +442,6 @@ def doc_schema(module_name, for_collection=False, deprecated_module=False): ) -def metadata_1_0_schema(deprecated): - valid_status = Any('stableinterface', 'preview', 'deprecated', 'removed') - if deprecated: - valid_status = Any('deprecated') - - return Schema( - { - Required('status'): [valid_status], - Required('metadata_version'): '1.0', - Required('supported_by'): Any('core', 'community', 'curated') - } - ) - - -def metadata_1_1_schema(): - valid_status = Any('stableinterface', 'preview', 'deprecated', 'removed') - - return Schema( - { - Required('status'): [valid_status], - Required('metadata_version'): '1.1', - Required('supported_by'): Any('core', 'community', 'certified', 'network') - } - ) - - # Things to add soon #################### # 1) Recursively validate `type: complex` fields diff --git a/test/units/parsing/test_metadata.py b/test/units/parsing/test_metadata.py deleted file mode 100644 index a82fc8047d..0000000000 --- a/test/units/parsing/test_metadata.py +++ /dev/null @@ -1,239 +0,0 @@ -# coding: utf-8 -# (c) 2017, Toshio Kuratomi <tkuratomi@ansible.com> -# -# This file is part of Ansible -# -# Ansible is free software: you can redistribute it and/or modify -# it under the terms of the GNU General Public License as published by -# the Free Software Foundation, either version 3 of the License, or -# (at your option) any later version. -# -# Ansible is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -# GNU General Public License for more details. -# -# You should have received a copy of the GNU General Public License -# along with Ansible. If not, see <http://www.gnu.org/licenses/>. - -# Make coding more python3-ish -from __future__ import (absolute_import, division, print_function) -__metaclass__ = type - -import ast - -import pytest - -from ansible.parsing import metadata as md - - -LICENSE = b"""# some license text boilerplate -# That we have at the top of files -""" - -FUTURE_IMPORTS = b""" -from __future__ import (absolute_import, division, print_function) -""" - -REGULAR_IMPORTS = b""" -import test -from foo import bar -""" - -STANDARD_METADATA = b""" -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['stableinterface'], - 'supported_by': 'core'} -""" - -TEXT_STD_METADATA = b""" -ANSIBLE_METADATA = u''' -metadata_version: '1.1' -status: - - 'stableinterface' -supported_by: 'core' -''' -""" - -BYTES_STD_METADATA = b""" -ANSIBLE_METADATA = b''' -metadata_version: '1.1' -status: - - 'stableinterface' -supported_by: 'core' -''' -""" - -TRAILING_COMMENT_METADATA = b""" -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['stableinterface'], - 'supported_by': 'core'} # { Testing } -""" - -MULTIPLE_STATEMENTS_METADATA = b""" -DOCUMENTATION = "" ; ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['stableinterface'], - 'supported_by': 'core'} ; RETURNS = "" -""" - -EMBEDDED_COMMENT_METADATA = b""" -ANSIBLE_METADATA = {'metadata_version': '1.1', - 'status': ['stableinterface'], - # { Testing } - 'supported_by': 'core'} -""" - -HASH_SYMBOL_METADATA = b""" -ANSIBLE_METADATA = {'metadata_version': '1.1 # 4', - 'status': ['stableinterface'], - 'supported_by': 'core # Testing '} -""" - -HASH_SYMBOL_METADATA = b""" -ANSIBLE_METADATA = {'metadata_version': '1.1 # 4', - 'status': ['stableinterface'], - 'supported_by': 'core # Testing '} -""" - -HASH_COMBO_METADATA = b""" -ANSIBLE_METADATA = {'metadata_version': '1.1 # 4', - 'status': ['stableinterface'], - # { Testing } - 'supported_by': 'core'} # { Testing } -""" - -METADATA = {'metadata_version': '1.1', 'status': ['stableinterface'], 'supported_by': 'core'} -HASH_SYMBOL_METADATA = {'metadata_version': '1.1 # 4', 'status': ['stableinterface'], 'supported_by': 'core'} - -METADATA_EXAMPLES = ( - # Standard import - (LICENSE + FUTURE_IMPORTS + STANDARD_METADATA + REGULAR_IMPORTS, - (METADATA, 5, 0, 7, 42, ['ANSIBLE_METADATA'])), - # Metadata at end of file - (LICENSE + FUTURE_IMPORTS + REGULAR_IMPORTS + STANDARD_METADATA.rstrip(), - (METADATA, 8, 0, 10, 42, ['ANSIBLE_METADATA'])), - # Metadata at beginning of file - (STANDARD_METADATA + LICENSE + REGULAR_IMPORTS, - (METADATA, 1, 0, 3, 42, ['ANSIBLE_METADATA'])), - - # Standard import with a trailing comment - (LICENSE + FUTURE_IMPORTS + TRAILING_COMMENT_METADATA + REGULAR_IMPORTS, - (METADATA, 5, 0, 7, 42, ['ANSIBLE_METADATA'])), - # Metadata at end of file with a trailing comment - (LICENSE + FUTURE_IMPORTS + REGULAR_IMPORTS + TRAILING_COMMENT_METADATA.rstrip(), - (METADATA, 8, 0, 10, 42, ['ANSIBLE_METADATA'])), - # Metadata at beginning of file with a trailing comment - (TRAILING_COMMENT_METADATA + LICENSE + REGULAR_IMPORTS, - (METADATA, 1, 0, 3, 42, ['ANSIBLE_METADATA'])), - - # FIXME: Current code cannot handle multiple statements on the same line. - # This is bad style so we're just going to ignore it for now - # Standard import with other statements on the same line - # (LICENSE + FUTURE_IMPORTS + MULTIPLE_STATEMENTS_METADATA + REGULAR_IMPORTS, - # (METADATA, 5, 0, 7, 42, ['ANSIBLE_METADATA'])), - # Metadata at end of file with other statements on the same line - # (LICENSE + FUTURE_IMPORTS + REGULAR_IMPORTS + MULTIPLE_STATEMENTS_METADATA.rstrip(), - # (METADATA, 8, 0, 10, 42, ['ANSIBLE_METADATA'])), - # Metadata at beginning of file with other statements on the same line - # (MULTIPLE_STATEMENTS_METADATA + LICENSE + REGULAR_IMPORTS, - # (METADATA, 1, 0, 3, 42, ['ANSIBLE_METADATA'])), - - # Standard import with comment inside the metadata - (LICENSE + FUTURE_IMPORTS + EMBEDDED_COMMENT_METADATA + REGULAR_IMPORTS, - (METADATA, 5, 0, 8, 42, ['ANSIBLE_METADATA'])), - # Metadata at end of file with comment inside the metadata - (LICENSE + FUTURE_IMPORTS + REGULAR_IMPORTS + EMBEDDED_COMMENT_METADATA.rstrip(), - (METADATA, 8, 0, 11, 42, ['ANSIBLE_METADATA'])), - # Metadata at beginning of file with comment inside the metadata - (EMBEDDED_COMMENT_METADATA + LICENSE + REGULAR_IMPORTS, - (METADATA, 1, 0, 4, 42, ['ANSIBLE_METADATA'])), - - # FIXME: Current code cannot handle hash symbols in the last element of - # the metadata. Fortunately, the metadata currently fully specifies all - # the strings inside of metadata and none of them can contain a hash. - # Need to fix this to future-proof it against strings containing hashes - # Standard import with hash symbol in metadata - # (LICENSE + FUTURE_IMPORTS + HASH_SYMBOL_METADATA + REGULAR_IMPORTS, - # (HASH_SYMBOL_METADATA, 5, 0, 7, 53, ['ANSIBLE_METADATA'])), - # Metadata at end of file with hash symbol in metadata - # (LICENSE + FUTURE_IMPORTS + REGULAR_IMPORTS + HASH_SYMBOL_HASH_SYMBOL_METADATA.rstrip(), - # (HASH_SYMBOL_METADATA, 8, 0, 10, 53, ['ANSIBLE_METADATA'])), - # Metadata at beginning of file with hash symbol in metadata - # (HASH_SYMBOL_HASH_SYMBOL_METADATA + LICENSE + REGULAR_IMPORTS, - # (HASH_SYMBOL_METADATA, 1, 0, 3, 53, ['ANSIBLE_METADATA'])), - - # Standard import with a bunch of hashes everywhere - (LICENSE + FUTURE_IMPORTS + HASH_COMBO_METADATA + REGULAR_IMPORTS, - (HASH_SYMBOL_METADATA, 5, 0, 8, 42, ['ANSIBLE_METADATA'])), - # Metadata at end of file with a bunch of hashes everywhere - (LICENSE + FUTURE_IMPORTS + REGULAR_IMPORTS + HASH_COMBO_METADATA.rstrip(), - (HASH_SYMBOL_METADATA, 8, 0, 11, 42, ['ANSIBLE_METADATA'])), - # Metadata at beginning of file with a bunch of hashes everywhere - (HASH_COMBO_METADATA + LICENSE + REGULAR_IMPORTS, - (HASH_SYMBOL_METADATA, 1, 0, 4, 42, ['ANSIBLE_METADATA'])), - - # Standard import with a junk ANSIBLE_METADATA as well - (LICENSE + FUTURE_IMPORTS + b"\nANSIBLE_METADATA = 10\n" + HASH_COMBO_METADATA + REGULAR_IMPORTS, - (HASH_SYMBOL_METADATA, 7, 0, 10, 42, ['ANSIBLE_METADATA'])), -) - -# FIXME: String/yaml metadata is not implemented yet. Need more test cases once it is implemented -STRING_METADATA_EXAMPLES = ( - # Standard import - (LICENSE + FUTURE_IMPORTS + TEXT_STD_METADATA + REGULAR_IMPORTS, - (METADATA, 5, 0, 10, 3, ['ANSIBLE_METADATA'])), - # Metadata at end of file - (LICENSE + FUTURE_IMPORTS + REGULAR_IMPORTS + TEXT_STD_METADATA.rstrip(), - (METADATA, 8, 0, 13, 3, ['ANSIBLE_METADATA'])), - # Metadata at beginning of file - (TEXT_STD_METADATA + LICENSE + REGULAR_IMPORTS, - (METADATA, 1, 0, 6, 3, ['ANSIBLE_METADATA'])), - - # Standard import - (LICENSE + FUTURE_IMPORTS + BYTES_STD_METADATA + REGULAR_IMPORTS, - (METADATA, 5, 0, 10, 3, ['ANSIBLE_METADATA'])), - # Metadata at end of file - (LICENSE + FUTURE_IMPORTS + REGULAR_IMPORTS + BYTES_STD_METADATA.rstrip(), - (METADATA, 8, 0, 13, 3, ['ANSIBLE_METADATA'])), - # Metadata at beginning of file - (BYTES_STD_METADATA + LICENSE + REGULAR_IMPORTS, - (METADATA, 1, 0, 6, 3, ['ANSIBLE_METADATA'])), -) - - -@pytest.mark.parametrize("code, expected", METADATA_EXAMPLES) -def test_dict_metadata(code, expected): - assert md.extract_metadata(module_data=code, offsets=True) == expected - - -@pytest.mark.parametrize("code, expected", STRING_METADATA_EXAMPLES) -def test_string_metadata(code, expected): - # FIXME: String/yaml metadata is not implemented yet. - with pytest.raises(NotImplementedError): - assert md.extract_metadata(module_data=code, offsets=True) == expected - - -def test_required_params(): - with pytest.raises(TypeError, match='One of module_ast or module_data must be given'): - assert md.extract_metadata() - - -def test_module_data_param_given_with_offset(): - with pytest.raises(TypeError, match='If offsets is True then module_data must also be given'): - assert md.extract_metadata(module_ast='something', offsets=True) - - -def test_invalid_dict_metadata(): - with pytest.raises(SyntaxError): - assert md.extract_metadata(module_data=LICENSE + FUTURE_IMPORTS + b'ANSIBLE_METADATA={"metadata_version": "1.1",\n' + REGULAR_IMPORTS) - - with pytest.raises(md.ParseError, match='Unable to find the end of dictionary'): - assert md.extract_metadata(module_ast=ast.parse(LICENSE + FUTURE_IMPORTS + b'ANSIBLE_METADATA={"metadata_version": "1.1"}\n' + REGULAR_IMPORTS), - module_data=LICENSE + FUTURE_IMPORTS + b'ANSIBLE_METADATA={"metadata_version": "1.1",\n' + REGULAR_IMPORTS, - offsets=True) - - -def test_multiple_statements_limitation(): - with pytest.raises(md.ParseError, match='Multiple statements per line confuses the module metadata parser.'): - assert md.extract_metadata(module_data=LICENSE + FUTURE_IMPORTS + b'ANSIBLE_METADATA={"metadata_version": "1.1"}; a=b\n' + REGULAR_IMPORTS, - offsets=True) |