Documentation testing

We treat documentation as code, and so use tests in our CI pipeline to maintain the standards and quality of the docs. The current tests, which run in CI jobs when a merge request with new or changed docs is submitted, are:

• docs lint: Runs several tests on the content of the docs themselves:
  • lint-doc.sh script runs the following checks and linters:
    • All cURL examples use the long flags (ex: --header, not -H).
    • The CHANGELOG.md does not contain duplicate versions.
    • No files in doc/ are executable.
    • No new README.md was added.
    • markdownlint.
    • Vale.
  • Nanoc tests:
    • internal_links checks that all internal links (ex: [link](../index.md)) are valid.
    • internal_anchors checks that all internal anchors (ex: [link](../index.md#internal_anchor)) are valid.
  • ui-docs-links lint checks that all links to docs from UI elements (app/views files, for example) are linking to valid docs and anchors.

Run tests locally

Apart from previewing your changes locally, you can also run all lint checks and Nanoc tests locally.

Lint checks

Lint checks are performed by the lint-doc.sh script and can be executed as follows:

1. Navigate to the gitlab directory.

2. Run:

   MD_DOC_PATH=path/to/my_doc.md scripts/lint-doc.sh

Where MD_DOC_PATH points to the file or directory you would like to run lint checks for. If you omit it completely, it defaults to the doc/ directory. The output should be similar to:

=> Linting documents at path /path/to/gitlab as <user>...
=> Checking for cURL short options...
=> Checking for CHANGELOG.md duplicate entries...
=> Checking /path/to/gitlab/doc for executable permissions...
=> Checking for new README.md files...
=> Linting markdown style...
=> Linting prose...
✔ 0 errors, 0 warnings and 0 suggestions in 1 file.
✔ Linting passed

This requires you to either:

• Have the required lint tools installed on your machine.
• A working Docker installation, in which case an image with these tools pre-installed is used.

Nanoc tests

To execute Nanoc tests locally:

1. Navigate to the gitlab-docs directory.

2. Run:

   # Check for broken internal links
   bundle exec nanoc check internal_links
   
   # Check for broken external links (might take a lot of time to complete).
   # This test is set to be allowed to fail and is run only in the gitlab-docs project CI
   bundle exec nanoc check internal_anchors

ui-docs-links test

The ui-docs-links lint job uses haml-lint to test that all links to docs from UI elements (app/views files, for example) are linking to valid docs and anchors.

To run the ui-docs-links test locally:

1. Open the gitlab directory in a terminal window.

2. Run:

   bundle exec haml-lint -i DocumentationLinks

If you receive an error the first time you run this test, run bundle install, which installs GitLab's dependencies, and try again.

If you don't want to install all of GitLab's dependencies to test the links, you can:

1. Open the gitlab directory in a terminal window.

2. Install haml-lint:

   gem install haml_lint

3. Run:

   haml-lint -i DocumentationLinks

If you manually install haml-lint with this process, it does not update automatically and you should make sure your version matches the version used by GitLab.

Local linters

To help adhere to the documentation style guidelines, and improve the content added to documentation, install documentation linters and integrate them with your code editor.

At GitLab, we mostly use:

• markdownlint
• Vale

markdownlint

markdownlint checks that Markdown syntax follows certain rules, and is used by the docs-lint test.

Our Documentation Style Guide and Markdown Guide elaborate on which choices must be made when selecting Markdown syntax for GitLab documentation. This tool helps catch deviations from those guidelines.

markdownlint configuration is found in the following projects:

• gitlab
• gitlab-runner
• omnibus-gitlab
• charts
• gitlab-development-kit

This configuration is also used within build pipelines.

You can use markdownlint:

• On the command line.
• Within a code editor.
• In a pre-push hook.

Vale

Vale is a grammar, style, and word usage linter for the English language. Vale's configuration is stored in the .vale.ini file located in the root directory of projects.

Vale supports creating custom tests that extend any of several types of checks, which we store in the .linting/vale/styles/gitlab directory within the documentation directory of projects.

Vale configuration is found in the following projects:

• gitlab
• gitlab-runner
• omnibus-gitlab
• charts
• gitlab-development-kit

This configuration is also used within build pipelines.

You can use Vale:

• On the command line.
• Within a code editor.
• In a Git hook. Vale only reports errors in the Git hook (the same configuration as the CI/CD pipelines), and does not report suggestions or warnings.

Install linters

At a minimum, install markdownlint and Vale to match the checks run in build pipelines:

1. Install markdownlint-cli, using either:

   • npm:

     npm install -g markdownlint-cli

   • yarn:

     yarn global add markdownlint-cli

     We recommend installing the version of markdownlint-cli currently used in the documentation linting Docker image.

2. Install vale. For example, to install using brew for macOS, run:

   brew install vale

   We recommend installing the version of Vale currently used in the documentation linting Docker image.

In addition to using markdownlint and Vale at the command line, these tools can be integrated with your code editor.

Configure editors

To configure markdownlint within your editor, install one of the following as appropriate:

• Sublime Text
• Visual Studio Code
• Atom
• Vim

To configure Vale within your editor, install one of the following as appropriate:

• The Sublime Text SublimeLinter-contrib-vale plugin.
• The Visual Studio Code errata-ai.vale-server extension. You don't need Vale Server to use the plugin. You can configure the plugin to display only a subset of alerts.
• Vim.

We don't use Vale Server.

Configure pre-push hooks

Git pre-push hooks allow Git users to:

• Run tests or other processes before pushing a branch.
• Avoid pushing a branch if failures occur with these tests.

lefthook is a Git hooks manager, making configuring, installing, and removing Git hooks easy.

Configuration for lefthook is available in the lefthook.yml file for the gitlab project.

To set up lefthook for documentation linting, see Pre-push static analysis.

Show subset of Vale alerts

You can set Visual Studio Code to display only a subset of Vale alerts when viewing files:

1. Go to Preferences > Settings > Extensions > Vale.
2. In Vale CLI: Min Alert Level, select the minimum alert level you want displayed in files.

To display only a subset of Vale alerts when running Vale from the command line, use the --minAlertLevel flag, which accepts error, warning, or suggestion. Combine it with --config to point to the configuration file within the project, if needed:

vale --config .vale.ini --minAlertLevel error doc/**/*.md

Omit the flag to display all alerts, including suggestion level alerts.

Disable Vale tests

You can disable a specific Vale linting rule or all Vale linting rules for any portion of a document:

• To disable a specific rule, add a <!-- vale gitlab.rulename = NO --> tag before the text, and a <!-- vale gitlab.rulename = YES --> tag after the text, replacing rulename with the filename of a test in the GitLab styles directory.
• To disable all Vale linting rules, add a <!-- vale off --> tag before the text, and a <!-- vale on --> tag after the text.

Whenever possible, exclude only the problematic rule and line(s).

For more information, see Vale's documentation.