Our CodeQL CLI pipeline creates autogenerated documentation for docs.github.com from the source code used to build the CodeQL CLI.

The pipeline is used to generate Markdown files that create article pages on the docs.github.com site.

A workflow is used to trigger the automation of the CodeQL CLI documentation. The workflow is manually triggered by a member of the GitHub Docs team approximately every two weeks to align to releases of the CodeQL CLI. The workflow takes an input parameter that specifies the branch to pull the source files from in the semmle-code repo. If the branch input is omitted, the workflow will default to the main branch.

The workflow runs the src/codeql-cli/scripts/sync.ts script, which generates Markdown files under content/code-security/codeql-cli/codeql-cli-manual.

The workflow automatically creates a new pull request with the changes and the label codeql-cli-pipeline.

To run the CodeQL CLI pipeline locally:

1. Clone the semmle-code repository inside your local docs-internal repository.
2. Install Pandoc. You can brew install pandoc on macOS.
3. Run src/codeql-cli/scripts/sync.ts.

• src/codeql-cli/lib/config.json - A configuration file used to specify metadata about the CodeQL CLI pipeline.
• src/codeql-cli/scripts - The scripts and source code used run the CodeQL CLI pipeline.
  • src/codeql-cli/scripts/sync.ts - The entrypoint script that runs the CodeQL CLI pipeline.

The content writers can manually update parts of the autogenerated Markdown files in content/code-security/codeql-cli/codeql-cli-manual. When new Markdown files are added they will get all of the frontmatter properties defined in the defaultFrontmatter property in src/codeql-cli/lib/config.ts.

When a new Markdown file is created, a writer can manually change any of the frontmatter. The pipeline will not overwrite the frontmatter on subsequent runs.

Writers can also add an introduction paragraph above the following Markdown comment:

<!-- Content after this section is automatically generated -->

Slack: #docs-engineering Repo: github/docs-engineering

If you have a question about the CodeQL CLI pipeline, you can ask in the #docs-engineering Slack channel. If you notice a problem with the CodeQL CLI pipeline, you can open an issue in the github/docs-engineering repository.

• Repository: github/semmle-code (private)
• Branch: Configurable (default: main)
• Source files: ReStructuredText documentation in semmle-code
• Output location: content/code-security/codeql-cli/codeql-cli-manual/

• CodeQL CLI releases approximately every 2 weeks
• Pipeline should be run to align with CLI releases
• Multiple CLI versions may be documented simultaneously

• Team: Docs Engineering
• Source team: Code Security (CodeQL team)

1. Pipeline failures → #docs-engineering Slack
2. Source data issues → #code-security-docs Slack
3. Pandoc conversion issues → Check Pandoc version and file format
4. Repo access issues → Request access to github/semmle-code

If the pipeline fails:

1. Check workflow logs for errors
2. Verify Pandoc is installed and working
3. Confirm access to semmle-code repo
4. Check for ReStructuredText format changes
5. Test conversion locally before re-running
6. Escalate to Code Security team if source data issue

• Pandoc not found - Install Pandoc locally or in CI
• semmle-code not cloned - Clone repo at docs-internal root
• Conversion errors - Check for unsupported RST syntax
• Branch not found - Verify branch name in workflow input

• Manual trigger only - No automated schedule (runs on-demand)
• Pandoc dependency - Requires external tool installation
• Repo access - Requires access to private semmle-code repo
• Local testing - Must clone semmle-code locally to test

• Frontmatter preservation - Only first sync sets frontmatter
• Introduction placement - Must be above auto-generation comment
• No inline editing - Cannot edit autogenerated sections
• Version handling - Limited liquid versioning support

• ReStructuredText format - Source must be valid RST
• Pandoc conversion - Limited control over conversion output
• File naming - Determined by source file structure
• Link rewriting - May need manual adjustment after generation

• Release alignment - Sync timing depends on CodeQL releases
• PR review required - Changes must be reviewed before merge
• Content coordination - Writers must coordinate intro additions
• Multiple versions - May need separate syncs for version branches

We are not expecting significant new work to this pipeline, but we will support incoming changes from semmle-code.