X Tutup
Skip to content

Use subtests in attestation verification integration tests#10463

Merged
williammartin merged 1 commit intotrunkfrom
wm/use-subtests-in-attestation-verification-integration
Feb 18, 2025
Merged

Use subtests in attestation verification integration tests#10463
williammartin merged 1 commit intotrunkfrom
wm/use-subtests-in-attestation-verification-integration

Conversation

@williammartin
Copy link
Copy Markdown
Member

Description

These tests look like they should be subtests, avoiding the need for manual instrumentation of test names.

No associated issue because this should be a minor change with no UX impact.

➜  go test -v -tags=integration ./pkg/cmd/attestation/verification -count 1

=== RUN   TestLoadBundlesFromJSONLinesFile
=== RUN   TestLoadBundlesFromJSONLinesFile/with_original_file
=== RUN   TestLoadBundlesFromJSONLinesFile/with_extra_lines
--- PASS: TestLoadBundlesFromJSONLinesFile (0.01s)
    --- PASS: TestLoadBundlesFromJSONLinesFile/with_original_file (0.01s)
    --- PASS: TestLoadBundlesFromJSONLinesFile/with_extra_lines (0.01s)
=== RUN   TestLoadBundlesFromJSONLinesFile_RejectEmptyJSONLFile
--- PASS: TestLoadBundlesFromJSONLinesFile_RejectEmptyJSONLFile (0.00s)
=== RUN   TestLoadBundleFromJSONFile
--- PASS: TestLoadBundleFromJSONFile (0.00s)
=== RUN   TestGetLocalAttestations
=== RUN   TestGetLocalAttestations/with_JSON_file_containing_one_bundle
=== RUN   TestGetLocalAttestations/with_JSON_lines_file_containing_multiple_bundles
=== RUN   TestGetLocalAttestations/with_file_with_unrecognized_extension
=== RUN   TestGetLocalAttestations/with_non-existent_bundle_file_and_JSON_file
=== RUN   TestGetLocalAttestations/with_non-existent_bundle_file_and_JSON_lines_file
=== RUN   TestGetLocalAttestations/with_missing_verification_material
=== RUN   TestGetLocalAttestations/with_missing_verification_certificate
--- PASS: TestGetLocalAttestations (0.01s)
    --- PASS: TestGetLocalAttestations/with_JSON_file_containing_one_bundle (0.00s)
    --- PASS: TestGetLocalAttestations/with_JSON_lines_file_containing_multiple_bundles (0.00s)
    --- PASS: TestGetLocalAttestations/with_file_with_unrecognized_extension (0.00s)
    --- PASS: TestGetLocalAttestations/with_non-existent_bundle_file_and_JSON_file (0.00s)
    --- PASS: TestGetLocalAttestations/with_non-existent_bundle_file_and_JSON_lines_file (0.00s)
    --- PASS: TestGetLocalAttestations/with_missing_verification_material (0.00s)
    --- PASS: TestGetLocalAttestations/with_missing_verification_certificate (0.00s)
=== RUN   TestFilterAttestations
--- PASS: TestFilterAttestations (0.00s)
=== RUN   TestVerifyCertExtensions
=== RUN   TestVerifyCertExtensions/passes_with_one_result
=== RUN   TestVerifyCertExtensions/passes_with_1/2_valid_results
=== RUN   TestVerifyCertExtensions/fails_when_all_results_fail_verification
=== RUN   TestVerifyCertExtensions/with_wrong_SourceRepositoryOwnerURI
=== RUN   TestVerifyCertExtensions/with_wrong_SourceRepositoryURI
=== RUN   TestVerifyCertExtensions/with_wrong_OIDCIssuer
=== RUN   TestVerifyCertExtensions/with_partial_OIDCIssuer_match
--- PASS: TestVerifyCertExtensions (0.00s)
    --- PASS: TestVerifyCertExtensions/passes_with_one_result (0.00s)
    --- PASS: TestVerifyCertExtensions/passes_with_1/2_valid_results (0.00s)
    --- PASS: TestVerifyCertExtensions/fails_when_all_results_fail_verification (0.00s)
    --- PASS: TestVerifyCertExtensions/with_wrong_SourceRepositoryOwnerURI (0.00s)
    --- PASS: TestVerifyCertExtensions/with_wrong_SourceRepositoryURI (0.00s)
    --- PASS: TestVerifyCertExtensions/with_wrong_OIDCIssuer (0.00s)
    --- PASS: TestVerifyCertExtensions/with_partial_OIDCIssuer_match (0.00s)
=== RUN   TestLiveSigstoreVerifier
=== RUN   TestLiveSigstoreVerifier/with_invalid_signature
=== RUN   TestLiveSigstoreVerifier/with_valid_artifact_and_JSON_lines_file_containing_multiple_Sigstore_bundles
=== RUN   TestLiveSigstoreVerifier/with_invalid_bundle_version
=== RUN   TestLiveSigstoreVerifier/with_no_attestations
=== RUN   TestLiveSigstoreVerifier/with_2/3_verified_attestations
=== RUN   TestLiveSigstoreVerifier/fail_with_0/2_verified_attestations
=== RUN   TestLiveSigstoreVerifier/with_GitHub_Sigstore_artifact
=== RUN   TestLiveSigstoreVerifier/with_custom_trusted_root
--- PASS: TestLiveSigstoreVerifier (2.21s)
    --- PASS: TestLiveSigstoreVerifier/with_invalid_signature (0.46s)
    --- PASS: TestLiveSigstoreVerifier/with_valid_artifact_and_JSON_lines_file_containing_multiple_Sigstore_bundles (0.69s)
    --- PASS: TestLiveSigstoreVerifier/with_invalid_bundle_version (0.00s)
    --- PASS: TestLiveSigstoreVerifier/with_no_attestations (0.00s)
    --- PASS: TestLiveSigstoreVerifier/with_2/3_verified_attestations (0.68s)
    --- PASS: TestLiveSigstoreVerifier/fail_with_0/2_verified_attestations (0.32s)
    --- PASS: TestLiveSigstoreVerifier/with_GitHub_Sigstore_artifact (0.03s)
    --- PASS: TestLiveSigstoreVerifier/with_custom_trusted_root (0.02s)
=== RUN   TestGitHubTUFOptions
--- PASS: TestGitHubTUFOptions (0.00s)
PASS
ok      github.com/cli/cli/v2/pkg/cmd/attestation/verification  2.536s

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot reviewed 1 out of 1 changed files in this pull request and generated no comments.

Comments suppressed due to low confidence (2)

pkg/cmd/attestation/verification/sigstore_integration_test.go:51

  • Consider capturing the loop variable (tc) inside the loop (e.g., adding 'tc := tc' before calling t.Run) to avoid potential issues with variable capture if subtests are run in parallel in the future.
t.Run(tc.name, func(t *testing.T) {

pkg/cmd/attestation/verification/sigstore_integration_test.go:59

  • [nitpick] While subtest names already provide context, consider adding additional descriptive messages to assertion errors if further clarification is needed for debugging in complex scenarios.
require.Error(t, err)

Tip: Copilot code review supports C#, Go, Java, JavaScript, Markdown, Python, Ruby and TypeScript, with more languages coming soon. Learn more

require.ErrorContains(t, err, tc.errContains)
require.Nil(t, results)
} else {
require.NoError(t, err)
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reordered the error check here for idiomatic reasons, but let me know if that's a functional change.

@williammartin williammartin merged commit 187a1bb into trunk Feb 18, 2025
17 checks passed
@williammartin williammartin deleted the wm/use-subtests-in-attestation-verification-integration branch February 18, 2025 15:57
tmeijn pushed a commit to tmeijn/dotfiles that referenced this pull request Mar 6, 2025
This MR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [cli/cli](https://github.com/cli/cli) | minor | `v2.67.0` -> `v2.68.0` |

MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot).

**Proposed changes to behavior should be submitted there as MRs.**

---

### Release Notes

<details>
<summary>cli/cli (cli/cli)</summary>

### [`v2.68.0`](https://github.com/cli/cli/releases/tag/v2.68.0): GitHub CLI 2.68.0

[Compare Source](cli/cli@v2.67.0...v2.68.0)

#### What's Changed

##### ✨ Features

-   \[gh repo view] Improve error message for forked repo by [@&#8203;iamazeem](https://github.com/iamazeem) in cli/cli#10334
-   Add signer-digest, source-ref, and source-digest options for `gh attestation verify` by [@&#8203;malancas](https://github.com/malancas) in cli/cli#10308
-   \[gh pr checkout] Add --no-tags option to git fetch commands in checkout by [@&#8203;latzskim](https://github.com/latzskim) in cli/cli#10479
-   \[`gh issue/pr comment`] Add `--create-if-none` and prompts to create a comment if no comment already exists  by [@&#8203;latzskim](https://github.com/latzskim) in cli/cli#10427
-   \[gh cache delete --all] Add `--succeed-on-no-caches` flag to return exit code 0 by [@&#8203;iamazeem](https://github.com/iamazeem) in cli/cli#10327
-   \[gh release create] Fail when there are no new commits since the last release by [@&#8203;iamazeem](https://github.com/iamazeem) in cli/cli#10398
-   update default upstream when forking repo during MR creation by [@&#8203;daviddl9](https://github.com/daviddl9) in cli/cli#10458

##### 🐛 Fixes

-   Refactor `GetLocalAttestations` and clean up custom registry transport by [@&#8203;malancas](https://github.com/malancas) in cli/cli#10382
-   Check `GH_REPO` too in addition to `--repo` for disambiguation by [@&#8203;williammartin](https://github.com/williammartin) in cli/cli#10539
    -   (Fixes `gh secret` subcommands not working outside of a repository)
-   Fix unhandled panic in FindWorkflow and add tests by [@&#8203;jtmcg](https://github.com/jtmcg) in cli/cli#10521
-   Fix checkout when URL arg is from fork and cwd is upstream by [@&#8203;williammartin](https://github.com/williammartin) in cli/cli#10512
-   \[gh api] Escape package name (URL encoding) for packages endpoint by [@&#8203;iamazeem](https://github.com/iamazeem) in cli/cli#10384
-   Fix `remoteResolver` caching issue by [@&#8203;iamazeem](https://github.com/iamazeem) in cli/cli#10456
-   Fix gh project item-edit to allow --number 0 as a valid value by [@&#8203;aryanbhosale](https://github.com/aryanbhosale) in cli/cli#10417
-   Add mutex to fix race in attestation test client by [@&#8203;codysoyland](https://github.com/codysoyland) in cli/cli#10439
-   Base64 decode GPG passphrase in deployment workflow by [@&#8203;BagToad](https://github.com/BagToad) in cli/cli#10546

##### 📚 Docs & Chores

-   Deep Dive Document Release Process by [@&#8203;williammartin](https://github.com/williammartin) in cli/cli#10503
-   Inconsistent format of examples in help text by [@&#8203;iamazeem](https://github.com/iamazeem) in cli/cli#10508
-   Inconsistent format of description of flags (starting with lowercase letter) by [@&#8203;iamazeem](https://github.com/iamazeem) in cli/cli#10507
-   Update Go version to 1.23 in CONTRIBUTING.md by [@&#8203;williammartin](https://github.com/williammartin) in cli/cli#10504
-   Fix minor auth login help typo by [@&#8203;williammartin](https://github.com/williammartin) in cli/cli#10501
-   docs: document how to revoke `gh` OAuth tokens in `auth logout`'s help by [@&#8203;BagToad](https://github.com/BagToad) in cli/cli#10490
-   chore: update codespaces Go version by [@&#8203;BagToad](https://github.com/BagToad) in cli/cli#10491
-   Allow injection of TUFMetadataDir in tests by [@&#8203;williammartin](https://github.com/williammartin) in cli/cli#10478
-   refactor: use a more straightforward return value by [@&#8203;beforetech](https://github.com/beforetech) in cli/cli#10489
-   Use subtests in attestation verification integration tests by [@&#8203;williammartin](https://github.com/williammartin) in cli/cli#10463
-   Fix typo in README by [@&#8203;iamazeem](https://github.com/iamazeem) in cli/cli#10445
-   Update usage to lower-kebab-case by [@&#8203;iamazeem](https://github.com/iamazeem) in cli/cli#10447
-   Standardize URLs by [@&#8203;iamazeem](https://github.com/iamazeem) in cli/cli#10429
-   Remove trailing whitespace by [@&#8203;iamazeem](https://github.com/iamazeem) in cli/cli#10430

##### :dependabot: Dependencies

-   Bump actions/attest-build-provenance from 2.2.0 to 2.2.2 by [@&#8203;dependabot](https://github.com/dependabot) in cli/cli#10518
-   Bump github.com/go-jose/go-jose/v4 from 4.0.2 to 4.0.5 by [@&#8203;dependabot](https://github.com/dependabot) in cli/cli#10499
-   Bump github.com/spf13/pflag from 1.0.5 to 1.0.6 by [@&#8203;dependabot](https://github.com/dependabot) in cli/cli#10338

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this MR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this MR, check this box

---

This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzOS4xODYuMCIsInVwZGF0ZWRJblZlciI6IjM5LjE4Ni4wIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJSZW5vdmF0ZSBCb3QiXX0=-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

X Tutup