Skip to content

Add verify stage and property test for expect-failure module

Horizon Bot requested to merge add-expect-failure-meta-test into master

Summary

This release adds:

  1. A new verify stage to the stage ordering system
  2. A property test for the expect-failure module that validates both code paths

Add verify stage to stage ordering

  • New stage between test (30) and publish (50)
  • verify = 40, publish = 50, deploy = 60
  • Useful for verification jobs that run after tests

Add property test for expect-failure module

Tests the property: expect-failure(expect-failure(succeeding-job)) = PASS

The test creates two jobs:

  1. job-that-fails: Exits 1 (fails)

    • expect-job-that-fails-fails verifies it failed → PASSES
  2. job-that-succeeds: Exits 0 (succeeds)

    • expect-job-that-succeeds-fails verifies it failed → FAILS (correctly!)
    • Apply expect-failure AGAIN: expect-expect-job-that-succeeds-fails-failsPASSES

This proves:

  • The module correctly handles jobs that fail
  • The module correctly detects when jobs succeed (when expected to fail)
  • The module can be composed with itself (property test)

Rename verify jobs to use expect-{jobName}-fails

  • Changes from verify-{jobName} to expect-{jobName}-fails
  • Makes the property test clearer and more descriptive
  • Updates module documentation and ChangeLog for 0.17.0

Fix expect-failure module for job merging

  • Changed from // (shallow merge) to lib.recursiveUpdate (deep merge)
  • Enables jobs to be both extended and generated in the same evaluation
  • Critical for property test where expect-job-that-succeeds-fails needs both the generated script and the extension attributes (allow_failure)

Changes

  • Add verify stage to stage ordering (nix/modules/flake/gitlab-ci.nix)
  • Add property test: test/negative/expect-failure-meta/
  • Rename verify jobs to expect-{jobName}-fails (nix/modules/gitlab-ci/expect-failure.nix)
  • Fix job merging with recursiveUpdate (nix/modules/gitlab-ci/expect-failure.nix)
  • Add meta test to parent pipeline (nix/configuration/strict-inputs-negative-test.nix)
  • Update ChangeLog to 0.18.0

🤖 Generated with Claude Code

Edited by Horizon Bot

Merge request reports