Implement module system for composable CI configuration

Implement Module System for Composable CI Configuration

Overview

Refactor nix/gitlab-ci to use a dedicated module system, enabling better composability of CI jobs and allowing domain-specific tools to ship their own CI modules.

Proposed Architecture

Core Module System

Implement in nix/gitlab-ci:

• Two-level module system (flake-parts wrapper + GitLab CI modules)
• Use lib.evalModules for proper module evaluation
• Define base types: jobs, stages, variables, artifacts, etc.

Common Modules (in nix/gitlab-ci)

Generic/reusable modules that stay in this repo:

• modules/omnix.nix - Omnix build backend
• modules/cachix.nix - Cachix integration
• modules/discover.nix - CI discovery
• modules/validate-check.nix - Generic validation pattern (run command, check git diff)

Domain-Specific Modules (in their own repos)

Tools ship their own CI modules:

• horizon-gen-nix would provide its own validation module
• Other domain tools can provide their own integrations
• Import via flake inputs

Implementation Design

1. Core Module System

# lib/mkGitLabCI.nix
{ lib }:
let
  evalGitLabCI = modules: lib.evalModules {
    modules = [ ./modules/base.nix ] ++ modules;
    specialArgs = { inherit lib; };
  };
in {
  mkGitLabCI = modules: (evalGitLabCI modules).config;
}

2. Base Types

# modules/base.nix
{ lib, ... }: {
  options = {
    jobs = lib.mkOption {
      type = lib.types.attrsOf (lib.types.submodule {
        options = {
          stage = lib.mkOption { type = lib.types.str; default = "build"; };
          script = lib.mkOption { type = lib.types.listOf lib.types.str; };
          needs = lib.mkOption { type = lib.types.listOf lib.types.str; default = []; };
          rules = lib.mkOption { type = lib.types.listOf lib.types.attrs; default = []; };
        };
      });
      default = {};
    };
    
    stages = lib.mkOption {
      type = lib.types.listOf lib.types.str;
    };
  };
  
  # Auto-collect stages from jobs
  config.stages = lib.mkDefault (
    lib.unique (lib.mapAttrsToList (_: job: job.stage) config.jobs)
  );
}

3. Common Modules

omnix module:

# modules/omnix.nix
{ config, lib, ... }: {
  options.omnix = {
    enable = lib.mkEnableOption "omnix build system";
    cachix = lib.mkOption { type = lib.types.str; };
    systems = lib.mkOption { type = lib.types.listOf lib.types.str; };
  };
  
  config = lib.mkIf config.omnix.enable {
    jobs = {
      # Generate devour and build jobs
    };
  };
}

Generic validate-check module:

# modules/validate-check.nix
{ config, lib, ... }: {
  options.validateCheck = lib.mkOption {
    type = lib.types.attrsOf (lib.types.submodule {
      options = {
        command = lib.mkOption { type = lib.types.str; };
        stage = lib.mkOption { type = lib.types.str; default = "validate"; };
      };
    });
    default = {};
  };
  
  config.jobs = lib.mapAttrs (name: cfg: {
    inherit (cfg) stage;
    script = [
      cfg.command
      "git diff --exit-code"
    ];
  }) config.validateCheck;
}

4. Flake-Parts Integration

# flakeModule.nix
perSystem = { config, ... }: {
  gitlab.ci = {
    imports = [
      # Common modules from gitlab-ci
      inputs.gitlab-ci.modules.omnix
      inputs.gitlab-ci.modules.cachix
      
      # Domain-specific from their repos
      inputs.horizon-gen-nix.ciModule
    ];
    
    # Configure modules
    omnix.enable = true;
    omnix.systems = ["x86_64-linux"];
    cachix.cache = "horizon";
  };
};

Example: Domain-Specific Module

In horizon-gen-nix repository:

# ciModule.nix
{ config, lib, ... }: {
  options.validateGeneratedNix = {
    enable = lib.mkEnableOption "generated nix validation";
    version = lib.mkOption {
      type = lib.types.str;
      default = "0.14.0";
    };
  };
  
  config = lib.mkIf config.validateGeneratedNix.enable {
    jobs.validate-generated-nix = {
      stage = "validate";
      script = [
        "nix run git+https://gitlab.horizon-haskell.net/haskell/horizon-gen-nix?ref=refs/tags/${config.validateGeneratedNix.version}"
        "git diff --exit-code"
      ];
    };
  };
}

# Export in flake.nix
outputs = { ... }: {
  ciModule = import ./ciModule.nix;
};

Benefits

1. Composability: Mix common modules (omnix, cachix) with domain-specific ones
2. Reusability: Common patterns stay in gitlab-ci, domain tools provide their own
3. Type safety: Full module system with proper option types
4. Discoverability: nix eval .#gitlab.ci.options shows available options
5. Less nesting: Flatter configuration structure
6. Extensibility: Any tool can ship a CI module

Usage Comparison

Before:

perSystem = { system, ... }: {
  gitlab.ci.omnix = {
    enable = system == local.config.horizon.systems.ci.host;
    cachix = "horizon";
    systems = local.config.horizon.systems.ci.targets;
  };
  
  gitlab.ci.jobs.validate-generated-nix = {
    stage = "validate";
    script = [...];
  };
};

After:

perSystem = { ... }: {
  gitlab.ci = {
    imports = [
      inputs.gitlab-ci.modules.omnix
      inputs.gitlab-ci.modules.cachix
      inputs.horizon-gen-nix.ciModule
    ];
    
    omnix.enable = true;
    omnix.systems = local.config.horizon.systems.ci.targets;
    cachix.cache = "horizon";
    validateGeneratedNix.enable = true;
  };
};

Repository Structure

nix/gitlab-ci/
├── flakeModule.nix              # Flake-parts integration
├── lib/
│   └── mkGitLabCI.nix          # Module evaluator
├── modules/
│   ├── base.nix                # Core types and options
│   ├── omnix.nix               # Omnix backend
│   ├── cachix.nix              # Cachix integration
│   ├── discover.nix            # CI discovery
│   └── validate-check.nix      # Generic validation pattern
└── README.md                    # Documentation

Migration Plan

1. Implement core module system in nix/gitlab-ci
2. Refactor existing functionality as modules
3. Update documentation with examples
4. Migrate existing projects (horizon-build-packages, horizon-core, etc.)
5. Create example domain-specific module (horizon-gen-nix)

Open Questions

1. Should we keep backwards compatibility with current structure?
2. What other common patterns should be modules?
3. Naming conventions for domain-specific CI modules?