Monorepo Optimization and Path Filtering

Use path filtering to trigger workflows only when specific paths change. In your workflow trigger, add: `on: push: paths: [services/auth/**, package-lock.json]`. This workflow only runs if files in `services/auth/` or `package-lock.json` changed. For a monorepo, create a separate workflow per service (or a dynamic workflow that detects changed services). Path filtering is at the event level—if paths don't match, GitHub doesn't even create a workflow run. This saves time and cost. For shared dependencies (root package.json), include them in the paths: `paths: [services/*/**, package-lock.json]`—if root deps change, all services re-run. Implement this per service: `services/auth/.github/workflows/ci.yml` with `paths: [services/auth/**]`. New services inherit the pattern. Caveat: path filtering doesn't support `.githubignore`—you must enumerate paths. For large monorepos (100+ services), use dynamic matrix generation instead: a setup job detects changed services, outputs a matrix, and only those services run CI.

Always include shared config/workflow paths in the path filter: `paths: [services/api/**, .github/workflows/**, .github/actions/**]`. This ensures changes to workflows or shared infrastructure trigger re-evaluation. Better practice: (1) Create a separate workflow trigger for infrastructure changes: `on: push: paths: [.github/**]` → runs on any CI infrastructure change, applies to all services. (2) Use a "workflow-changed" flag: if workflow files changed, force all service CI to run (don't filter). Example: check if `.github/workflows/ci.yml` is in the diff; if so, skip path filtering. (3) Document the rule: "If you change shared infrastructure or workflows, all services will re-run CI. This is intentional—workflows are foundational." (4) For pull_request events, use both `paths` and `pull_request: paths:` to ensure PR-specific filtering works. (5) Test your path filter: create a test PR that only modifies a workflow file and verify all services re-run. This catches configuration mistakes before they affect real workflows.

Update the path filters to support both old and new paths during migration: `on: push: paths: [services/**, app/services/**]`. This triggers on both locations, allowing gradual migration. Run this for 1-2 weeks while you complete the move. Document: "Services have moved to /app/services/. Old /services/ path is deprecated and will be removed 2025-12-31." Once all services are migrated, remove the old path. For dynamic path filters, query the directory structure at runtime: a setup step lists all service directories under the new path and generates a matrix. This doesn't require hardcoding paths in the workflow. Example: `for dir in app/services/*/; do echo "${dir%/}" >> services.txt; done`. Parse this to build the matrix. This approach scales: adding new services auto-updates the matrix without workflow changes.

Use `paths-ignore` to exclude unrelated files: `on: push: paths: [services/auth/**] paths-ignore: [services/auth/**/*.md, services/auth/docs/**]`. This triggers CI for code changes but ignores documentation changes within the service directory. For PR-only filtering (e.g., don't run CI on PRs that only change docs): `pull_request: paths: [services/auth/**] paths-ignore: [**/*.md]`. Combine both: code changes trigger, but pure-doc PRs don't. For monorepos, be explicit about what triggers CI: (1) code/config changes → run CI, (2) docs → skip, (3) tests → run, (4) comments/examples → skip. Define a `.ciignore` convention or document in README. Caveat: `paths` and `paths-ignore` are mutually exclusive per event type; if a file matches `paths-ignore`, it bypasses `paths` matching. So be careful with the order. Example: `paths: [app/**] paths-ignore: [app/**/*.md]` works—only `/app/` files trigger, but .md files in /app/ don't. To skip ONLY specific .md files, use: `paths-ignore: [docs/**, **/*.example]`.

Use a dependency-aware path filter: (1) Create a configuration file (e.g., `.github/service-deps.json`) listing service dependencies: `{ "auth": ["lib", "shared"], "api": ["lib", "shared", "auth"] }`. (2) In a setup step, detect which paths changed via git diff. If a changed path is in dependencies (e.g., `/lib/`), mark all services that depend on it as needing tests. (3) Output a matrix: `services=[auth, api]`. The test job uses this matrix to run only affected services. (4) For example: `/lib/` changes → all services depend on it → test all services. `/services/auth/` changes → only auth needs testing (plus services that depend on auth). (5) Alternatively, use tooling designed for monorepos: NX, Turborepo, or Lerna have built-in dependency graphs. They can compute affected services automatically without manual configuration. (6) For large monorepos (100+ services with complex dependency graphs), consider a dedicated service: a service runs after push and computes the dependency diff, then posts it to a webhook that triggers the appropriate CI jobs.

Path filtering optimizes CI per service, but loses integration testing across services. Add an integration test job: (1) Create a separate workflow for integration tests: `on: pull_request: paths: [services/**, lib/**]` (matches any service or lib change). This workflow runs all services' integration tests together. (2) The integration test job depends on all service CI jobs passing: `needs: [auth-ci, api-ci, web-ci]`. (3) The integration test runs a full system test: spin up all services, test their interactions, ensure no breaking changes. (4) If any service's unit tests fail, integration tests don't run (fast feedback). (5) If all unit tests pass but integration tests fail, mark the PR as "integration failure—manual review required." (6) For monorepos, implement a contract testing approach: each service defines a contract (API signatures, message formats). When a service changes its contract, a breaking-change check fails the build. This catches incompatibilities early, before integration tests. (7) Use a monorepo tool (NX, Turborepo) that has built-in integration test support: specify which services should be tested together, and the tool auto-discovers the dependency graph.

Use dynamic workflow dispatch: (1) Create a single "service-ci" workflow in the root `.github/workflows/service-ci.yml` that's generic. It accepts an input: `inputs: service-name:`. (2) Create individual thin trigger workflows per service (e.g., `.github/workflows/auth-ci.yml`): they use path filtering and call the main workflow: `jobs: trigger: uses: ./.github/workflows/service-ci.yml with: service-name: auth`. (3) The boilerplate per service is just 5 lines; the main logic is centralized. (4) Better: use a dynamic trigger. In the root workflow, detect which services changed and dispatch to the main workflow for each: a setup step runs `git diff`, finds changed services, and calls `workflow_dispatch` for each. (5) For 100 services without individual workflows: use a script in the root workflow to orchestrate. On push, the script detects changed services and runs tests for each (using a matrix or loop). (6) Most scalable: use a monorepo tool (NX, Turborepo) that auto-manages this. You define services once in a config, and the tool generates everything else. No need to manually maintain 100 workflow files.

Path filtering is too granular—it missed a hidden dependency. Better strategy: (1) Identify true independent services: services with zero cross-service dependencies can use path filtering safely. Document: "auth, payments, logging are truly independent; path filtering applies." (2) For services with dependencies, be conservative: if serviceA depends on serviceB, include both in path filters. Example: if web frontend depends on backend APIs, either: (a) both services share a path filter: `paths: [services/web/**, services/backend/**]`, or (b) use dependency declaration (see previous question). (3) For risky changes (API changes, shared styles), skip path filtering: add a label in PRs (e.g., `breaking-change`), and if that label is present, run all tests. (4) Use a cross-cutting filter: if changes to `/lib/`, `/shared/`, `/types/`, run all tests (these affect everything). (5) Implement test coverage tracking: if a file has low test coverage across services, flag it—when it changes, manually review which services might be affected. (6) For true safety, periodically (e.g., nightly on main) run full integration tests on all services. This catches subtle breakages that path filtering misses.