Jinja2 Templating: Advanced Patterns

Implement template modularization using Jinja2 `{%#123;% import %%}#125;` and `{%#123;% include %%}#125;` with clear separation of concerns. Create base templates for common patterns (config headers, sections, footers) and extend with application-specific blocks. Store template variables in YAML files grouped by concern: database_vars.yml, networking_vars.yml. Use Jinja2 `{%#123;% macro %%}#125;` for reusable template blocks—macros provide better scoping than includes. Limit templates to <100 lines by extracting complex logic into Ansible filters (Python code in collection plugins). Create template hierarchy: global.j2 → app-type.j2 → app-specific.j2 where each layer adds customizations. Document template variables with comments at file top. Implement template tests with Molecule: render templates with test data and validate output structure. Use variable defaults to reduce conditional complexity: `{{ var | default('default_value') }}`. Create templates that output other formats (JSON/YAML) to shift complex logic to programmatic processing.

Templates are deterministic by default but issues arise from: 1) dict iteration order (use `sort_keys=True` in filters), 2) timestamp variables that change each run, 3) random values. Fix by: using `sort` filter on dict keys: `{{ config | to_nice_yaml(sort_keys=True) }}`. Remove timestamps from templates or use fixed values for validation: `{{ fixed_build_timestamp }}`. Use `dictsort` filter for dict iteration. Implement template normalization: remove whitespace differences, sort lists that don't require order. Create test that renders template multiple times with same input, comparing outputs for exact byte match. Use `skip_idempotent_check: false` to validate idempotency. For critical templates, implement cryptographic checksums: generate SHA256 of template output and verify against known good hash before deployment. Use `changed_when: "False"` on template rendering tasks in validation playbook—rendering itself shouldn't be considered a change. Test with `ansible-playbook --check` to ensure dry-run is deterministic.

Use Jinja2 i18n support via `gettext` library integrated with Ansible. Create translation files (.po files) for each language. In templates, wrap strings with `{{ _('translatable_string') }}`. Use `do` extension to call gettext functions: `{%#123;% do _(variable) %%}#125;`. For date formatting, use `locale` filter: `{{ timestamp | strftime('%x') }}` which respects system locale. For pluralization, use `ngettext`: `{{ _n('item', 'items', count) }}`. Create Ansible filter that handles translations: `{{ text | translate(language='es') }}`. Store translation mappings in YAML and load via lookup plugin. Implement fallback mechanism: if translation missing, default to English. Create template that auto-detects target system locale and applies it. For complex translations, preprocess translations in Ansible tasks and pass to template as vars. Test template rendering with multiple locales to catch translation issues. Document translatable strings in template for translation team. Use tools like `gettext` or `django-i18n` for translation management.

Use Jinja2's `default` filter extensively: `{{ variable | default('fallback_value') }}` returns fallback if undefined. Use `default(omit)` to skip template sections when variables missing. Implement `undefined` handling by setting `jinja2_undefined: StrictUndefined` in ansible.cfg for development (catch errors early), but use `Undefined` in production (silent skipping). Create playbook validation that checks all required variables before template rendering: fail playbook if critical variables undefined. Use `required: true` in filter: `{{ variable | default(required=true) }}` to explicitly fail on critical missing variables. Implement template conditionals to check variable existence: `{%#123;% if variable is defined %%}#125;{{ variable }}{%#123;% endif %%}#125;`. For optional sections, use `default: {}` on dict lookups: `{{ config | default({}) }}`. Create template documentation specifying required vs. optional variables. Pre-populate all variables in playbook, even if empty/null, to prevent undefined issues. Use Ansible `set_fact` to ensure variables exist before template processing.

Create custom Jinja2 filters for each format that handle escaping and serialization. Use built-in `to_json`, `to_yaml`, `to_nice_yaml` filters. For custom formats, implement custom filters in a collection plugin: `plugins/filter/to_ini.py` that accepts dict and returns properly formatted INI. Use Jinja2 variable to specify format: `config_format: json` and conditional include based on format. Implement template hierarchy: `config.j2` that includes format-specific template: `{%#123;% include 'config_' + config_format + '.j2' %%}#125;`. Use data-driven approach: store all configuration data as nested dicts in vars, use filters to serialize to desired format. For complex logic, implement custom filters in Python that handle format-specific rules. Create reusable template blocks that work across formats using filter chains. Implement format validation in playbook: generate config in all formats and validate using language-specific linters. Document filter behavior with examples for each supported format.

Use `no_log: true` on tasks that process sensitive templates—this prevents template output from being logged. Reference secrets from Ansible Vault: `{{ vault_api_key }}` instead of inline. Mark sensitive template variables with `[[ ]]` or custom filter to indicate redaction. Implement custom filter that redacts sensitive values in output: `{{ api_key | redact_sensitive }}`. Use `register: result` and immediately `set_fact: result_redacted` with manual redaction before passing to other tasks. Implement callback plugin to intercept task output and redact sensitive strings. Configure logging to exclude sensitive patterns: update Ansible logging filters to redact known sensitive patterns. Use `ansible.builtin.debug` with `msg` parameter marked `no_log`. For generated files, restrict file permissions with mode `0600`. Implement secret detection scanning in CI/CD to catch leaks before deployment. Use separate vault files for sensitive variables—this makes redaction easier. Test secret handling in staging environment to verify no leakage occurs.

Refactor complex conditionals into Ansible tasks that set intermediate facts. Instead of 15 nested `if` statements in template, use playbook tasks to compute derived facts: `{{ is_production and is_highavail }}` computed as fact before template rendering. Use `vars` section to pre-compute conditional values that template will use. Implement decision matrices: create dict mapping conditions to template variations, then template simply looks up the appropriate variant. Use Ansible filters to transform variables before template sees them. Create test playbook that iterates through all condition combinations and validates output. Implement conditional blocks in templates but keep each block to <5 lines. Use descriptive variable names in templates: `{%#123;% if enable_redundancy %%}#125;` instead of `{%#123;% if var.x == 2 %%}#125;`. Break template into smaller focused templates, each handling specific conditional logic. Implement template debugging by using `debug` module to print intermediate facts and condition values. Create documentation table showing condition combinations and expected output.

Create custom Jinja2 filters in Ansible collection for each transformation. Implement `filter/dns_to_configname.py` that handles all DNS normalization rules. Implement `filter/iprange_to_cidr.py` for IP transformations. Store transformation rules in data files (YAML/JSON) and load them via lookup plugin for external management. Create filter that loads rules from database for runtime flexibility. Use Jinja2 custom test functions for condition checking: implement `test_is_valid_dns.py` to validate DNS names. Centralize filters in collection plugin directory: `collections/ansible_collections/myorg/utils/plugins/filter/`. Document filter behavior with examples. Create tests for each filter using Python `unittest` framework. Implement filter versioning—document when rules change and require version bump. Use filter chaining to compose transformations: `{{ hostname | dns_to_config | sanitize | lowercase }}`. Create Molecule tests that validate filter behavior across use cases. Monitor filter usage to catch breaking transformations before they affect production.

Ansible limits Jinja2 recursion to prevent stack overflow. Work within the limit: default is 1000 levels—most real-world data doesn't exceed this. If you hit limits, restructure data processing: flatten nested structures in Ansible tasks before passing to template. Use loops in playbook instead of template recursion: process each nesting level in separate playbook tasks. Implement iterative algorithm instead of recursive: use `while` loop in playbook to process nested data depth-first or breadth-first. For recursive data processing, implement custom Jinja2 test or filter in Python that handles deep recursion with controlled stack. Set `DEFAULT_JINJA2_NATIVE` to enable native Python rendering for complex types. Create separate playbook passes: first pass renders level-1 configs, second pass uses those as input for level-2, etc. Implement recursion depth monitoring: if rendering takes too long, log and use cached results. For truly recursive data, consider generating in task code (Python) instead of Jinja2 template.