Create dict based on values from list in Ansible

Question

I have Ansible role that creates buckets in minio based on dict of "clients", and every "client" is a dict itself containing data needed for it.

The problem is: every "client" is stored as its own Vault secret, and I need to combine them into one dict.

So, for example, we have initial list of "clients":

client_names:
  - exchange
  - models

And every "client" has secret in Vault on path secrets/data/minio/clients/{{client}}:

access_key: exchange
secret_key: exchange-password

And what I need to get as result is a dict like this:

minio_clients:
  exchange:
    access_key: exchange
    secret_key: exchange-password
  models:
    access_key: models
    secret_key: models-password

I have seen some implementations of dynamic dict creation by using set_fact, but it would be better if I could implement this in group_vars variables.

larsks · Accepted Answer · 2023-06-28 14:51:24Z

1

You can do this using jinja templating and the community.hashi_vault collection.

First, make sure the collection is installed:

ansible-galaxy collection install community.hashi_vault

Now create a variables file like this:

client_names:
  - exchange
  - models

minio_clients: |
  {% filter from_yaml %}
  {% for name in client_names %}
  {% set res = lookup("community.hashi_vault.vault_kv2_get", "minio/clients/%s" % name).data.data %}
  {{ name }}:
    access_key: "{{ res.access_key }}"
    secret_key: "{{ res.secret_key }}"
  {% endfor %}
  {% endfilter %}

With the above content in group_vars/all.yaml (and your example entries in my local vault instance), running this playbook...

- hosts: localhost
  gather_facts: false
  tasks:
    - debug:
        msg: "{{ minio_clients }}"

...produces this output:

TASK [debug] *******************************************************************
ok: [localhost] => {
    "msg": {
        "exchange": {
            "access_key": "exchange",
            "secret_key": "exchange-password"
        },
        "models": {
            "access_key": "models",
            "secret_key": "models-password"
        }
    }
}

answered Jun 28, 2023 at 14:51

larsks

318k50 gold badges474 silver badges483 bronze badges

Sign up to request clarification or add additional context in comments.

2 Comments

Роман Леухин Over a year ago

I tried to implement your approach, and Ansible throws an error "sequence item 0: expected str instance, dict found"

larsks Over a year ago

I tested this exact configuration using Ansible 2.15.1 and it runs without errors. Did you copy and paste the exact variables file and playbook? Are you successfully authenticated to your vault (using the VAULT_ADDR and VAULT_TOKEN environment variables, and possibly also VAULT_CACERT depending on your configuration)?

Vladimir Botka · Accepted Answer · 2023-06-28 19:55:19Z

Given the directory for testing

  vault_dir: /tmp/ansible/vault/clients/

Create the files

shell> tree /tmp/ansible/vault/clients/
/tmp/ansible/vault/clients/
├── exchange
└── models

0 directories, 2 files

shell> cat /tmp/ansible/vault/clients/exchange 
access_key: exchange
secret_key: exchange-password

shell> cat /tmp/ansible/vault/clients/models 
access_key: models
secret_key: models-password

^{and encrypt them
shell> ansible-vault encrypt /tmp/ansible/vault/clients/exchange
Encryption successful

shell> ansible-vault encrypt /tmp/ansible/vault/clients/models
Encryption successful

shell> cat /tmp/ansible/vault/clients/exchange
$ANSIBLE_VAULT;1.1;AES256
35333063626236396534646161373832336231386561633331396465366439653364623433646433
3635393634386330393131653239346232656563623033390a343639333066633732306364613839
31663438643334386665383334386337353438613166633035653535656264323563383662303538
3037366337663465360a383733313964313936633834366234386163656531326161643833333933
34353763313863373864356634313163376138383130663461663565666165663236613538396537
36363032613662323861353136366362666465313661613061366530613330383763393532383938
623835633639356437663938323064343933

shell> cat /tmp/ansible/vault/clients/models
$ANSIBLE_VAULT;1.1;AES256
30323130633566353338313663343464313137643035353963646635616435353364633965633038
6266353366383735613336353238613261376164633166390a373966386330663366346563356263
35363931643230323232353130303434336139326433663434323731616635663434396238393438
6431363266373564360a666132363664663764306438353939623663333439643331396339373839
65393663393939616538623363633532386533393163373034356665643663613864616331643966
3363616531303038646234633163383363633733653136643562}

Given the list client_names the below template

  client_names: [exchange, models]
  clients: |
    {% filter from_yaml %}
    {% for client in client_names %}
    {{ client }}:
      {{ lookup('file', vault_dir ~ client)|indent(2) }}
    {% endfor %}
    {% endfilter %}

gives

  clients:
    exchange:
      access_key: exchange
      secret_key: exchange-password
    models:
      access_key: models
      secret_key: models-password

If you want to include all files the list of the clients is not necessary. The below template gives the same result

  fileglob: "{{ vault_dir }}*"
  clients: |
    {% filter from_yaml %}
    {% for path in q('fileglob', fileglob) %}
    {{ path|basename }}:
      {{ lookup('file', path)|indent(2) }}
    {% endfor %}
    {% endfilter %}

^{Example of a complete playbook for testing
- hosts: all

vars:

vault_dir: /tmp/ansible/vault/clients/

client_names: [exchange, models]
clients: |
{% filter from_yaml %}
{% for client in client_names %}
{{ client }}:
{{ lookup('file', vault_dir ~ client)|indent(2) }}
{% endfor %}
{% endfilter %}

fileglob: "{{ vault_dir }}*"
client2: |
{% filter from_yaml %}
{% for path in q('fileglob', fileglob) %}
{{ path|basename }}:
{{ lookup('file', path)|indent(2) }}
{% endfor %}
{% endfilter %}

tasks:

- debug:
var: clients
- debug:
var: client2}

I see we have interpreted "vault" differently. I'm pretty sure the OP was asking about HashiCorp Vault (based on their language and the example paths), but I guess we will only know if they come back :).
Yeah, i was talking about hashi_vault, but your example still stands, Thank you!

Collectives™ on Stack Overflow

Create dict based on values from list in Ansible

2 Answers 2

2 Comments

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

2 Comments

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related