When we want to perform some repetitive tasks on our client nodes for example creating 5 users on each of the worker machines. In this case need to repeat user creation task 5 times and to achieve that we need 5 plays which sometimes could be quite tedious.

As any other language Ansible also provides the looping feature.

Ansible offers two directives for loops creation: loop and with_<lookup>.

Common use cases for Ansible loops:

• Ensure certain number of users are present on remote systems
• Ensure each users home directory contains .ssh directory
• Ensure the desired File/Directory Ownerships permissions are set

Ansible has added loop starting version Ansible 2.5. It is not yet a full replacement for with_<lookup>, but they recommend it for most use cases.

Let us start some practical demonstrations of looping in Ansible playbooks.

Looping over a Simple List

We can use a simple list of input strings which can be defined directly in the task:

Here is our playbook which will create 5 users on our worker nodes by iterating over the list one by one:

loops $ cat loops-simple-list.yml
---
- name: loops test
  hosts: workers
  become: true
  tasks:
    - name: "Create local users"
      user:
        name: "{{ item }}"
        state: present
      loop:
        - test1
        - test2
        - test3
        - test4
        - test5

Here we have used "{{ item }}" variable to access individual data in the loop.

Let us run the playbook.

loops $ ansible-playbook -i myinventory loops-simple-list.yml

Playbook execution is successful.

Verify if users have been created on worker nodes:

loops $ ansible -i myinventory workers -m shell -a "tail -5 /etc/passwd"
worker1 | CHANGED | rc=0 >>
test1:x:1002:1003::/home/test1:/bin/bash
test2:x:1003:1004::/home/test2:/bin/bash
test3:x:1004:1005::/home/test3:/bin/bash
test4:x:1005:1006::/home/test4:/bin/bash
test5:x:1006:1007::/home/test5:/bin/bash
worker2 | CHANGED | rc=0 >>
test1:x:1002:1003::/home/test1:/bin/bash
test2:x:1003:1004::/home/test2:/bin/bash
test3:x:1004:1005::/home/test3:/bin/bash
test4:x:1005:1006::/home/test4:/bin/bash
test5:x:1006:1007::/home/test5:/bin/bash

This is the reason I love Ansible Ad Hoc commands. Great! So users are created.

Please note that you can always define your variables which is a list here in a variable file, or in the vars section of your play, then refer to the name of the list in the task.

Looping over List of Hashes

Hashes are a type of data structure that stores key-value pairs.

Here is our playbook:

loops $ cat loops-hashes.yml
---
- name: loops test while iterating over hashes
  hosts: workers
  become: true
  tasks:
    - name: "Create local users"
      user:
        name: "{{ item.name }}"
        state: present
        groups: "{{ item.groups }}"
      loop:
        - { name: 'test6', groups: wheel }
        - { name: 'test7', groups: wheel }
        - { name: 'test8', groups: root }

Here we have a list of hashes under loop section. We are referring those subkeys in loop.

Let us run the playbook.

loops $ ansible-playbook -i myinventory loops-hashes.yml

Verify the user creation and their group information.

loops $ ansible -i myinventory workers -m shell -a "tail -3 /etc/passwd"
worker2 | CHANGED | rc=0 >>
test6:x:1007:1008::/home/test6:/bin/bash
test7:x:1008:1009::/home/test7:/bin/bash
test8:x:1009:1010::/home/test8:/bin/bash
worker1 | CHANGED | rc=0 >>
test6:x:1007:1008::/home/test6:/bin/bash
test7:x:1008:1009::/home/test7:/bin/bash
test8:x:1009:1010::/home/test8:/bin/bash

loops $ ansible -i myinventory workers -m shell -a "groups test8"
worker1 | CHANGED | rc=0 >>
test8 : test8 root
worker2 | CHANGED | rc=0 >>
test8 : test8 root

loops $ ansible -i myinventory workers -m shell -a "groups test7"
worker1 | CHANGED | rc=0 >>
test7 : test7 wheel
worker2 | CHANGED | rc=0 >>
test7 : test7 wheel

loops $ ansible -i myinventory workers -m shell -a "groups test6"
worker1 | CHANGED | rc=0 >>
test6 : test7 wheel
worker2 | CHANGED | rc=0 >>
test6 : test7 wheel

It's working as expected as you can see users test6 and test7 are part of group wheel and user test8 is part of group root as defined in the playbook.

Looping over Dictionary

A Dictionary consists of Key-Value pairs. In Ansible to iterate over a dictionary, you need to use the dict2items filter. It transforms a dictionary into a list of items.

Here is my playbook:

loops $ cat loops-dictionary.yml
---
 - name: Working with loop module
   hosts: worker1
   tasks:
     - name: Iterate over a dictionary
       debug:
         msg:
          - "The user {{ item.key }} is a {{ item.value }}"

       loop: "{{ user_types | dict2items }}"
       vars:
         user_types:
           test1: dev user
           test2: non-sudo test user
           test8: prod user

We are using debug module just to print some output based on our msg directive. Here keys are : test1, test2, test8 and Values are dev user , non-sudo test user and prod user.

Let us execute the playbook. We are running this playbook only on worker1 this time just to cut short our run output.

loops $ ansible-playbook -i myinventory loops-dictionary.yml

Our playbook execution is successful. We have iterated over the key value pair of a dictionary.

So far we have used loop directive for looping in all above examples. I will take few examples where I will go over with_<lookups>.

with_items - iterate over a predefined list

Here we will loop over a variable list which are defined in a separate variable file. That file we will be referring in our playbook using vars_files directive.

loops $ cat vars.yml
user_names:
  - test9
  - test10
  - test11

Here is our playbook.

loops $ cat loops-with-items-list.yml
---
- name: loops test using with_items
  hosts: workers
  become: true
  vars_files:
       - vars.yml
  tasks:
    - name: "Create local users"
      user:
        name: "{{ item }}"
        state: present
      with_items: '{{ user_names }}''

Let us run the playbook.

loops $ ansible-playbook -i myinventory loops-with-items-list.yml

Verify the user creation.

loops $ ansible -i myinventory workers -m shell -a "tail -3 /etc/passwd"
worker1 | CHANGED | rc=0 >>
test9:x:1010:1011::/home/test9:/bin/bash
test10:x:1011:1012::/home/test10:/bin/bash
test11:x:1012:1013::/home/test11:/bin/bash
worker2 | CHANGED | rc=0 >>
test9:x:1010:1011::/home/test9:/bin/bash
test10:x:1011:1012::/home/test10:/bin/bash
test11:x:1012:1013::/home/test11:/bin/bash

Great! So with_items works very well here. We have all the 3 users created as it iterates over all the items in the list one by one.

with_items - iterate over a predefined dictionary

In this example we will create a bit complex loop. We will iterate over a dictionary defined in a variable file.

loops $ cat vars1.yml
package_names:
  - name: nmap
    state: present
  - name: vsftpd
    state: present
  - name: rsync
    state: absent

Here is our playbook.

loops $ cat loops-with-items-dict.yml
---
- name: loops test using with_items over a dictionary
  hosts: workers
  become: true
  vars_files:
       - vars1.yml
  tasks:
    - name: "Perform package deployments"
      package:
        name: '{{ item.name }}'
        state: '{{ item.state }}'
      with_items: '{{ package_names }}'

Here we are referring package_names variable and dictionary keys and values for name and state as item.name and item.state respectively.

loops $ ansible-playbook -i myinventory loops-with-items-dict.yml

Verify the package status:

loops $ ansible -i myinventory workers -m shell -a "rpm -qa | grep vsftpd" -b -kK
SSH password:
BECOME password[defaults to SSH password]:
worker2 | CHANGED | rc=0 >>
vsftpd-3.0.2-28.el7.x86_64
worker1 | CHANGED | rc=0 >>
vsftpd-3.0.2-28.el7.x86_64

loops $ ansible -i myinventory workers -m shell -a "rpm -qa | grep rsync" -b -kK
SSH password:
BECOME password[defaults to SSH password]:
worker2 | FAILED | rc=1 >>
non-zero return code
worker1 | FAILED | rc=1 >>
non-zero return code

loops $ ansible -i myinventory workers -m shell -a "rpm -qa | grep nmap" -b -kK
SSH password:
BECOME password[defaults to SSH password]:
worker1 | CHANGED | rc=0 >>
nmap-ncat-6.40-19.el7.x86_64
nmap-6.40-19.el7.x86_64
worker2 | CHANGED | rc=0 >>
nmap-ncat-6.40-19.el7.x86_64
nmap-6.40-19.el7.x86_64

The resulting output is as expected. nmap and vsftpd packages have been installed and rsync has been removed from both the worker nodes.

Nested Loops: with_nested

There might be situations where we might need to use nested loops.

Here is our variable file. Under variable keys we have two items (public keys of two users) as part of a list.

loops $ cat vars2-keys.yml
keys:
  - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKsuuvRJV9x6Kpueb5GX5zYn2npScmCLOc+UbMBm1Fsr/EylE4I/Cy1fPjiQ0OAIMx5M7uy1QWstXaDnsekqQVOGlFzMB48NI7IG90exTFj1Sw52pCgZDB9jgbV54f8erGqgdWAm/fVaiVI4GZU67E1dEhSLCoeKT0vgS0wAi2iYNd4nk6QCijNG4nEazt9BBQ0lWvX40UrGts7iW1nDhwCx4nqB8EnZ8du5oo+751JgRycdZiQcCG0vaLePczY4ziQiLM3Z+FlgRVu50wy9iDtRGBs6BlgbkAo0TViHaDRtFDyGD9SYqTuB+XRvUaMUZX7bJl8NXAaxFLBx4WbnWv test2@lco-worker1.example.com
  - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDyumK6lZvu6jFMBMDm4Q8OZJ+OTogJ9RLPYiSvl4XhWo2zCqCfSEnfIrASKGJyx5JZx0PlUVx417fg2ak/2/TEYA+iWTiKi6miZ3t4khF2fFKzmpqmRlXojcvrd9YpND+hMel253dbRrAAVQ3/WaLRDBxduU7CENwgluyJrNKHY9B4LiaLL6/51mkSC/2WyMBL9TzwdSDKr6NVMgbA7mBwaALE5+wW8C7PzwE1X+lAq0pX09k0ypvRugaj3OwaoK6fiPP4QOce8YKq3lP9YTBFi3AgdOAvuRQLntQMAFzRjQBjhQIBcaWJ1Rz/tsgoIGXbJQSK45B19pR2Difugf9f test1@lco-worker1.example.com

Here is our playbook.

loops $ cat loops-with-nested.yml
---
- name: loops test using with_items
  hosts: worker1
  become: true
  vars_files:
    - vars2-keys.yml
  vars:
    usernames:
      - test1
      - test2
  tasks:
    - name: create empty authorized_keys file
      file:
        path: "/home/{{ item }}/.ssh/authorized_keys"
        state: touch
      with_items: '{{ usernames }}'
    - name: "Distribute SSH keys among multiple users"
      lineinfile: dest=/home/{{ item[0] }}/.ssh/authorized_keys line={{ item[1] }} state=present
      with_nested:
        - [ 'test1', 'test2' ]
        - '{{ keys }}'

First task in this playbook will loop over each user part of variable ( usernames) defined in playbook itself and create an empty authorized_keys file in each users home directory.

Second task of playbook will populate those users authorized_keys file with the 2 keys defined in the variable file as a list.

In other words, it’ll iterate over the first value as a list, call it item[0], then get the list from that value’s field named keys, and iterate over that as well, calling it item[1].

Let's execute the playbook.

loops $ ansible-playbook -i myinventory loops-with-nested.yml

It has successfully ran. Let us verify the password less key based authentication between test1 and test2 users on worker1 node.

[test2@lco-worker1 ~]$ ssh test1@lco-worker1.example.com
Last login: Wed Jan 20 08:13:30 2021 from 127.0.0.1

This system is built by the Bento project by Chef Software
More information can be found at https://github.com/chef/bento
[test1@lco-worker1 ~]$
[test1@lco-worker1 ~]$ ssh test2@lco-worker1.example.com
Last login: Wed Jan 20 08:13:17 2021

This system is built by the Bento project by Chef Software
More information can be found at https://github.com/chef/bento
[test2@lco-worker1 ~]$

It works!

I request all the readers to practice it few times to make yourself comfortable as this is one of the important concept of Ansible.

That's all for now.

Hope you like the article. Stay Tuned for more.

Thank you. Happy learning!