Looping through a file and using columns as variables

When you have to write a lot of scripts to automate things eventually you are going to hit the need to have a variable, but what if you need 2 or 3 variables, and what if those need to be read from a file and imported into your running script in real-time as variables? No problem we have you covered, with this simple guide you should be looping through files and fetching variables for your scripts like a professional!

First of all, let’s create our file with a bunch of variables, as you will see below we have some variables, in column 1 we have the ping command, in column 2 the tracepath (traceroute) command, and then finally in column 3 we have and IP address

Example data:
ping tracepath
ping tracepath
ping tracepath

After putting the data above in a file save it as data in the /tmp directory.

Now we need to create the script that will use that data, for example:

while read -r first second third; do
$first $third > /tmp/$thrid.ping.results
$second $third > /tmp/$second.traceroute.results

What that does line by line is:

Line 1: read line by line and capture the first, second and third as variables (you can do more than three) then do something.

Line 2: because the first word in the first column is ping, $first is equal to writing ping in your script $third is the IP as it is the third column so you are pinging the IP then you are also using the IP address in the file name ($third) to record the results $third.ping.results will produce files named and so on.

Line 3: This is similar to line 2 except instead of ping we are using tracepath with is represented by column 2 $second and again is recording the results using the IP address, this will product files named

Line 4: This is where we tell the script where to find the input file, in plain English,

it means loop through this file until done.

This is a very simple example however it is functional and should give you the tools you need to create your own scripts,

Another practical example using only 2 columns:

You want to create a set of iptables rules that blocks a list of IP’s you may have got from your security logs from various ports

You build your file and save it as /tmp/data

Example data: 22 80 443

Now you have 2 options, you can either use the data to run the desired commands directly or you can run a script that then builds a list of iptables rules you might want to execute on multiple servers later:

To run the commands directly using the variables:

while read -r first second; do
iptables -A INPUT -p tcp -i eth0 ! -s $first --dport $second

In the above example, $first represents the IP address, $second represents the port

If you want to make a script that simply generates a bunch of iptables commands to be saved and used later it would look something like this:

while read -r first second; do
echo "iptables -A INPUT -p tcp -i eth- ! -s $first --dport $second" >> /tmp/ruleset1

In the above example, the commands containing the variables will be converted into the actual data 1 line at a time from your data file will be exported to /tmp/dataset1 and will look something like this:

iptables -A INPUT -p tcp -i eth0 ! -s --dport 22
iptables -A INPUT -p tcp -i eth0 ! -s --dport 80
iptables -A INPUT -p tcp -i eth0 ! -s --dport 443

The lists can be as long or as short as you want.