How to eliminate duplicate IP Table entries through python program

Question

 iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination
ACCEPT     tcp  --  10.10.10.10          anywhere            tcp dpt:6379
ACCEPT     tcp  --  10.10.10.10          anywhere            tcp dpt:6379

I know iptables-save | awk ' !x[$0]++' | iptables-restore will do the work in shell, but I want to eliminate duplicate IP table entries through python script which runs every 30 seconds.

Answer 1

Something like this?

import subprocess

old_rules = subprocess.run(
    ["iptables-save"], capture_output=True, text=True, check=True)
new_rules = "".join(f"{rule}\n" for rule in set(old_rules.stdout.splitlines()))
saved = subprocess.run(
    ["iptables-restore"], text=True, check=True, input=new_rules)

The middle line is slightly compact; it could be rephrased more readably as

new_rule_lines = set(old_rules.stdout.splitlines())
new_rules = "\n".join(new_rule_lines) + "\n"

The set operation is what removes duplicates here; a Python set is defined as a collection of items where no duplicates are possible.

The final newline is important for some applications, while others will happily read input which lacks the final newline (though it is required by POSIX for text files and streams).

If keeping the original order is a requirement, set() in recent versions of Python should do that, but you might want to explore e.g. Does Python have an ordered set? for a discussion.