sed to delete all lines except last line based on string match

Question

Got autorized_keys with old key from a specific user say alice and would like to retain last amended public key & delete rest of them.

Here is example content of autorized_keys file:

ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAIEA0YnNYVo/lHtSLtDUpQIqgS8WIpoBA7NRfaheQZXjqOjDMmBZA0cJy+ng3kmUuh5SbNapjaAYFwMHjLv6biHAasWgE77rJYXN8+JpHi/JwuXUnVUTwHrKXvkMr0sPgDxCEZO15GtiJwVPfmxpl3RBB/we5a+A6fhvXCyk/dxQRdU= alice-publickey
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDp6d0GmH5Q+lbx832n2W43Ikx8cpTeM8MRQ1yJmnhzaCSkQk9jWkwB5Msvj7uR923k0qLCZQ4ZbdPQ7liK9+Ks/7c0yDrBkU1KVOBNDQD1ck9Iwo2qwsS72b1kf4zeNKVJWkKBcFg7qxOmjIy9wNJ0XhbHmo/9gkp/hoOY3g0I5tExIlVdTH+uhEmOerghChTHTWU/kymZW0yYbDKl9zyPEDY9Hf/0Mt6g2dz+q3/cfcwCIZIeR5/Fjt7xXbt+fqmaiws1veGacw60u8ouMJ4DL9CcXRUbZHdEY4v2yIcEZRXMv9Q/6s0TcuC9s6Z5n2RzV8eW3RCZuzMWmYlkcA/H bob-publickey
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAIEAsB3XsrmTHWygTrL8dT/Z5+qjjXfPxH7IPlyFZt0QcHA1a16g0ZPeEMB1RUu90vtbEvKjq+hAAANeLcmVW85i8LSsO3xcuCGTFNVNx2up7Fv7YxiHMJpfQqyiBW09HixrPiu+9Wl/olTB9kPu+fZ/jRqok3ZTARnQDoE2g1RWnNU= alice@mcosmac
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2UFK8Y7mODUZ7a2glErZbKk9hUy1pCOiGz4arAlTBiJ884vrjGaYNGFyWiNvqNKxLWn6Cul1eyPF1M/Z6l6QE3biJgJPln2ZZIoMcsNiCrFl0x5tUdnyVnVGz5pZTI1/0IFFLE4jbW2grMHOC7z+ik3WOD5aTj8pUWeTIeppMKLwjnJhm0ALsmLZqhtwnhgLSg9m4NQmcLWt5s0P/Z91hUKw2tY5iV9yOu5tS+oTMoeV7ays9MXM9ozpehLhsbqn98O7k2UpQzoWEo093lAjzydnBICd5tKdVJyGcyi3BLY4xhU+dnJ9NIUuYtnLZEhAGCbOQEGNivkxDQv1ccRTx trudy@newMacs
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPbEzZDSZNKdGsH0sP10abmosX0yFquCEx2ibNzmh0AvvsR6pbA/pdl1i9XqIJlDBustWfmnChXEQz9Nft5LbM/moR6DAe7Wd70kmY8CreGgZ2kF8KCZ3bTkKYfO5PexKRCZ8IFRs6xswbfV0DJOmGnNo+wKOFDK/J7GVd9e1u6FebSo/Nr0QN/maXKN+Tc57GnV+j34ihD9PoRmfGYRIjm/vMkfUB9l1fZDEawdMO+Ats+G9gSvlhIm4TSP4sHk+zUJJff+j6ubMsGW0Flh4YjVJMMSU9vrpsTS5WiEbsZVxuWkrd+9jMo5RGDoWt2HADo4Dup0DlyM1pj2u1zbfl alice@brandNew

Below option looks to help me filter with sed & as we know with -i option we can do in-file edits directly. But I am unsure on # of old keys registered on machine for user alice.

grep alice authorized_keys | sed '1,2d'
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPbEzZDSZNKdGsH0sP10abmosX0yFquCEx2ibNzmh0AvvsR6pbA/pdl1i9XqIJlDBustWfmnChXEQz9Nft5LbM/moR6DAe7Wd70kmY8CreGgZ2kF8KCZ3bTkKYfO5PexKRCZ8IFRs6xswbfV0DJOmGnNo+wKOFDK/J7GVd9e1u6FebSo/Nr0QN/maXKN+Tc57GnV+j34ihD9PoRmfGYRIjm/vMkfUB9l1fZDEawdMO+Ats+G9gSvlhIm4TSP4sHk+zUJJff+j6ubMsGW0Flh4YjVJMMSU9vrpsTS5WiEbsZVxuWkrd+9jMo5RGDoWt2HADo4Dup0DlyM1pj2u1zbfl alice@brandNew

Is there a way to delete all old keys of alice except last one in-place holding this string match of alice ?

Answer 1

Remove a line with alice when you have another alice in the file.
Repeat until all alices except the last one are removed.

sed -rz ':a;s/(\n|^)[^\n]*alice[^\n]*\n(.*alice)/\1\2/;ta' authorized_keys

Answer 2

sed does not really have a good concept of "last of the matching lines"; but this is relatively easy with Awk.

awk '{ l[++n] = $0 }
  /alice/ { alice=NR }
  END { for(i=1; i<=n; i++) if (alice == i || !(l[i] ~ /alice/)) print l[i] }' file >file.new

This slurps the entire file into memory, so if the file is really big, maybe look for a different solution.

Awk out of the box does not have the capability to overwrite its input file, though GNU Awk has an -i inplace option which works roughly like sed -i

Answer 3

Consider an awk solution:

awk 'NR==FNR {if (/alice/) last=FNR; next} !/alice/ || FNR == last' file file

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDp6d0GmH5Q+lbx832n2W43Ikx8cpTeM8MRQ1yJmnhzaCSkQk9jWkwB5Msvj7uR923k0qLCZQ4ZbdPQ7liK9+Ks/7c0yDrBkU1KVOBNDQD1ck9Iwo2qwsS72b1kf4zeNKVJWkKBcFg7qxOmjIy9wNJ0XhbHmo/9gkp/hoOY3g0I5tExIlVdTH+uhEmOerghChTHTWU/kymZW0yYbDKl9zyPEDY9Hf/0Mt6g2dz+q3/cfcwCIZIeR5/Fjt7xXbt+fqmaiws1veGacw60u8ouMJ4DL9CcXRUbZHdEY4v2yIcEZRXMv9Q/6s0TcuC9s6Z5n2RzV8eW3RCZuzMWmYlkcA/H bob-publickey
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2UFK8Y7mODUZ7a2glErZbKk9hUy1pCOiGz4arAlTBiJ884vrjGaYNGFyWiNvqNKxLWn6Cul1eyPF1M/Z6l6QE3biJgJPln2ZZIoMcsNiCrFl0x5tUdnyVnVGz5pZTI1/0IFFLE4jbW2grMHOC7z+ik3WOD5aTj8pUWeTIeppMKLwjnJhm0ALsmLZqhtwnhgLSg9m4NQmcLWt5s0P/Z91hUKw2tY5iV9yOu5tS+oTMoeV7ays9MXM9ozpehLhsbqn98O7k2UpQzoWEo093lAjzydnBICd5tKdVJyGcyi3BLY4xhU+dnJ9NIUuYtnLZEhAGCbOQEGNivkxDQv1ccRTx trudy@newMacs
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPbEzZDSZNKdGsH0sP10abmosX0yFquCEx2ibNzmh0AvvsR6pbA/pdl1i9XqIJlDBustWfmnChXEQz9Nft5LbM/moR6DAe7Wd70kmY8CreGgZ2kF8KCZ3bTkKYfO5PexKRCZ8IFRs6xswbfV0DJOmGnNo+wKOFDK/J7GVd9e1u6FebSo/Nr0QN/maXKN+Tc57GnV+j34ihD9PoRmfGYRIjm/vMkfUB9l1fZDEawdMO+Ats+G9gSvlhIm4TSP4sHk+zUJJff+j6ubMsGW0Flh4YjVJMMSU9vrpsTS5WiEbsZVxuWkrd+9jMo5RGDoWt2HADo4Dup0DlyM1pj2u1zbfl alice@brandNew

To save change inline

For gnu-awk:

awk -i inplace 'NR==FNR {if (/alice/) last=FNR; next}
   !/alice/ || FNR == last' file file

For posix awk:

awk 'NR==FNR {if (/alice/) last=FNR; next}
   !/alice/ || FNR == last' file file > file.out && mv file.out file

Answer 4

This is more of a job for ed than it is sed, because ed has ways to find matches starting from the end of the file:

ed -s keys.txt <<'EOF'
1;?alice?-1 g/alice/d
w
EOF

will delete every line matching alice in the range from the first line to the one before the last occurrence of alice and then write the file back to disk.

Answer 5

This might work for you (GNU sed):

sed -n '/alice/=' file | sed '$d;s/$/d/' | sed -f - -i file

Find the line numbers of Alice keys.

Remove the last one and convert all others into sed delete command.

Apply the above commands to the original file.