How to remove all special characters in Linux text

Question

How to remove the special characters shown as blue color in the picture 1 like: ^M, ^A, ^@, ^[. In my understanding, ^M is a windows newline character, I can use sed -i '/^M//g' to remove it, but it doesn't work to remove others. The command dos2unix doesn't work, neither. Are there exist any ways that I can use to remove them both?

Answer 1

Remove everything except the printable characters (character class [:print:]), with sed:

sed $'s/[^[:print:]\t]//g' file.txt

[:print:] includes:

• [:alnum:] (alpha-numerics)
• [:punct:] (punctuations)
• space

The ANSI C quoting ($'') is used for interpreting \t as literal tab inside $'' (in bash and alike).

Answer 2

Try this inside vi or vim:

[in ESC mode] type: :%s/^M//g

or:

sed -e "s/^M//" filename > newfilename

Important: To enter ^M, type CTRL-V, then CTRL-M

Answer 3

To ensure that the command works with limited scope in Sed, force use of the "C" (POSIX) character classifications to avoid unpredictable behavior with non-ASCII characters:

LC_ALL=C sed 's/[^[:blank:][:print:]]//g' file.txt

Answer 4

Try running below command on linux command prompt

Option - 1: (If dos2unix command is installed on Linux machine)

dos2unix sample_file.txt

Option - 2:

cat sample_file.txt | tr -d '\015' > new_sample_file.txt