Merge all Yara rules from a Yara github repository in one .yar file

Question

There is an index file in the official Yara rules repository git hub. This one : https://github.com/Yara-Rules/rules/blob/master/index.yar

I want to create a script in bash or other language able to merge all yara files in a .yar (like the index.yar) that include all my yara files. I can do it manually but the problem is that there are too many rules to write it down manually in the script.

For instance this repository : https://github.com/citizenlab/malware-signatures/tree/master/malware-families I don't really know how to create a file that create a yara file that include all these yara rules from the repository

Thanks in advance for all your help, I appreciate.

Answer 1

I am not sure of your question but why not just make a list of the raw yar files and download them in a loop ? In my example below I pull down 3 files and print them but I strip out all but the file name for the lines with include. I would then of course run this script into another file using '>' :

Example :

#!/bin/bash

raw_list_urls="https://raw.githubusercontent.com/Yara-Rules/rules/master/index.yar https://raw.githubusercontent.com/Yara-Rules/rules/master/index.yar https://raw.githubusercontent.com/Yara-Rules/rules/master/index.yar"
IFS=' ' read -r -a urls <<< "${raw_list_urls}"

for url in "${urls[@]}" ; do
        echo "Getting ${url}" ; sleep 2
        curl "${url}" | grep include | cut -d'"' -f2
done

Output example:

./malware/RAT_ShadowTech.yar
./malware/RAT_Shim.yar
./malware/RAT_Terminator.yar
./malware/RAT_Xtreme.yar
./malware/RAT_ZoxPNG.yar
./malware/RAT_jRAT.yar
./malware/RAT_xRAT.yar
./malware/RAT_xRAT20.yar
./malware/TOOLKIT_Chinese_Hacktools.yar
./malware/TOOLKIT_Dubrute.yar
./malware/TOOLKIT_FinFisher_.yar
./malware/TOOLKIT_Gen_powerkatz.yar
./malware/TOOLKIT_Mandibule.yar

Answer 2

I wrote a small script that looks for *.yar files in local folders and creates a index file with the para rules found:

for f in $(find . -mindepth 1 ! -path "./$(basename $PWD)_all.yar" -type f -name "*.yar"); do
  if [[ "$f" == *"Mobile_Malware"* || "$f" == *"mobile"* ]]; then  # exclude Android rules
    continue
  fi
  echo "include \"$f\"" >> $(basename $PWD)_index.yar;
done;