I installed perl5 using perlbrew using the following command
perlbrew install -v perl-5.18.2 -Dusethreads -n --as perl-5.18.2_WITH_THREADS
After the installation, I find that this folder "~/perl5/perlbrew/build" takes 182MB. Is this folder necessary? If not could I remove that?
It is not necessary, and you can safely remove it.