Flash attention argument throwing error while finetuning falcon_7b_instruct

Question

I am exploring the flash attention in my code to fine-tune the falcon-7b-instruct model as it is explained on the huggingface.

I am getting an error:

TypeError: FalconForCausalLM.__init__() got an unexpected keyword argument 'use_flash_attention_2'

As mentioned on hugging face:

1. I am using data type float16
2. Experimenting on falcon model since it can work only on falcon or llama models.

Please let me know if anyone else has also used it, I might be missing something

Answer 1

Upgrade transformers to transformers 4.35.0 using

pip install transformers==4.35.0