Getting Bad Request Error while using Laravel Socialite for Fitbit on page Refresh

Question

I have integrated fitbit for storing device info of a user with Laravel 5.3 using Socialite 2.0 and everything worked fine for me until I got this error

Client error: `POST https://api.fitbit.com/oauth2/token` resulted in 
a `400 Bad Request` response:
{"errors":[{"errorType":"invalid_grant","message":"Authorization code 
invalid: e05d30bf81ff78c29da7f4553d929709fe4ecfea (truncated...)

I have done this to accomplish this far

class fitbit extends Controller {
    public function index() {
        return view( 'fitbit.index' );
    }

    public function redirectToProvider( Request $request ) {

        return Socialite::driver( 'fitbit' )->redirect();
    }

    public function handleProviderCallback() {


        $fitbitUser = Socialite::driver( 'fitbit' )->stateless()->user();
        if ( $fitbitUser->token ) {
            $patient_id = Auth::user()->Patient->PatientID;
        }
        $fitbit                   = new FitbitMaster();
        $fitbit->patient_id       = $patient_id;
        $fitbit->fitbit_client_id = $fitbitUser->id;
        $fitbit->save();
        flash( 'Connected!', 'success' );

        return view( 'fitbit.index' );
    }

}

Now what I want is I want if the user is connected then I should display the message "Connected" in view.

I tried to check other solutions on SO but nothing worked for me. How should I pass header information? I'm quite confident that it is happening because of I am not able to set refresh_token.