Using Palatte API in ListView Adapter

Question

Ok, first off all, I have not yet converted to using RecylerView; if for whatever reason doing that answers this question, by all means let me know.

In the meantime, I am trying to add a splash of different colors into a ListView using Palette.

Picture this: In Twitter, you have a timeline, each row has a user profile image. I am doing the equivalent of making the background of each row based on the user profile image.

I am also using Picasso to load the images.

In my Adapter, I do this:

try {
    Picasso.with(context).load(url).placeholder(R.drawable.usericon).transform(new CircularTransform())
            .into(holder.i1);

    holder.i1.buildDrawingCache();
    Bitmap b = holder.i1.getDrawingCache();

    final ViewHolder finalHolder = holder;
    Palette.generateAsync(b, new Palette.PaletteAsyncListener() {
        @Override
        public void onGenerated(Palette palette) {

            Palette.Swatch vibrant = palette.getVibrantSwatch();
            if (vibrant != null) {
                finalHolder.r1.setBackgroundColor(
                        vibrant.getRgb());
                finalHolder.tv3.setTextColor(
                        vibrant.getTitleTextColor());
                finalHolder.l1.setBackgroundColor(
                        vibrant.getRgb());
                finalHolder.tv8.setTextColor(
                        vibrant.getTitleTextColor());
            }

        }
    });


} catch (Exception e) {

}

this does work; but the problem is when the list is first loaded, the rows are the default colors. I can sit there without scrolling for 5 seconds or 5 minutes and it will not change until I start scrolling. Then the rows below ARE the new color based on the Swatch. If I go back up to the top, those rows are now colored as well.

So my question: Is what I am doing too ambitious for this api? if not, is there a better way to speed things up?