How can I get the participants of a Vanity experiment?

Question

tl:dr

Is there anyway to get something like Vanity.experiment(:landing).participants_for_option(:a) returning a array of users?

The long story

I'm using the gem Vanity with a Rails 4.2 application and it is working nicely, but I want to inspect further the behaviour of participants.

I tested what kind of page converted more users: A classical signup page versus a signup with order page. The classical signup page led to almost three times more signups, but I'm still in the dark in the sense that I don't know, among the signup-only-users, how many ordered a product.

Answer 1

It sounds like you're trying to understand more about how an experiment affects different parts of your funnel.

At the aggregate level, one way to do that may be to to use multiple metrics for your experiment at different parts of your funnel, e.g. track!ing both signups and then purchases.

Unfortunately, Vanity isn't set up very well to query for individual participants per alternative, because testing itself is aggregate. If you want to access alternatives per user, there are methods for that, for example, Vanity.playground.adapter.ab_showing(experiment, identity), see the docs.

If you're interested in doing more in depth analytical queries, it might be worth using the SQL adapter, the schema tracks per participant and you could join to other tables that hold data about purchases/etc.

Edit:

It looks like this has changed in the most recent version of Vanity: https://github.com/alobato/vanity/blob/master/lib/vanity/playground.rb#L231 Vanity.playground.connection.ab_assigned(experiment_name, identity) Vanity.playground.connection.ab_showing(experiment_name, identity)