Seven reasons to choose metric conjoint analysis over choice-based conjoint analysis

Conjoint analysis is usually divided into two broad types: traditional conjoint analysis (CA) and Choice Based Conjoint (CBC) - sometimes referred to as Discrete Choice Modelling (DCM).

Traditional conjoint analysis uses one of either rating (metric), ranking or pairwise forms of data collection, while CBC uses choices. The profiles can either be in a full or partial profile form.

In the 90s software vendors started promoting CBC as a better alternative. The theory was that CBC should provide substantial gains in accuracy, since choices are more natural. In reality it failed to deliver clear repeatable gains; while adding complexity and losses in functionality.

With the benefit of hindsight it clear that metric conjoint analysis is a better all-rounder in practice. It can be used on its own, or if larger attribute sets are required, hybridized by bridging partworths with partworths from a self-explicated exercise.

Metric conjoint analysis parameters can be estimated for a single individual. You scale the sample size to represent a population more accurately, not for the sake of parameter estimation. In contrast, CBC requires a much larger sample (usually 300+) just to be able to estimate parameters, since choices provide less information than ratings. You know which option is most preferred, but you have no data on the degree of preference of each option not chosen.

In fact CBC, and its various flavours, do not estimate individual level parameters directly. The simplest form of CBC only provides one set of parameters for the entire sample. To ‘estimate’ individual-level parameters, or at least come close to it, information is 'borrowed' from the global or latent class parameters to estimate individual parameters and then an assumption is made regarding the distribution form of these parameters across individual respondents. For example hierarchical bayes assumes a multivariate normal distribution across individuals. As to whether an individual really has those parameters, is not always certain since insufficient data is collected at the individual level.

Metric conjoint analysis isn't hampered by smaller screens. CBC conjoint is burdened with the requirement that at least two alternatives (usually more) are shown next to each other on a screen. Fitting an entire 'choice set' on screen – in a legible way - is often impossible. As many respondents now choose to complete surveys on smartphones, rather than desktops or laptops, this isn’t a trivial concern. While respondents could conceivably scroll horizontally to view all of the options, this is inconvenient and risks respondents not seeing all of the options.  Metric conjoint analysis, with its 'one at a time' (sequential monadic) approach doesn't have this problem. Each profile usually fits comfortably on screen, and even if scrolling is required, it is normally vertically; a process necessary to find the rating scale and button to 'continue' at the bottom, making it unlikely they will miss any aspect of the profile.

Metric conjoint analysis requires less effort on the respondent's part, since respondents only need to rate one profile at a time. In contrast, CBC conjoint requires respondents to read through multiple profiles in each choice set before making a decision. I have seen some run as large as eight profiles across – a large cognitive load, for very little yielded in terms of information gained (i.e. a single choice).

CBC conjoint (also referred to as discrete choice modelling) is a blanket term that conceals a bewildering array of options, and things can get complicated very quickly. Not only are there numerous models (such as Hierarchical Bayes and Latent Class) and software packages to choose from, there are numerous decisions you must make prior to launch and after the study completes. These include decisions regarding the sample design, survey mode, experimental design, parameter estimation, partial profiles, hybridization and so on. Each of these can take considerable design time, and I haven't even gotten to issues regarding the simulator setup, which is an entire topic on its own.

So CBC can become enormously involved for the practitioner. Worse, research users are going to have a harder time grasping the end result.

Let's take the parameters as an example. Metric part-worths are easy to explain as simple deviations from the average. In contrast, the parameters of CBC are expressed in terms of log odds, which even when exponentiated and expressed as odds ratios is still confusing.

So unless you can be certain that there will be very substantial gains in accuracy for your project, CBC and its various flavours don’t seem to be worth the trouble.

It's often been claimed that CBC is somehow more 'realistic'. While the choice sets of CBC might seem ideal in the world of FMCG, there are numerous industries where you are unlikely to have an array of competitors standing right in front of you at the moment of choice.

Many real world choice situations are more realistically measured in a sequential monadic way. Examples include online stores, universities, cars, banks, insurance, software and housing.

Irrespective, it seems that both classes of conjoint still work in practice, even when the data collection approach doesn’t mimic reality in this one aspect.

While certain software developers have claimed - since at least the 90s - that CBC should be more accurate than metric conjoint analysis and other traditional conjoint analysis methods, this hasn’t proven true.  It seems the claim relies on theory rather than empirical data. The chain of logic usually goes “choices are more realistic / natural and therefore the resulting model that is estimated using those choices as input must somehow be more accurate”. However, this ignores the fact that conjoint analysis has multiple stages – not just data collection – and the other stages had to be altered in ways that were - in many ways - a step backward.