There are a number of excellent reviews on the DCE methodology 455152 and its application in health research 535455. Therefore, this section only provides a brief introduction to the basic principles for readers who are not familiar with this research method. Choice experiments are a quantitative methodology for evaluating the relative importance of the different product attributes that influence consumer choice behaviour 45. This technique has its origins in the economic theory of demand, and especially in the work of Lancaster, who proposed that the demand for goods was effectively demand for their specific combination of product characteristics 56.

In choice experiments, respondents are asked to make choices between hypothetical alternative goods or services. Each good or service (or job description in HR applications) is described by several characteristics. Study objectives and preliminary work are usually key for the researcher to identify this limited set of characteristics that will be included 5758. In this paper we refer to these characteristics as attributes, while the combination of different attributes is termed a scenario. The responses given over a number of carefully selected scenarios enable the researcher to infer the relative importance of the different attributes.

According to random utility theory, individuals are assumed to choose the alternative that provides the highest individual benefit or utility. In the case of a binary choice between two different jobs, one would have:

where Y_k is a choice variable that equals 1 when job k is chosen, and where U_ik, the utility of individual i for job k, is assumed to be a function of the n attributes of the job:

Since evaluating all possible combinations of a set of product attributes would require an extremely large number of questions, a limited number is chosen, using experimental design techniques that ensure the selection of scenarios that optimize the information obtained from respondents. The selected scenarios are then organized into a series of choice sets. Each study participant then evaluates a number of choice sets and is asked to choose the preferred scenario (more details on DCE design can be found in other studies specifically addressing these methodological aspects – 55).

DCE studies then use regression techniques to model respondents' choices as a function of the scenario attributes. Common modelling techniques used include random-effects logit or probit model and conditional logit models 5354, while there is an increasing use of more sophisticated tools such as mixed logit models 59. The significance and magnitude of the regression coefficients indicate the relative importance of those attributes that statistically influence respondents' choices. The negative of the ratio of any two coefficients represents the trade-offs made between the two attributes, and is called the marginal rate of substitution. When cost (or salary) coefficient is used as denominator, the ratio of coefficient provides an estimate of the willingness to pay for a particular characteristic. Performing regressions for different subgroups of health workers can be used to reveal differences in their preferences.

Papers were primarily identified through a search of the following databases: Popline, PubMed, Econlit, HEED (Health Economics Evaluation Database), Emerald and Business Source Premier. These databases were used to cover most of the relevant literature from health and economics. Combinations of the following terms were used to search: "choice experiment", "discrete choice", "stated preferences", "human resources", "health personnel", "staff", "doctor", "nurse". A complementary search was made using Google Scholar.

In addition to the database searches, a snowballing approach was used to identify potential articles from the reference lists of relevant articles already identified. Some experts in the field of DCE and authors who had already published HR DCE studies were also contacted. The scope of the review included both developed and developing country research. However, only studies on health workers responsible for direct patient care were included in the review – so we focused on nurses, doctors and clinical officers but excluded a study on pharmacists 60 because we assumed that the nature of their work and their job preferences would be somewhat different.

The identified articles were compared in relation to study design, attributes included and key results. Although DCE results quantify the importance of attributes relative to one another, comparing the coefficients or marginal rates of substitution from one study to another is not valid (because an attribute's coefficient estimates are directly driven by all other included attributes). Therefore this overview presents a narrative synthesis and comparison of the findings of the identified studies.

Nine studies were found that used DCE techniques to investigate HR problems. Four of them 44616263 were from the developed world (all in the United Kingdom) and the rest were carried out in developing countries: namely Indonesia 42, South Africa 41, Malawi 43, Ethiopia 64 and Tanzania 65. The majority of study participants were doctors, while two studies focused on nurses, one reported results on both nurses and doctors 64 and one on clinical officers 65 (see Additional file 1).

Not all the studies clearly specified in their objectives that the design was meant to address issues related to health workforce maldistribution, but all did at least underline the direct relevance of the findings for policies tackling these issues.

In terms of DCE design, all studies employed an unlabelled experiment 51 with two choices: study participants had to choose between a generic job A and job B. They all used a fractional factorial (most of them creating 16 choice sets), and the predominant method to construct choice sets was to use a constant job scenario against which all other choice sets were compared.

The choice of attributes and levels was usually based on preliminary qualitative work and some literature review (Additional file 1). In most studies, pilot-testing the questionnaire enabled the researchers to refine the attribute levels and their wording. Despite the variety of attributes chosen, a few characteristics were common to all study contexts (Additional files 2 and 3). A salary variable was always present – not only because it is likely to be an important determinant of job preferences but also because it makes it possible to compute monetary equivalents for all other attributes in the DCE.

Beyond salaries, the range of attributes included in each study reflects elements identified in each context as determinants of health workers' motivation and choices. For example, in the United Kingdom, workload appeared as particularly important, and was included in different forms (hours worked per week, amount of after-hours work, patient list size, staffing levels). In developing countries, location appeared as a crucial job characteristic and was a DCE attribute in all but one study. Other recurring attributes were the provision of housing, the opportunity to benefit from further education, and improving the level of drugs and equipment in the facilities.

Finally, in all studies, the authors made use of one of the key strengths of DCE methods – the possibility of including job characteristics that are not currently present on the labour market, which could help identify policy levers to alter health workers' choices. In particular the range of salaries was often widened, which allowed to test the potential effects of an increase in remuneration. In some studies, certain job characteristics were completely different from existing possibilities in the labour market. For example in Indonesia, the authors delinked the opportunity to specialize from the obligation to do so in the public sector.

Certain methodological specificities limit the direct comparison and synthesis of the results obtained in the studies reviewed here. Indeed, study findings are dependent on the attributes included and influenced by the levels chosen in the experiment. In particular, some authors have argued that a distortion in the level range, for the salary variable in particular, could have important consequences for the results 6667. Furthermore, the comparison of results is also limited by variation in the choice of econometric models and differences in model specifications. For example, some studies have modelled the salary as a continuous variable, while others have treated different levels categorically though using dummy variables. This obviously modifies the interpretation of regression coefficients and the relative ranking of attributes. Despite these caveats, several findings emerge from this literature review.

Overall, the results from existing DCE studies on health workers' preferences show the relative importance of both pecuniary and non-pecuniary interventions (see Additional file 4). Non-financial strategies have the potential to make attractive incentives, and were often found to be more powerful than financial ones. This important finding is confirmed by a wide literature reviewing the effects of financial versus non-financial interventions 2068.

DCE methods both build upon and complement other types of studies traditionally used in the HRH literature (see the brief overview in the introduction). On the one hand, DCEs rely partly on the findings from qualitative studies on job satisfaction to adequately define the range of attributes that will be relevant in the choice experiment. On the other hand, unlike studies based on ranking or rating methods, DCE studies force respondents to make trade-offs, thereby revealing and quantifying their underlying hierarchy of preferences. Although increased salaries always come up as a key determinant of job satisfaction, studies based on traditional questionnaires have failed to provide evidence of the relative importance of salary compared to other job characteristics. Using marginal rates of substitution, it is possible to compare the relative valuation of job characteristics in a DCE, as showed by the two examples reported in Table 1.

Finally, it should be noted that the studies reviewed here might be compromised by some methodological limitations. Some have criticized the lack of rigour in the experimental designs used by studies in the health economics literature 69. Based on the information available in the articles, the studies summarized here are likely to have suffered from some flaws due to inappropriately constructed designs. For example, the use of a constant comparator in all but one study 65 suggests that they are unlikely to have used optimal designs 52.

Because DCE studies quantify the relative importance of determinants they provide more policy relevant information. All the studies reviewed here identified potential policy implications of their findings in each country context. Several aspects relating to quality of life (fewer hours per week and less after-hours work in developed countries; the provision of better housing and improved work conditions in developing countries) are positively valued by health workers in most countries, and can be therefore used as policy levers. Intellectual satisfaction in the United Kingdom, and education opportunities in developing countries also appear as important job characteristics that would increase the satisfaction of health workers.

In the more recent studies, most authors provided a list of possible interventions to influence health workers' job choices, particularly in addressing staff maldistribution and encouraging them to take up positions in under-served areas. By means of basic modelling techniques, the authors also computed the expected effects of such policies. For example, the study in Ethiopia 64 shows that the provision of a superior housing would increase the proportion of doctors willing to take up positions in rural areas by more than 250% (from 7.5% to 26.9%), while a reduction in time commitment to the public service would have a non-significant effect. Combined with an assessment of the costs of each package, such scenarios can provide essential estimation tools to approximate the relative cost-effectiveness of different HR incentives.

The results reviewed here also emphasize the importance of better understanding the preferences of different subgroups of health workers. This aspect is particularly crucial for health authorities to plan human resource interventions. Indeed, understanding the various aspirations and possible behaviours of subgroups of health workers might enable policy-makers to craft better policies to recruit and deploy health professionals where they are needed. Some studies reviewed here provide insights into such issues. For example, the recent study on clinical officers in Tanzania 65 shows that women are less responsive than men to pecuniary incentives, and tend to be more concerned with other working conditions (infrastructure, sufficient equipment). By providing information on how to target specific groups, and what would be the differential effects of policies for different subpopulations, DCEs are better suited than traditional forecasting tools used by policy-makers to predict health personnel needs in various geographical or professional areas.

All the DCE studies we identified used unlabelled study designs, which assume that people value attributes equally across all job situations. For example, the studies from developing countries imply that health workers value a housing opportunity in urban and rural areas in the same way, or that they value the opportunity to work in the private sector equally in rural or urban areas. These are strong assumptions that could be investigated by the use of labelled or alternative-specific designs, which have often been used in transport or environmental economics. Such studies could also allow more flexibility and realism in the definition of the scenarios proposed, and be even more policy-relevant 70. For example, the relative importance of job preferences across sectors (private versus public) could be explored with such designs. This question could be particularly relevant for countries where internal migration from the public to the private sector is a critical issue.

Stated preference methods have been critiqued because they may not predict real behaviours and choices. There is a growing literature in other fields trying to evaluate the correspondence between stated and revealed preferences 4551. In the field of HRH, the format of choice experiments, in asking respondents to choose their preferred job from two or more job descriptions, closely resembles the real decisions faced by individuals in their everyday life. As a result, health care markets are one of the areas where the comparison between revealed and stated preferences could be more common in the future, providing a richer set of data where choices made and options not selected would inform individual decision processes better.

Another field of investigation concerns the external validity of DCEs, which raises fundamental questions when interpreting the results of some of these studies. A complex issue is the extent to which the context and the individual experience have an impact on an individual's responses. As most DCE questionnaires present only very brief descriptions of attributes, there is some variation in how the attributes and levels will be interpreted by respondents. For example, Scott 44 mentions that the use of guidelines was valued positively by GPs, although researchers had expected this to be seen as a restriction to their autonomy. Clearly, respondents interpreted the attribute differently from what the researchers intended.

Similarly, several of the studies reviewed here from developing countries used rather imprecise terminology for job location, such as "city" or "rural", but it is not clear which characteristics the respondents associated with those labels. Finally, when using job attributes currently not available on the job markets (such as potential policy interventions), it is difficult to assess the extent to which respondents will be able to easily appreciate or believe those possibilities. These are examples of areas where other research methods, such as qualitative tools, could be helpful in understanding the results of the DCEs better, and take the debate on the limitations and interpretation of DCEs forward.