Patient and physician questionnaires, visual analogue scales, rating scales for the assessment of efficacy, effectiveness, safety, quality of life (QoL), resource use, and treatment costs are getting more and more important in clinical research. In particular, the recently increasing emphasis on QoL, treatment costs, real-life effectiveness of drugs, and the need for patient-focused outcomes to achieve market access leads to an increase in the development and use of scales. Selection, use, analysis, and reporting of questionnaires are tasks that need to be integrated throughout the whole clinical development process and this article explains the most important steps necessary for optimal leverage of questionnaire data.

Questionnaires to evaluate efficacy and QoL have been used for decades in clinical research in those areas where no objective measurement of efficacy is possible, such as psychiatry, and where QoL has been found to be of major importance for acceptance of a treatment, such as oncology. People who have been working in these areas of clinical research are probably well aware of the caveats and potential pitfalls that go together with analyses of questionnaire results. However, recently, several developments in clinical research have led to an even more widespread use of questionnaires, and most probably this development will continue in the near future. The reasons for this are the increasing importance of data that are closer to clinical practice and which are needed to achieve market access. As an example, the FDA and EMA may require comparative effectiveness data from post-marketing settings in the context of a risk management plan [1]. Quality of life, patient wellbeing, and patient-centred outcomes are requested by reimbursement agencies such as NICE in the UK and IQWIG in Germany. In this area, new concepts such as quality-adjusted life years (QUALY) or the standard gamble (or time trade-off) have emerged to measure economical effectiveness, important for reimbursement decisions, and new concepts are still being developed [2,3]. Most of this information will need to be gathered in a non-interventional setting or in clinical studies that need to be as simple as possible, again excluding the use of laboratory or other “hard” data.

The implementation, analysis, and interpretation of scales in general is not as straightforward as, e.g., measuring blood pressure, and this is even more true if scales are to be introduced into fields where this hasn’t been standard previously and no generally accepted gold standards for questionnaire use are available. With complex questionnaires – often comprising dozens of questions – a huge amount of data is usually generated, from which the important information first needs to be extracted. Errors in the application of questionnaires can lead to the generation of data where only towards the end of the process it is realised that these data cannot be interpreted in a way that is required to provide convincing evidence. This will lead to time-consuming and costly re-analyses or, in the worst case, to data graveyards that are unusable. It is therefore important to plan the appropriate selection, use, and analysis of questionnaires early during a clinical trial or even better, when planning a whole clinical programme.

The first step is to identify the need for certain types of questionnaires and the selection of the best-suited ones. There are several ways in which questionnaires can be classified (see Figure 1). Usually therapeutic questionnaires, i.e. those that are sensitive to change over time, are used in clinical research, where changes from baseline to endpoint are of interest. Sometimes however, at least at baseline, diagnostic questionnaires may be needed to confirm a certain disease as inclusion criterion or to assess the disease stage (e.g. tumour staging in oncology). Furthermore the subject filling in the questionnaire is important, and different aspects of achieving registration and market access for a drug patient, physician, parent, and caregiver perspective might be of importance. Also, the type of the questionnaire regarding data collection is important as it will impact the data that will need to be processed by data management, statistics and scientific communications. Visual analogue scales (VAS) give one single number anywhere between 0 and 100, whereas Likert (ordinal) scales have pre-defined values that are usually coded as digits (e.g. 1, 2, 3, 4, 5). Full questionnaires consist of an array of questions and the answers are coded in various ways into total scores or subscores (or domains), or may be analysed as single items.

Other criteria for the suitability of a questionnaire are generic (i.e. you can compare results between diseases) vs. specific (i.e. you can address your indication more specifically), short (easy) vs. comprehensive (difficult), suitability for the target population (e.g. children, elderly), possibilities for data entry (e.g. paper, computer, smartphone, datapen). This diversity of available questionnaires can lead to the desire to use a large number of questionnaires to encompass all possible aspects in a product development or even in the pivotal study. However, the time and effort needed by patients and physicians to complete the questionnaires need to be taken into account, as the time needed to complete all questionnaires of a single study visit then quickly reaches a timeframe of several hours. This will compromise the quality and completeness of answers and will even make recruitment of centres and patients more difficult. Thus it should always be challenged if an additional questionnaire is needed in a clinical programme or a specific clinical study.

A comprehensive overview of the concept of rating scales and questionnaires is given by the book “Measuring Health” by Ian McDowell [4]. In addition, it provides an overview of standard questionnaires available and their advantages and disadvantages in a number of areas such as social health, psychological wellbeing, anxiety, depression, mental status testing (e.g. dementia), pain measurement, general health status and QoL.

At the beginning of a clinical development programme involving questionnaires, it is crucial to define which questionnaires should be used throughout the programme. This decision should be made at an early stage (at the latest before the first Phase II is initiated) and decisions should already be made on the use of questionnaires needed for later stages of development, as earlier studies could already be suitable for gathering these data, saving time and effort later on. Consistency of questionnaire use throughout a development programme is an absolute requirement to maintain comparability between studies. Furthermore, consistency between different development programmes in the same indication (e.g. depression) or drug class is also very desirable, and should be included in the decision-making process. Sometimes it is better to include an established questionnaire that has already been used and is accepted, instead of using a new one that claims to be better but is not comparable to data already published. Furthermore, in some indications the regulators mandate the use of specific questionnaires for granting approval (e.g. HAM-D or MADRS as primary objectives in depression studies). Therefore a close communication with, e.g., FDA and EMA is often helpful in identifying the relevant questionnaires.

If, in addition to established questionnaires, newer or less frequently used questionnaires should be employed, it is important to check their quality/credibility. In particular, the questionnaire should be validated, and validated translations for all languages used should be available [5]. A newly-devised “tailor-made” questionnaire should never be used for studies relevant for registration or reimbursement decisions, unless the necessity for devising a new questionnaire was discussed and agreed upon with the respective agencies.

At the end of this initial decision-making, there should be an overall plan available which defines mandatory questionnaires that should be used in every clinical trial of the programme, and optional questionnaires associated to a list of topics (e.g. non- interventional, quality of life, treatment cost, parent perspective) that can be used if the respective topics are to be evaluated in a certain part of the clinical programme or a specific clinical study.

Another important pitfall during the execution of a clinical programme is inconsistency in the analysis approach to a questionnaire. For a number of questionnaires, specific approaches to analysis exist and are presented in scientific manuscripts, instructional books, or websites (e.g. www.euroqol.org for the EQ- 5D). These documents define the way scales are to be analysed, how total and subscores are calculated, and how missing data is to be handled during the analysis. Also in a number of cases different approaches for the analysis are possible, but it needs to be assured if the approach selected is suited for the required outcome.

This is often further complicated by the fact that there is more than one version of a questionnaire (e.g. the HAM-D 17, 19, and 24), and it has to be specified from the beginning which version of the questionnaire will be used and which analysis method is to be applied. This decision needs to be communicated to all functions that are involved in performing the clinical trials. This includes data management for the setup of the CRF, clinical research associates for training the investigators and monitoring, statisticians who must state the standard analysis method to be used, physicians who need to make sure that the analyses are interpreted correctly, and medical writers who are informed about the meaning of the specific analyses. During the course of a clinical programme, usually spanning several years, this has to be refreshed and communicated to new team members to avoid unnecessary work and wrong analysis or interpretation of results.

In addition, most analyses of questionnaires require some extensive data handling after extracting the raw data from the CRF, as missing data need to be addressed, factor analyses and statistical models are often performed (e.g. regression analyses or ANCOVA), to be able to interpret the data in a meaningful way [6]. This means that, ideally, project statisticians and statistical programmers, as well as team members from other functions, should have experience in analysing, selecting and interpreting complex questionnaire data.

Even after a correct and focused analysis of questionnaires, usually a large amount of data is generated, presenting results for, e.g., a total score, subscores, and down to the level of single questions by subgroup and sometimes different methodologies. Maybe the most important step in the communication of results of questionnaires is to be able to select the important parts of the data. This includes, of course, a first step focusing on those analyses that were pre-specified as part of the primary and secondary objectives of a study. However, in a second step it is also important to focus on the target audience for a specific delivery. Whereas for a CSR or a CTD document to be sent to a regulatory agency, the standard primary objective analysis is usually the most important one, a reimbursement dossier or a clinical or health outcomes-related manuscript might require additional analyses from the questionnaires that answer the specific questions of the respective audience.

The crucial step in the successful communication of questionnaire results is, therefore, a detailed publication plan that integrates the results of all questionnaires gathered in the clinical programme with the requirements of the diverse audiences. This plan should contain a grid, linking the questions to be answered with the studies providing the respective questionnaire results, and should allow the communication of these results in a coordinated manner in time from initial communications with FDA or EMA, over publications of the pivotal results, up to targeted publications dealing with specific topics including targeting locally specific issues. In addition, the target audiences and stakeholders should be identified along with a time schedule that is in accordance with the overall clinical development plan, striving to provide information when it is needed most. A consistent approach in developing all documents related to questionnaires – from initial planning documents and protocol development, up to final publications in congresses, scientific journals, and also including marketing material that can be used to disseminate the findings of questionnaires to daily clinical practice – is mandatory for optimised drug development.

In summary, the use of questionnaires requires a coordinated approach between many functions of clinical research that needs to be maintained across the whole timeframe of a clinical programme, from late Phase I to post-marketing activities. Questionnaires should always be selected, used and communicated, with communication to the target audience in mind – and this includes the use of team members with experience in the field and a dedication to the final goals of the development programmes.

REFERENCES

1. Committee on Comparative Effectiveness Research Prioritization, Institute of Medicine. Initial national priorities for comparative effectiveness research. The National Academies Press, 2009.
2. Robinson R. Cost-utility analysis. Br Med J. 1993;307:859-862.
3. Torrance GW, Feeny D. Utilities and quality adjusted life years. Int J Technol Assess Health Care. 1989;5:559-575.
4. McDowell I. Measuring health: A guide to rating scales and questionnaires. Oxford University Press. 2006(3rd ed.).
5. DeVellis RF. Scale development. Theory and applications. Applied Social Research Methods Series 26 (Eds: Bickman L, Rog DJ), Sage Publications, 2012 (3rd ed.).
6. Svensson E. Guidelines to statistical evaluation of data from rating scales and questionnaires. J Rehab Med. 2001; 33:47- 48.

Thomas Wagner
Published in: Journal for Clinical Studies, Vol. 6, No. 1, 2014