Brief Description

This questionnaire is designed to systematically record and evaluate the methods used to assess the validity and reliability of research instruments such as scales, tests, and questionnaires. It focuses on analyzing the procedures that ensure the quality and accuracy of measurements in research and clinical settings.

Purpose

The aim of the tool is to provide a structured method for recording and evaluating the techniques applied to confirm the validity (content, construct, criterion) and reliability (internal consistency, stability, inter-rater agreement) of research measurements. It can be used for the critical appraisal of existing tools or for the development of new ones.

Scoring Method

Participants or evaluators respond to each item using a five-point Likert scale (1 = not at all, 5 = to a very great extent) to assess the degree of implementation or completeness of each method. Certain items may be dichotomous (Yes/No) for a straightforward record of whether specific techniques have been used.

Validity

The validity of the questionnaire is ensured through a literature review and evaluation by a panel of experts with experience in psychometrics and research methodology. Construct validity can be verified through factor analysis, while criterion validity can be examined by comparing the results with established evaluation standards.

Reliability

Internal consistency reliability is calculated using Cronbach’s α, with a target value of ≥0.80. Test–retest reliability can be assessed in a sample of users over a period of 1–2 weeks, while inter-rater reliability is calculated using agreement coefficients (e.g., Cohen’s κ).

Data Analysis and Use

Analysis includes calculating means and standard deviations for each category of methods, as well as comparing different tools or evaluator groups. Results can be used to revise instruments, improve their psychometric quality, and document their suitability for research or clinical use.

References

Anastasi, A., & Urbina, S. (1997). Psychological Testing (7th ed.). Upper Saddle River, NJ: Prentice Hall.
DeVellis, R. F. (2017). Scale Development: Theory and Applications (4th ed.). Thousand Oaks, CA: Sage.
Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric Theory (3rd ed.). New York: McGraw-Hill.