Field-testing items prior to their use as scored items on the exam.ĭuring the examination process, fairness is addressed through standardized procedures regarding the registration process, accessibility issues, roles of proctors, and security of test materials and equipment.Selecting subject matter experts who are OTR and COTA practitioners and educators from diverse geographical areas, practice experiences, and cultures.Referencing items to approved and published resources in occupational therapy.Coding items to the approved examination content outline.Editing items for issues of bias and stereotyping.Additional fairness criteria include, but are not limited to: NBCOT adheres to recognized item writing, test development, and review procedures to ensure readability, neutral language, and the universal accuracy of terms used in its items. NBCOT takes into account the fairness of its examinations during item development.
This review is designed to: validate that the knowledge and tasks measured are compatible with the domain-level content specifications assess the relative importance and frequency of each item to occupational therapy practice and confirm that each item meets generally accepted fairness guidelines. After questions or “items” for NBCOT examinations are developed, the items then undergo a rigorous review process by an additional committee of SMEs. SMEs are OTR and COTA certificants who represent the profession in terms of practice experiences, geographic regions, gender, and ethnicity. Specifically, NCCA standards (2014) indicate SMEs must represent the appropriate demographics of the population to be certified and provide insight and guidance into examination processes. NBCOT annually recruits OTR and COTA subject matter experts (SMEs) to develop new items for the certification examinations in accordance to the accreditation standards relating to the use of qualified personnel for examination development. The content outline is used to determine the percentage of items, or weighting, to be allocated to each of the domain areas of the examinations, as required for accreditation by the National Commission for Certifying Agencies (NCCA, 2014) and the American National Standards Institute (ISO, 2012). Results of the practice analysis studies led to the formation of the certification examination content outline (blueprint) relative to the validated domain, task, and knowledge statements for each respective credential (NBCOT, 2018b NBCOT, 2018d). Additionally, respondents are provided an opportunity to identify critical elements of occupational therapy practice that are not captured within the survey. For each major domain area, respondents are asked to identify the percentage of the exam that should be devoted to that performance domain. For each task and knowledge statement on the survey, respondents are asked to rate the frequency and importance of the tasks and the importance of knowledge required for occupational therapy practice. The studies involve administering a large-scale survey to gather practice data from thousands of entry-level OTR and COTA certificants. A separate practice analysis study is completed for each credential.
NBCOT conducted a revalidation of the OTR and COTA practice analysis studies in 2016-2017 (NBCOT, 2018a NBCOT, 2018c).