Scholarship of Teaching & Learning Workshop
"SMART" Learning Objectives:
By the end of this section, you will be able to:
1) Explain the relevance of reliability and validity in educational research design.
2) Compare and contrast the basic principles of qualitative and quantitative research design.
3) Outline a basic research plan for a qualitative or quantitative study using the principles described in this workshop.
1) Explain the relevance of reliability and validity in educational research design.
2) Compare and contrast the basic principles of qualitative and quantitative research design.
3) Outline a basic research plan for a qualitative or quantitative study using the principles described in this workshop.
Introduction & Educational Technology:
In this workshop we will discuss basic principles of educational research, describe key terminology important in the scholarship of teaching and learning, and evaluate examples of qualitative and quantitative research design. The technology we will focus on in this workshop involves voice-recorded and text-based versions of a digital pathology module created with Microsoft PowerPoint. The module will consist of patient case scenarios with images, self-assessment questions with feedback, and a brief tutorial highlighting relevant diagnostic features. The purpose of these modules is to allow second year medical students to independently learn about basics of skin diseases so class time can be used to focus on application type activities.
Virtual slides and web-based learning modules are often used in pathology education because they are more convenient and standardized than glass microscopic slides. Studies reported in the medical education literature have shown the positive influence of digital pathology modules on student engagement (Kumar et al., 2004). However, research on the impact of audio recordings in image-based digital pathology modules is limited, especially in the context of pre-clinical medical student education. Identifying the best multimedia design principles to diminish extraneous cognitive load in this context should help improve student knowledge retention and transfer (Mayer & Fiorella, 2014).
Description of Major Components of Measurement:
Reliability and validity are important factors to consider when designing a research study to ensure that the results obtained are accurate and reproducible. Reliability refers to the ability of a test to obtain the same results when repeated. It addresses the amount of error in a measurement (Salkind, 2012). Validity describes how well a test measures what it is designed to measure, and is necessary to ensure the accuracy of results (Salkind, 2012).
Of the various types of reliability discussed by Salkind (2012), test-retest and inter-rater reliability have the most relevance to my clinical practice in pathology. Since lab tests often guide further management decisions, it is important to ensure that results are reliable. Test-retest reliability refers to implementing the same test at a different time to see if the results obtained are similar (Salkind, 2012). We use this procedure in the clinical pathology lab when unusual test results, such as a very high electrolyte level, are retested to decrease the possibility of testing error. Inter-rater reliability measures the degree of consistency between two different raters who observe the same process (Salkind, 2012). This is also used in my field when it comes to quality control of specimens. We like to ensure that 2 different pathologists reviewing the same specimen reach similar conclusions, to reduce the possibility of diagnostic error.
Of the types of validity described by Salkind (2012), the two which are most relevant to my field of pathology education are content and construct validity, because they help ensure that any new concepts discussed with our students are accurate. Content validity refers to measuring the accuracy of the content presented, and is typically achieved by asking an expert to review the material and provide feedback (Salkind, 2012). Construct validity refers to the ability of a test to measure an underlying construct, and is usually determined by statistical analysis (Salkind, 2012). In my field as a medical educator, it is important to ensure content validity of any new materials presented to the students in class. This is typically done by having a content expert review materials before the start of class to ensure accuracy of materials. We also critically evaluate statistical evidence for construct validity before incorporating new study data in our teaching.
Sample Qualitative Research Study:
Below please find a recorded Prezi presentation describing a qualitative research study using focus group interviews to evaluate student perceptions of learning using a narrated digital pathology module compared to a text-based version of the same module. Since I am currently developing these modules for medical education, determining student perceptions of the different versions will help me design modules that are most effective for student learning and engagement, while minimizing extraneous cognitive processing (Mayer & Fiorella, 2014).
Sample Quantitative Research Study
Below please find a recorded Prezi presentation describing a quantitative research study using the pre-test, post-test, and control group experimental design model. The purpose of this study is to evaluate the impact of a narrated online pathology module on medical student knowledge recall and application ability compared to a text-based version of the same module. Since I am currently designing these modules for medical student education, results of this study will be used to guide development of digital modules that enhance student learning, while minimizing extraneous cognitive processing (Mayer & Fiorella, 2014).
References:
- Kumar, R. K., Velan, G. M., Korell, S. O., Kandara, M., Dee, F. R., & Wakefield, D. (2004). Virtual microscopy for learning and assessment in pathology. Journal of Pathology, 204, 613–618.
- Mayer, R. E., & Fiorella, L. (2014). Principles for reducing extraneous processing in multimedia learning: Coherence, signaling, redundancy, spatial contiguity, and temporal contiguity principles. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (2nd ed.) (p. 279-309). New York, NY: Cambridge University Press.
- Salkind, N. J. (2012). Exploring research (8th ed.). Upper Saddle River, NJ: Pearson Education Inc.