As Cohen et. al recognize, “subjectivity is inescapable” (slide 7) when discussing qualitative research; however, in this context the use of individuals’ responses (i.e. qualitative responses) is highly relevant. Specifically, data will be gathered in the form of a survey from staff at the school (the teachers deliver the sessions, the admin and support staff oversee it). Staff input is being involved not simply because they are participants in the program, but also as a means to gauge the technical expertise and cultural competence of the population that delivers the program: these concepts and concerns are borrowed from Chen (2005), who articulates the importance of the consideration of the readiness of the program executors and assessors.
The staff survey will be separate from the one sent to the Grade 10 students, though both will offer 5-10 questions with responses on a scale from Strongly Disagree to Strongly Agree. As indicated earlier in this site, while it may be opportune to seek information from all students, the conversation of depth vs. breadth must be introduced when I am a single researcher. I have stated earlier that I teach nearly all grade 10 English courses (a required class) at the school, and thus can reach the widest audience here. I do not teach senior classes, and the younger students haven’t had an opportunity to experience this PLT model being addressed. Again, please note Step 4 to see further the rationale, as well as the variables addressed. While all students would be invited to complete the survey, volunteers will be taken to participate in the focus group study. To ensure fairness, we would either accept:
- All volunteers (if there are enough staff to help with the interview process, and depending on the number of volunteers)
- Randomly-selected students (in the event the volunteers over-represent the number of researchers, each volunteer will be given a code number, input into a randomized online generator like https://www.randomlists.com/random-picker, and selected from there. This is to avoid any bias and ensure a generalized, accurate sample of the student body).
Whether a or b is executed, again, depends on the number of staff to support and the number of volunteers. As Chen (2005) articulates, the success of a program (and, by my extension the evaluation) needs to account for the manpower available. As well, which I will outline, means will be taken to carefully gather, store, and interpret this evidence.
One theme of this course has been that program evaluation is not a one-shot, singular action; it is an ongoing process that is much like a Design Cycle in its research, then piloting, then interpreting, reflecting and researching, and then testing again. Further, it is recommended there be a follow-up method to address validity of findings and consider varied courses of action (Stufflebeam, 2004). As such, we will begin first with descriptive analysis: documenting our survey and focus group findings in a way that identifies more than draws connections (SpeakUp, p.51). From there, our next stage will be to move to interpretive. This is as a means to avoid bias. With this data, I/ my team will codify the qualitative responses on an Excel document with a Likert scale, borrowing from the SpeakUp document (p. 54) to codify, categorize, and then quantify our findings. Thus, data will be qualitative.
Data will be documented via Word in our team’s OneDrive. In place of aliases and code names, we will use initials (SpeakUp). I understand the demerits of transcription, largely being the time consumption and how the scripts omit nonverbal cues (Cohen et. al, N.D.), but my suggestion is to have an audio recording running through the focus group interviews. This can then be matched against the interviewer/ researcher’s typed notes. I indicate the use of typing as oppose to handwriting with respect to Cohen et. al’s (N.D.) note that the use of computer-assisted qualitative data analysis (CAQDAS) can allow for easy managing of group data (e.g. using search term function; categorizing easier to manage text and notice trends). With this, the surveys and data collected will be qualitative - specifically, descriptive data.
References
Chen, H.-T. (2005). Practical program evaluation: Assessing and improving planning, implementation, and effectiveness. Thousand Oaks, CA: Sage Publications
Cohen, L. et. al. (n.d.) Approaches to qualitative analysis. https://onq.queensu.ca/content/enforced/575070-PME802002F21/Approaches%20to%20Qual%20Data%20Analysis.pdf?ou=575070
SpeakUp (n.d.). Chapter 5 In You are the student voice. https://onq.queensu.ca/content/enforced/575070-PME802002F21/Qualitative%20Data%20Analysis%20StAR_Toolkit.pdf?ou=575070
Stufflebeam, D. (2004). Evaluation Design Checklist. Western Michigan University. https://wmich.edu/sites/default/files/attachments/u350/2014/evaldesign.pdf
Nov. 11, 2021 (updated Nov. 21, 2021)
Comments
Post a Comment