As part of my final course with the Digital Educational Leadership master’s program at Seattle Pacific University, I engaged in a 12 week practicum, job-shadowing two instructors who teach a technology elective course for middle school students. The course, a high school graduation requirement, focuses on preparing students to use technology for various academic and job-related objectives. It also seeks to explore the use of different digital tools and is taught using the Learning Management System (LMS), Moodle.
Although in many ways I was an independent observer, I did actively engage in the classroom. I worked with individual students on demand, had regular conversations with the instructors about their approach to the curriculum, and taught a few classes in their absence. These combined experiences provided me with an authentic and immersive understanding of the curriculum’s implementation.
After completing this practicum, I chose to engage further in my understanding of this course by conducting a program evaluation of its curriculum.
Focal Points for Evaluation
Utilizing the work of Sanders and Sullins (2006), the following focus was identified for this evaluation (p. 7):
Process of strategies for providing services to learners: to provide insights about how best to organize a school to facilitate learning
1. Curriculum design: to provide insights about the quality of program planning and organization
2. Classroom processes: to provide insights about the extent to which educational programs are being implemented
3. Materials of instruction: to provide insights about whether specific materials are indeed aiding student learning
Sanders and Sullen’s (2006) Figure 2.2 “Evaluation Information Collection and Analysis Worksheet” was used to expand upon my evaluation questions (p. 15).
|Evaluation Question||Why is the Question Important||Information Needed to Answer the Question||When and How the Information Will be Collected||Data Analysis and Interpretation Procedures|
|In what units and lessons does the course content align with ISTE Student Standards?||It is important to ensure that the objectives of the course, as outlined in the course description, are being achieved.||A review of every unit and lesson, with concurrent objectives, will be conducted.||After twelve weeks of job-shadowing, the course curriculum will be outlined and carefully examined by unit and lesson.||A spreadsheet will be created to outline ISTE Student Standards. For each standard, course content that meets this standard will be identified.|
|Does the course content align adequately , minimally, or not at all with ISTE Student Standards?||If instructors or curriculum designers were to use this evaluation, it would be important to understand where areas of improvement exist. Differentiating the extent of alignment can assist in this process.||Breaking down each standard to determine whether the specific performance indicators are being met can work to determine this, although there is acknowledged subjectivity to this process.||The extent of alignment will be evaluated simultaneously to the overall alignment.||Content marked green in the spreadsheet will identify that the performance indicator has adequate alignment to the curriculum. Yellow will designate that the performance indicator shows minimal alignment, and red will indicate that the performance indicator shows no evidence of alignment.|
|Are there recommendations for how to better align curriculum with performance indicators showing minimal or no alignment?||An evaluation is only as good as what you do with it. So, if possible recommendations are not made for future adjustments, the evaluation is not useful.||My graduate work in the ISTE Student Standards will support my professional suggestions.||Recommendations for further alignment will be made simultaneously to the overall alignment.||Suggestions recommended for fully meeting standards and performance indicators will be color coded in purple.|
Upon request, a document was provided for the course that outlined a bevy of standards aligning with the content. Given the large quantity of standards in this document, I made the decision to focus on one set of standards only, the ISTE Student Standards, for the evaluation. The ISTE Student Standards are neither formally adopted by this district nor the designers of this curriculum. However, I made the decision to use these widely accepted standards due to their comprehensive nature, applicability to a wide variety of content areas, and focus on skills beyond technology alone. Furthermore, I chose to use the draft version of the new ISTE Student Standards to ensure that this evaluation is as relevant as possible for future iterations of the course.
The stakeholders in this evaluation are predominantly the students who devote time to this class, are taught this curriculum and are expected to pass assessments to meet a high school graduation requirement. It is also important to note that this course has the potential to greatly benefit their technology skills for the future. The teachers, responsible for delivering the curriculum, are also stakeholders, as are the administrators who oversee it, the district instructional technologists who create, monitor and review the curriculum, and the parents, who have a vested interest in the academic success of their children.
Process and Results
For each ISTE Student Standard, content was methodically reviewed and culled to determine alignment with individual performance indicators. A spreadsheet was then used to identify and describe units, lessons and projects that met each one. Color coding was used to identify whether this alignment was adequate (green), minimal (yellow) or absent (red). After this initial step, recommendations were made for all performance indicators that had minimal or absent alignment and color coded in purple. Methodically working through the curriculum in this manner provided insight possible future iterations of the course.
Pie charts of the alignment were also used to provide quick visual summaries of the evaluation’s results. The pie chart seen below represents the curriculum’s overall alignment with the standards.
Individual pie charts were also created to visually demonstrate the alignment specific, individual standards. Of note was that ISTE Student Standards 4. Innovative Designer and 5. Computation Thinker demonstrated the strongest alignment, while ISTE Student Standards 1. Empowered Learner and 7. Global Collaborator demonstrated the weakest. The most powerful part of this experience was working to develop recommendations for how these standards could be better met. This process holds a lot of potential for my ability to evaluate future sets of course material, something I could easily see myself engaging in again.
Sanders, J. R., & Sullins, C. D. (Eds.). (2006). Evaluating school programs: An educator’s guide. Corwin Press.