CET 720

Evaluation Plan

Watertown, SD School District (1)

Sue Mullin, Lori Fox, Tiffany Beste, Susan Fairchild, Lori Rook

Back to Leadership Theme

Phase 1

1. Evaluation Purpose

The purpose of the evaluation is to give ongoing direction to the Watertown School District board, administration, staff and students when setting priorities and making decisions about technology in the district. Another purpose is to find out if the district is accomplishing the goals of the technology plan. The third purpose is to evaluate the plan’s usefulness for the improvement of technology instruction and teacher effectiveness.

2. Evaluation Audience

The audience for this evaluation is the South Dakota Department of Education, which includes the South Dakota State Office of Curriculum, Technology and Assessment, the local school board, which is also the governing body for Lake Area Technical Institute and Lake Area Multi-District, the district superintendent, building principals, staff, parents, students, and community members. The community is a part of the audience because the taxpayers provide the funding for district technology. The school district needs to demonstrate to the community the current uses and future needs of technology. Support from the community is vital to the successful advancement of technology and the continuation of the district's leadership in the state of South Dakota. Further audiences include: TIE and North Central Accreditation Board.

3. Description of the Evaluand

The Watertown School District is located in Watertown, which is in Codington County of northeastern South Dakota, along the I-29 corridor. Watertown has a population of nearly 22,000 people. Watertown is primarily a blue-collar community made up of business and industry with an uptown business development and mall. Other businesses include Prairie Lakes Hospital and Health Care Center, Premier Bank Card, Verifications, and the Redlin Art Center. The tax base for 2007-2008 is approximately $1,373,461,933.00, which generates $14,511,324.00 in support of education. Our community has not yet been involved in an opt out. Therefore, our levy is set by the state of South Dakota. The Watertown community is overall, very supportive of education as long as their taxes are not raised. The school district has 5 elementary buildings (K-6), one middle school (7-8), and 1 high school (9-12). The district student population K-12 is approximately 4,000 students.

A technology director and support staff maintains the technology infrastructure in the Watertown School District. Other staff includes a high school technology integrationist, a K-8 coordinator who works with integration at the middle school and elementary, and 2 elementary technology integrationists that work between the five buildings. Three to five technicians are available to take care of district servicing and network needs. The high school sustains a 1:1 computer to student ratio while the middle school and elementary schools maintain a 2:1 ratio.

Each of the 5 elementary schools is comprised of the following technology components:

* All K-6 classrooms have 4 to 6 (RJ45 connector) network drops.

* Kindergarten classrooms have 1 to 3 networked student workstations (iMacs).

* 1st grade classrooms have 2 to 4 networked student workstations (iMacs).

* 2nd/3rd grade classrooms have 4 networked student workstations (iMacs).

* 4th-6th grade classrooms have 4 networked student workstations (iMacs).

* K-6, art, music, physical education, and learning center teachers have a teacher iBook with wireless network capabilities.

* Learning centers have from 4-7 networked iMacs plus 10 clamshell iBooks with wireless capabilities.

* Wireless coverage throughout each building.

* Networked laser printers.

* One or two computers labs with 25-27 iMacs.

* 6th grade iBook cart (16-26 computers) with wireless network capabilities.

* 3 – iBook carts (14 – 22 computers) with wireless network capabilities.

· 1 – OS X server.

Each of our classroom teachers has a laptop for school use. Each building has a variety of peripherals, such as digital cameras, scanners, etc. available for use. Most classrooms have mounted LCD projectors to use with laptops or TabletPC's. Most classrooms include SmartBoards, Sympodiums, Elmo’s etc. Several classrooms are utilizing responder systems.

Professional development activities are provided throughout the school year, as well as, sessions that can be taken for credit during the summer. Staff surveys are one of the tools used to determine the staff development needs for the district. Many district staff is the presenters and facilitators for these staff development opportunities.

The district technology plan is currently being rewritten. A committee of staff members representing the various buildings in our district, administrators, and a student representative are part of the process. There are five goals in the technology plan: (1) improving technology instruction and teacher effectiveness, (2) all district staff members will be provided with professional development opportunities in technology for the purpose of improving student achievement, (3) provide relevant student learning and assessment using technology, (4) provide equitable technology opportunities across the district, and (5) use of technology to promote parental involvement and increase communication with parents.

4. Guiding Evaluation Questions

• What are the ways the technology plan will improve technology instruction, teacher effectiveness, and productivity?
• How and in what ways do teachers and staff perceive the professional development in technology integration as useful?
• What are the ways technology provides relevant student learning and assessment?
• Are equitable technology opportunities being provided across the district?
• What ways will technology be used to promote parental involvement and increase communication with parents?
• What ways does the technology impact student learning?

5. Description of Evaluation Approach

The plan will be evaluated by the collection of data in a number of ways:

· Formative-Internal Evaluation

o Through presentation to the school district board, administration, staff, and community

o By needs assessments data accumulated by staff surveys, short interviews, and discussion sessions

o By continuous re-visitation of the plan and amendments to it, based on ever-changing technology needs

· Summative-External Evaluation

o By presentation to the State Department of Education in Pierre, South Dakota as part of the on-going state technology planning requirements for various funding sources

The graphic representation below is a compilation of the five evaluation questions, the information needed to answer each question, the data collection methods, and the data analysis methods. Online and paper surveys will be provided to student, staff and parent stakeholders, an attitudes assessment will be given, interviews will be done by the technology integrationists, and observations will take place in the classroom. Reports will be compiled and printed using SuccessMaker, STAR Reading and Math, DDN Campus, etc. The data will be gathered and analyzed by the Technology Department Staff.

Phase 2

Evaluation Question	Information Needed to Answer the Question	Data Collection Methods	Data Analysis Methods
What are the ways the technology plan will improve technology instruction, teacher effectiveness, and productivity?	What technology instructors are currently teaching in class; how students currently feel about technology classes; what technology areas are currently being covered	Administrative observations, online surveys, lesson plans, district summative evaluations, A class-by-class listing of what is being taught, including standards and activities	Student surveys will be compiled and recurring answers analyzed using frequency counts. Means, standard deviations and frequency distributions will be used to describe group data.
How and in what ways do teachers and staff perceive the professional development in technology integration as useful?	Which professional development opportunities involving technology have been most useful? How are you using these newly learned technologies?	Professional development questionnaire given at the close of each session, short interviews, and discussion sessions with technology integrationists	Staff surveys will be compiled and recurring answers analyzed using frequency counts.
What ways does technology provide relevant student learning and assessment?	Data result from online Benchmark testing and the technology assessment to be given at 8^th grade.	Semester and/or yearly reports of student data from pre/post testing results from District Benchmark testing program. Continuous re-visitation of the student data during monthly District in-services based on changing technology needs	Quantitative assess- ments of numerical data from District Benchmark testing and technology reports. These reports will be analyzed during grade level, staff meetings, and TCAP sessions to determine areas of strength and areas of concern.
Evaluation Question	Information Needed to Answer the Question	Data Collection Methods	Data Analysis Methods
Are equitable technology opportunities being provided across the district?	Data from the Technology Competency Survey distributed to all instructional staff during the first quarter of the school year	Periodic and cyclical inventory of all technological aspects in the district—hardware, software, peripherals, infrastructure, etc.	The quantitative percentages of the responses for each question on the Technology Competency Survey
What ways will technology be used to promote parental involvement and increase communication with parents?	What ways parents use and access technology to gain information, communicate with teachers and staff about their child, and become involved in their child’s education	Annual parent surveys, use of DDN Campus Parental Portal, tracking of hits to District Websites, attendance at technology training sessions	Reports of summary statistics of parent responses to surveys, DDN Campus Parent Portal use, and hits on District websites. A course by course listing of sessions and attendance
What ways does the technology impact student learning?	Data from the student survey, enrollment figures from online classes and technology courses, progress information from Successmaker, STAR and AR reports	Student surveys, student enrollment in courses based upon technology use by instructor, completed courses and credits earned in computer classes, SuccessMaker course reports, STAR Reading and Math reports, Accelerated Reader Diagnostic Reports	Summary of the responses for each question from the student survey, percentage of students who participated in and completed online courses, …progress reports from SuccessMaker, STAR, and Accelerated Reader

Based on the planning above, several documents have been developed, including a Watertown School District Staff Survey, a student technology survey, a parent technology survey, an inventory checklist, rubric for classroom web pages, etc. Several samples are included as representatives of the tools that will be created within this document. The online surveys can be accessed at the following locations:

•Parent Technology Survey http://www.quia.com/sv/183519.html

•Student Technology Survey http://www.quia.com/sv/183541.html

•Watertown School District Staff Survey http://watertown.k12.sd.us/curres/k-8needs.asp

•A+Rubric for Classroom Web Pages http://www.uwstout.edu/soe/profdev/webpagerubric.html

Because the Watertown School District has not opted out of the current local tax structure, the cost of this evaluation must be kept to a minimum. A small cost will be incurred to create, copy, and disperse some of the tools. Because most of the surveys will be online utilizing the district’s website and free of charge or the district’s Quia subscription, this will help defuse the cost of the evaluation.

The following pages contain other tools to be used for evaluation purposes:

A. Software Evaluation

B. Hardware Audit

Software Evaluation

The following questions need to be answered prior to purchasing software:

1) Will the software be used district wide, by grade level, or within subject area?

2) Is the software intended to teach or entertain?

3) Will it reinforce what is currently taught within the curriculum?

4) Will it replace existing curriculum?

5) Will it be used strictly as a tutorial?

6) Which state standards will the software meet?

The following rubric is for evaluating software.

Software Evaluation Rubric

0 = not present 1 = unsatisfactory 2 = satisfactory

3 = outstanding

	0	1	2	3	NA
Technical Considerations
Easy to install
Loads quickly
Technically sound (works as expected/indicated)
Does not take up excessive disk space
Allows for teacher individualization
Can be adapted for special needs students
Cost Effectiveness
Instructional Content
Clearly defined objectives
Activities support objectives
Activities interacts with the learner
Content is in the right order
Offers variability for repeat users
Promotes Cultural Diversity
Contains accurate information
Cross Curriculum
Instructional Design
Summaries and reviews are provided
Feedback is appropriate
Feedback is immediate
Material easy to review and go back to
Progress report (record keeping and reporting)
Student and/or teacher can control the rate, sequence and level of difficulty (higher order thinking skills)
Variety of color, sound and graphics to hold interest of learner
Sound can be turned on and off and is appropriate
Handles a wide variety of student responses
Can be used cooperatively
Provides a variety of learning techniques (games, tutorials, simulations and quizzes)
Available outside of school
Ease of Use
Can be used independently
Help command is easily accessed or available online
User can easily manipulate between screens
Manual is easy to read
Training is readily available
Navigation
Clear directions
Easy to navigate and control
Easy to go back to prior steps
Easy to check progress
Support
User guide/documentation
Accompanying printed material and suggestions for use in curriculum
Technical support via phone or WWW

Hardware Audit

The following list contains the goal and objectives related to hardware. Please mark the appropriate box for each item.

Goal: Provide equitable technology opportunities across the District.
Objective 1: A computer ratio is established and maintained. Funding will support the ratio.
	In Place	In Progress	Not in Place
1:1 at the High School
2:1 at the Middle School
2:1 at the Elementary Schools
All classroom and specialty teachers will have a laptop for school use.
District will follow the purchasing cycle to meet the needs of staff and students with regards to technology integration.
Objective 2: Peripheral hardware will be purchased to support the hardware/ curriculum needs.
Continued upgrade and purchase of peripheral materials
Explore emerging technologies and implement those which enhance teaching and learning
Objective 3: Update and upgrade infrastructure each year as necessary.
Maintain wireless coverage throughout the District

	In Place	In Progress	Not in Place
Replace and upgrade switches and servers as necessary to support the integration of technology
Maintain the security of the network throughout the District
Add adequate bandwidth for the District
Objective 4: Monitor on a yearly basis the technology support in order to meet the goals and visions of technology.
Each building will have a full-time Technology Integrationist.

How Data will be Handled:

· Administrative Observations/Summative Evaluations: These annual administrative evaluations will reveal staff strengths and weaknesses in the areas of technology integration. This data can be compiled at the district level to provide a full spectrum of abilities and needs to be addressed through professional development opportunities.

· Surveys: The technology department and the evaluation team will further review the survey results and detail analysis. Item analysis will determine strengths as well as areas of exigencies.

· Interviews and Discussion Sessions: Integrationist will be interviewing staff members using open-ended questions. Responses will be categorized in relation to similarity, frequency and percentages in comparison to the whole group. This information will indicate areas of strength and weakness of staff members in the areas of technology integration and skills. Technology integrationists will share insight relating to district professional development opportunities and the integration in the classroom.

· Benchmark Testing: Each staff member will generate an Item Response Analysis Report and a Class Mastery Report. These reports are analyzed and interpreted at team meetings. The data is also compared to grade level curriculum standards in order to determine classroom lessons.

· Software Evaluation: Each team of evaluators uses the rubric to evaluate the software options and compile the final recommendations. As per District policy, all recommendations must proceed to the building principals and the technology department before purchase.

· Attendance: Numerical data will be collected at each parent meeting and staff development session. This data will be analyzed to determine interest in and necessity for future sessions.

· Enrollment: Data from the number of students enrolling in classes integrating technology versus traditional paper-pencil environments.

· Reports: STAR Reading and Math tests are given to students in grades third through sixth at least twice per year via computer. The homeroom teacher generates Diagnostic and Growth reports. These reports are utilized to determine student needs, summer school placement, etc. Accelerated Reader reports are also used to diagnose areas of strength and needed interventions for students on an individual basis.

Reporting:

The evaluation results will be reported throughout the school year to all stakeholders at staff meetings, parent advisory, PTO/PTA, and school board meetings. The evaluation committee will use the results of the surveys to create visual presentation charts and graphs to provide stakeholders with a summary of the results. Frequency tables will also be used to address strengths and weaknesses throughout the district. At periodic intervals during early dismissal in-services, staff will take a closer look at the frequency tables and utilize the information to make appropriate changes in curriculum. The teacher in-service at the beginning of January will be the perfect time for the staff to take a more in-depth look at the data and work as a department or grade level to design lessons or activities that meet these needs. Is the staff using technology effectively and are the students comfortable? At this time staff can make changes for the second half of the year.

In February, staff will examine the results of the software portion of the evaluation. Because this is budget time, decisions will need to be made expeditiously as to the type of software necessary for purchase recommendations. As per District policy, purchases are all made through the building principal and the District Technology Department.

At the end of April, the technology evaluation team will re-evaluate the results to determine the types of training the staff needs to change the next year’s evaluation of the technology plan. Teachers will have an opportunity to ask questions and provide input.

Finally at the May School Board meeting, results of the evaluation will be presented in chart and graph format using PowerPoint. These meetings are broadcast through the community public access channel. Stakeholders will be made aware of strengths and weaknesses throughout the district. Because the teachers will have summer training in place by this time, the evaluation committee will be able to assure the school board and other stakeholders that progress is being made for the following year.

Conclusion:

An evaluation of this type is never completely finished. Given the ever-changing world of technology, the plan must also evolve and transform in order to further the Watertown School District’s mission statement: “Enabling all students to succeed in an ever-changing world.”

Back to Leadership Theme