How Should the Tennessee Department of Education Encourage Data-Driven Communication Across Districts to Promote Regional Best Practice Sharing?

Executive Summary Project Question
How should the Tennessee Department of Education encourage data driven communication across districts to promote regional best practice sharing?

Client: The Tennessee Department of Education

Background Information The education policy landscape in Tennessee is changing in many ways. Amidst ambitious goals outlined in Tennessee’s Race to the Top Grant, the adoption of the Common Core State Standards, and the implementation of a more comprehensive teacher evaluation system, the Tennessee Department of Education (TDOE) rests in an ambiguous role as a state education agency. While it enforces compliance for procedural issues, the majority of the TDOE’s efforts revolve around providing district support without enforcement authority. Thus, the materials, training, and supplements provided by the TDOE must be transparent, of high quality, and usable in order for districts to perceive their state education agency as a credible source of management and support that will aid in improving educational outcomes for Tennessee students.

The goal of this project is to not only support districts in purposeful data analysis but also to build trust across districts through data transparency in order to foster idea exchanges throughout the state. This project will achieve these goals through the creation of a user-friendly tool that incorporates publicly available district demographic and achievement data. The tool will identify demographically similar districts and subsequently note which of the comparable districts are particularly high achievers. Superintendents will be able to use the tool to choose which districts to visit for professional development credit.

Within the context of both the goals outlined in Tennessee’s Race to the Top application and a thorough literature review to provide legitimacy for the tool’s direction, this tool will identify district matches, provide users with the data from which the matches were created, and show differences in achievement outcomes with the ultimate goal of state-wide data-driven conversations around student achievement.

Bright Spot Matching Tool in its Original Form

Modeled after the DART tool set from the Massachusetts Department of Elementary and Secondary Education, the first iteration of the Bright Spot Matching Tool (BSMT) in Microsoft Excel enabled regional and district staff to review the demographic and achievement data from their own district; it then provided a list of comparable districts to their own based on demographic indicators.

Within the list of comparable districts, the tool subsequently identified the district with the highest level of academic achievement in each subject area and student subgroup with available testing data for all students and subgroups.

TDOE staff could then use the comparable groups to pair high performing districts with those districts struggling in certain subject or subgroup areas in order to facilitate sharing of best practices across districts and better student achievement outcomes. Superintendents and other TDOE staff received professional development credit for participating in a site visit designed by the tool.

Feedback from Tool Users

Interviews with TDOE staff, data analysts, and regional directors revealed several strengths, areas for growth, and suggested changes to the BSMT.

Areas of Strength

• Organization and labeling of information • Quick identification of subject area and/or achievement gap concerns • Comprehensive overview of comparable districts

Areas for Growth

• Confusing portrayal of percentages
• Ambiguous interpretation of color coding • Inflexible and unintuitive general navigation • Slow performance speed • Miscellaneous formula, spelling, and other general presentation errors

Suggested Changes

• Add additional data • Incorporate additional features for comparison

BSMT in its New Form

The second version of the tool incorporates all subject area, grade, and subgroup data available for download from the Tennessee Department of Education website. Additionally, it includes more explicit directions for moving throughout the different tabs of the tool, with an introductory tab that explains the purpose of the tool and how to navigate it. Because all elements of the user manual have now been incorporated into the tool itself, the user manual will no longer be distributed with the tool. Additionally, each tab is numbered by the order of suggested use. The tool also employs a more consistent display, with all “actionable items” colored in orange. This trains users’ eyes to assume that orange sections require action from the user in order for the user to receive feedback. Each tab also links to other tabs within the tool to make navigation more intuitive and flexible. Each tab now displays a side bar with directions and other explanations that orient the user to each tab’s function. Additionally, users must down distinctly define the data they desire the tool to display through a series of drop down menus.

The tool now incorporates an additional tab that enables users to select three districts of their choice for a deeper look at achievement levels of a smaller group. Similar to the tools offered by Consumer Reports, this tab trades a wide view of one subject area across many different districts for a more holistic view of achievement of three comparable districts. Limitations of the BSMT

The BSMT’s greatest limitation is the amount of unavailable data that was not included within the tool. The district files available for download from the TDOE website are inconsistent from year to year regarding subject areas, grades, and subgroups, resulting in an incomplete data set across all subject areas, grades, and subgroups in the BSMT.

Sustainability Recommendations

• Update the BSMT twice per year • Incorporate TVAAS data into the BSMT • Expand the data set to include school-level data transition to a web based version.