How should the Tennessee Department of Education encourage data driven communication
across districts to promote regional best practice sharing?
Client: The Tennessee Department of Education
The education policy landscape in Tennessee is changing in many ways. Amidst ambitious
goals outlined in Tennessee’s Race to the Top Grant, the adoption of the Common Core
State Standards, and the implementation of a more comprehensive teacher evaluation
system, the Tennessee Department of Education (TDOE) rests in an ambiguous role as
a state education agency. While it enforces compliance for procedural issues, the
majority of the TDOE’s efforts revolve around providing district support without enforcement
authority. Thus, the materials, training, and supplements provided by the TDOE must
be transparent, of high quality, and usable in order for districts to perceive their
state education agency as a credible source of management and support that will aid
in improving educational outcomes for Tennessee students.
The goal of this project is to not only support districts in purposeful data analysis
but also to build trust across districts through data transparency in order to foster
idea exchanges throughout the state. This project will achieve these goals through
the creation of a user-friendly tool that incorporates publicly available district
demographic and achievement data. The tool will identify demographically similar
districts and subsequently note which of the comparable districts are particularly
high achievers. Superintendents will be able to use the tool to choose which districts
to visit for professional development credit.
Within the context of both the goals outlined in Tennessee’s Race to the Top application
and a thorough literature review to provide legitimacy for the tool’s direction, this
tool will identify district matches, provide users with the data from which the matches
were created, and show differences in achievement outcomes with the ultimate goal
of state-wide data-driven conversations around student achievement.
Bright Spot Matching Tool in its Original Form
Modeled after the DART tool set from the Massachusetts Department of Elementary and
Secondary Education, the first iteration of the Bright Spot Matching Tool (BSMT) in
Microsoft Excel enabled regional and district staff to review the demographic and
achievement data from their own district; it then provided a list of comparable districts
to their own based on demographic indicators.
Within the list of comparable districts, the tool subsequently identified the district
with the highest level of academic achievement in each subject area and student subgroup
with available testing data for all students and subgroups.
TDOE staff could then use the comparable groups to pair high performing districts
with those districts struggling in certain subject or subgroup areas in order to facilitate
sharing of best practices across districts and better student achievement outcomes.
Superintendents and other TDOE staff received professional development credit for
participating in a site visit designed by the tool.
Feedback from Tool Users
Interviews with TDOE staff, data analysts, and regional directors revealed several
strengths, areas for growth, and suggested changes to the BSMT.
Areas of Strength
• Organization and labeling of information
• Quick identification of subject area and/or achievement gap concerns
• Comprehensive overview of comparable districts
Areas for Growth
• Confusing portrayal of percentages
• Ambiguous interpretation of color coding
• Inflexible and unintuitive general navigation
• Slow performance speed
• Miscellaneous formula, spelling, and other general presentation errors
• Add additional data
• Incorporate additional features for comparison
BSMT in its New Form
The second version of the tool incorporates all subject area, grade, and subgroup
data available for download from the Tennessee Department of Education website. Additionally,
it includes more explicit directions for moving throughout the different tabs of the
tool, with an introductory tab that explains the purpose of the tool and how to navigate
it. Because all elements of the user manual have now been incorporated into the tool
itself, the user manual will no longer be distributed with the tool. Additionally,
each tab is numbered by the order of suggested use.
The tool also employs a more consistent display, with all “actionable items” colored
in orange. This trains users’ eyes to assume that orange sections require action
from the user in order for the user to receive feedback. Each tab also links to other
tabs within the tool to make navigation more intuitive and flexible. Each tab now
displays a side bar with directions and other explanations that orient the user to
each tab’s function. Additionally, users must down distinctly define the data they
desire the tool to display through a series of drop down menus.
The tool now incorporates an additional tab that enables users to select three districts
of their choice for a deeper look at achievement levels of a smaller group. Similar
to the tools offered by Consumer Reports, this tab trades a wide view of one subject
area across many different districts for a more holistic view of achievement of three
Limitations of the BSMT
The BSMT’s greatest limitation is the amount of unavailable data that was not included
within the tool. The district files available for download from the TDOE website are
inconsistent from year to year regarding subject areas, grades, and subgroups, resulting
in an incomplete data set across all subject areas, grades, and subgroups in the BSMT.
• Update the BSMT twice per year
• Incorporate TVAAS data into the BSMT
• Expand the data set to include school-level data transition to a web based version.