The Lessons of Middle Massachusetts

The 50 most demographically enriched communities of Massachusetts - Advantaged Massachusetts - are fine places to live and attend school. On the other hand, these districts may not have much to teach us about making public education more effective. Students in these districts, by dint of parental involvement and positive demographics, are well prepared academically. Just about all of them do well on just about any test.

Communities in that part of the state known as Middle Massachusetts are much more interesting from an education reform perspective. In the demographic ranking of Massachusetts school districts, about 140 districts are concentrated in the demographic center of the state. These districts, with over two-and-a-half million people, make up what might be called Middle Massachusetts. They are the every-towns that comprise the "typical" Massachusetts community - Norwood, Beverly, Abington, South Hadley. These systems, which are essentially demographically neutral, may be well suited to play a crucial role in education reform.

For the state as a whole, demographic differences among the districts explain 83% of the variation in the districts' average overall test scores. All or much of the other 17% of the variation is probably explained by the differences in how the school districts themselves organize their educational delivery systems.

When looking at either end of the demographic ladder (Advantaged Massachusetts with Weston, Wellesley, Medfield; Challenged Massachusetts with Lawrence, Lynn, New Bedford), we notice that scores tend to be very high or very low. While there is some variation, the solid suburbs score well. Conversely, the cities have low scores, again with some variation. In short, the overwhelming majority of students in advantaged districts pass MCAS easily; most students in disadvantaged districts are very far away from passing. The pattern in Middle Massachusetts is different. Its districts exhibit a wide range of test scores - although their demography is relatively similar. This variation can be seen in the following bar graphs that show total MCAS scores for the 2000 MCAS Grade 8 tests. (I am using the Grade 8 tests because these students are members of the second class that will have to pass MCAS in order to qualify for a high school diploma.)

For each of the 14 districts in the demographic middle of the state - the seven districts above the demographic mean and the seven below the mean - the tip of the bar represents its average test score on the 2000 MCAS. While demography for this group is a constant, scores range widely. The top scaled score of these 14 districts in Grade 8 ELA is 247 (Braintree); the lowest score is 238. In Grade 8 Math, the top scaled score is 239; the lowest score is 225. A 9-point difference in ELA scores and a 14-point difference in Math scores in communities that are very similar is significant in that top-scoring systems may well have lessons to teach systems that have not done as well.

The range in the percentage of students who failed the 2000 ELA MCAS varied from 1% (Tyngsborough) and 3% (Braintree) to 14%. Similarly, the range of Fail in Grade 8 Math was 14% (Tyngsborough) to 33%, again a wide performance range in communities that are very similar demographically. Middle Massachusetts school systems looking to improve their achievement might look to Braintree, Woburn, Tyngsborough, and Beverly for guidance.

As you can see, there is wide variation in actual test scores and fail rates in towns that are very similar demographically.

Since the demographic variation is slight, but the variation in test scores is great, this pattern suggests that much of the variation is explained less by demography than by differences in what the schools of Middle Massachusetts are doing. Further, the test scores of Middle Massachusetts districts with high positive numbers on the Effectiveness Index are just as high as the scores of many of the advantaged districts. For example, for Grade 4, a class that started school after the Education Reform Act of 1993 was enacted, the total MCAS Grade 4 scores of Woburn, a Middle Massachusetts district, are equal to the scores of Newton, a very advantaged community. Similarly, fourth graders in Arlington, another Middle Massachusetts district, scored one point better than students in Franklin and Milton, two demographically advantaged districts. It is important to note that Newton, Franklin, and Milton all made the Effective School Districts list for 2000; these systems are doing a good job adding value to the educational performance of their students. The story here is how well schools in Woburn and Arlington did in moving their students beyond their demographics.

ELA Grade 4: 2000 MCAS Scaled Scores (ELA, Math, Science and Technology)

If more Middle Massachusetts districts become as effective as Woburn and Arlington, then more Middle Massachusetts districts will see improved student achievement. Because Middle Massachusetts districts are demographically similar, what makes for effective schools in these districts is more likely to be replicable for other Middle Massachusetts districts looking to improve. Thus, Middle Massachusetts can be an especially fruitful place to seek, and expect to find districts whose schools are well positioned to help all of their students achieve high standards.

Student Achievement in Demographically -Challenged Districts

Three years of evaluating school district effectiveness with respect to demography has demonstrated that it is much more likely that an upper- or middle-demography district will over-perform its demographically-predicted score. Thus, it is not helpful to compare cities and disadvantaged towns to more middle- and upper-class communities when assessing performance. Lower demography communities should be placed together and compared against each other to identify systems that are doing an exemplary job.

This does not mean that standards should be lowered for some districts. It does mean that developing the kinds of schools where all children in fact do learn will require more creativity, commitment, and time when dealing with demographically disadvantaged communities.

Developing educational strategies that can overcome the impact of disadvantageous demography is a major challenge for many districts. This reality tends to make it much more difficult for an urban district or a disadvantaged town to score above predicted levels. Achieving even a few points above the demographic prediction may indicate positive reform efforts in a lower-demography system. A disadvantaged system that scores at state average will have had to do much more work than a middle-class system with the same score

The following reflects an analysis of the MCAS performance of the lower 50 school districts as sorted by demography. This research combines results from the 1998, 1999, and 2000 MCAS. The scaled scores listed below reflect total performance on all three MCAS administrations. (See Appendix E for more detail.)

• In Grade 4 ELA, Ayer and Everett scored 9 scaled score points above each district's predicted performance level. They also scored above the state average.

• In Grade 4 Math, Ayer and Everett scored 15 points above predicted score, and both districts scored at or above state average. Orange and Clinton were also strong performers.

• In Grade 4 Science and Technology, Orange scored 14 points above predicted level. Everett scored 12 over prediction, and Ayer scored 11 points above the demographically predicted score. All three systems were above state average.

• In Grade 8 ELA, Clinton scored 10 points above predicted score, Berkshire Hills scored 9 points over predicted level, and Gardner scored 8 points over prediction. All of these systems scored above state average.

• In Grade 8 Math, Ludlow scored 14 points and Methuen scored 8 points over predicted score, with both systems above state average.

• In Grade 8 Science and Technology, Gardner and Berkshire Hills scored 10 points above prediction with both systems scoring above state average.

• In Grade 10 ELA, Southbridge's actual score was 12 points over predicted score, Fitchburg's students scored 9 above prediction, and Hull scored 6 points over predicted performance. All three districts were above state average.

• In Grade 10 Math, Ayer topped the demographically-challenged over-scorers at 18 points over predicted score. Southbridge and Ware scored 16 over predicted level. All three were above the state average.

• In Grade 10 Science and Technology, Webster, North Adams, and Clinton scored 9 points over predicted level with all three scoring above the state average.

Large Urban Systems

Conspicuous in their absence from the lists of schools that over-perform demography are most large urban districts. Lawrence and Chelsea, two very disadvantaged systems that have developed collaborations with the state Department of Education and Boston University respectively, are recognized in this research for producing scores that exceed demography (see following section). There are few other major urban districts that are identified for noteworthy performance.

This is not a surprise in that a basic finding of this research is that demographically challenged systems have a much more difficult time in exceeding demographic expectations than do middle-class or advantaged districts. (See page 5). A superintendent in a district like Lowell or Worcester faces a much more difficult challenge in moving students up to meeting high standards than does a superintendent in a district like Beverly or Dedham. Despite this, there are some large urban systems that have demonstrated a capacity to move beyond demography.

The most encouraging work has been done in Grade 4. These children started school after education reform had been enacted. They are young and susceptible to improved classroom organization and more effective pedagogy. Unfortunately, urban students in Grade 8 and especially in Grade 10, have spent much of their academic careers in classrooms where meeting high standards was not a priority. Therefore, Grade 4 and earlier is where many savvy superintendents have concentrated their reform resources.

In ELA 4 on the 2000 MCAS, Springfield, Chelsea, Holyoke, Worcester, and New Bedford have shown the most over-performance of major urban systems. In Math 4, Chelsea, Lawrence, Worcester, and Springfield have exceeded demography. And in Grade 4 Science and Technology, Chelsea, Lawrence. Fall River, Worcester, and Springfield have moved student performance beyond their community's characteristics. The overscores for these systems are relatively modest when compared to those of middle-class and advantaged systems, but, since the challenge is so much greater in urban districts, the gains are important. (See Appendix E.)

Lawrence, Chelsea, and Holyoke

Lawrence, Chelsea and Holyoke are the three most demographically disadvantaged communities in the Commonwealth. They are in a class by themselves when it comes to the challenges they face each day in overcoming demographic disadvantage. Chelsea schools have been in partnership with Boston University for many years, and the Massachusetts Department of Education has taken a special interest in the Lawrence public schools.

Lawrence and Chelsea tend to be identified by this research as over-performing in terms of their demography. These systems are not designated Effective because they do not score at or above state average, a requirement for an Effective School District in this report. This year, as was the case in the past two reports, Chelsea and Lawrence are among those challenged systems that over-perform on MCAS. The fact that, even with a strong over-performance over their demographically-predicted scores, they are still so far below even state average, is a measure of the enormity of the challenge that faces urban educators.

Holyoke, with similar demography, does not fare as well as Lawrence and Chelsea in its quest to add value to the learning readiness of its children. Holyoke has not had special attention paid to it by the Commonwealth, nor has it been able to collaborate with an outside higher education institution to help improve education. While neither Lawrence nor Chelsea is even close to creating schools that are successful in helping their children reach basic standards, they do fare better than Holyoke. The extra help received by Lawrence and Chelsea has helped their situations, although each district has miles to go before it even begins to meet the goal of ensuring that all children learn.

The Importance of Reading and Writing

While students need to develop a broad range of skills and competencies, reading and writing are the foundations for academic success. E. D. Hirsch, Jr., author of The Schools We Need and the developer of the Core Knowledge Curriculum, observes that "... good competency tests in reading turn out to be powerfully indicative of achieved abilities that go far beyond reading." (Hirsch, Education Week, Feb 2, 2000; p, 40 in "The Tests We Need," p, 64) Veteran educators understand that reading is the linchpin of academic success.

Having solid reading and writing skills are necessary conditions for doing well on the MCAS tests. This is true even of the tests in mathematics. Many of the problems on the mathematics tests, particularly in grade eight and ten, are word problems. You cannot understand these problems if you cannot understand the words. In all subjects, moreover, many questions call for a written answer, as short as a sentence or two or as long as an essay of several paragraphs.

The tables in Appendix A lists Effective and Noteworthy districts in terms of student reading performance by grade level on MCAS 2000 (See ELA scores by grade). Several interesting districts that are effective in teaching reading (ELA) in Grade 4 are Eastham, Woburn and Ayer. Each of these districts had MCAS scores substantially above their predicted value and above state average. In Grade 8, Bedford, Nauset regional, Brookline, and Gardner solidly over-scored their demography in ELA and scored above state average.

Grade 10 presents an interesting situation. Some people say that today's high school students understand that MCAS does not count towards graduation so they do not invest significant personal resources in doing their best on the test. Time will tell if that is correct, but it is still important to recognize those systems that do perform beyond demographic expectations. Grade 10 over-performers in English Language Arts include Hatfield, Stoneham, Harvard, Shrewsbury, and Southbridge.