The Effectiveness Index lets us identify two types of school districts that are interesting in terms of education reform:Effective Districts and Noteworthy Districts.

An EFFECTIVE district meets two specifications:

1. Its Effectiveness Index is a positive number -that is,its actual score on the test is substantially higher than its demographically likely score.

2. Its actual score is equal to or higher than the average MCAS score for the state as a whole.

Thus,a district that meets both of these specifications invites further scrutiny to determine whether its practices provide a worthwhile model for other districts.

In this research,Braintree,Pembroke,and Woburn are districts who have consistently over-performed demography while scoring above state average. Orange,a demographically challenged Western Massachusetts district,has been identified consistently as moving students beyond their demography and scoring over state average.All of these districts kept up their performance on the 2001 MCAS and are identified as Effective.Other such districts include the following:

• Grade 4 solid performers include Marshfield,Arlington,Plymouth,and East Longmeadow.

• Districts that did particularly well in terms of their demography on the 2001 Grade 8 MCAS include Central Berkshire,Ayer,Palmer,Hadley,and Southern Berkshire in Western Massachusetts,and Norton,Brookline,Hanover,and Ipswich,with Ipswich continuing its impressive performance on the Math Grade 8 MCAS.

• On the Grade 10 MCAS,Pentucket,Nauset,Masconomet,Stoneham,Lenox,and Northampton all performed consistently well in terms of demography.

A NOTEWORTHY district fits the first specification but does not fit the second.It does not score at or above state average.Since its performance helps its students to go beyond their demography,it is still worthy of note.What such a district is doing educationally can hold useful lessons for districts that are demographically similar, but do not outscore their demography.Such a district is more likely to deliver a return on future public investment than is an ineffective district.

Noteworthy districts are usually demographically challenged.Since the major goal of education reform is to lift all students up to a basic skills level in Math and English,Noteworthy districts have much to teach us about making schools more effective for all students.Here Everett continues to perform well,as do Ware and Clinton.Gardner and Ludlow perform very well on the Grade 10 MCAS.

Please see Appendix A for a listing of Effective and Noteworthy school districts on the 2001 MCAS .

Education Achievement and Basic Kinds of Community in Massachusetts

It is very difficult to understand how well we are doing in terms of implementing education reform unless the various characteristics of Massachusetts'communities are factored into the evaluation.The Effectiveness Index research is based on a methodology that statistically analyzes demographic data so that individual school districts can be compared to their demographic peers.For purposes of this report,I will place the Commonwealth ’s school systems into four categories:Advantaged Massachusetts;Middle Massachusetts;Challenged Massachusetts;and the Fifteen Cities. 6 Each of these groupings is based on the demography of their communities. 7

Advantaged Massachusetts communities (the top 50 in the state ’s demography)are characterized by high education levels,high incomes,and high property values.Weston, Hingham,Franklin and Natick are part of Advantaged Massachusetts.In terms of educational characteristics,per-pupil expenditures (PPE)are generous,averaging $6,217, with a range of $10,567 to $5,270.(These numbers are Fiscal Year 1999 figures.Current spending is higher).State education reform aid tends to be low,averaging $648 per year compared to state average of $955.8 The range of state aid is $1,797 to $245.(Please note that most districts in this cluster receive very little state aid.The most generous aid goes to regional school districts.)There are very few students (4%)who qualify for free or reduced lunch,a measure of poverty.The statewide average for Free/Reduced Lunch is 26%. Almost all of the students speak English as a primary language.Parents tend to have been successful in education,and they understand the importance of good schools.

Middle Massachusetts consists of about 110 communities (the number varies with grade due to the fluctuating number of regional systems as grade level changes) clustered around the demographic center of the state.These are typical Massachusetts hometowns,where average is the norm.Middle Massachusetts is home to communities like Norwood,Beverly,Abington,and South Hadley.These districts average a $6,289 per-pupil expenditure.PPE ’s range from $9,557 to $5,261. Typically,each receives $833 per year per pupil in new education reform aid, compared to the state average of $955.There are relatively few students (12%)who qualify for free or reduced lunch,a measure of poverty.The statewide average for Free/Reduced Lunch is 26%.

Challenged Massachusetts is made up of the 50 communities at the lower end of the state ’s demography.Districts in this category include communities such as Hull, Palmer,Waltham,Cambridge,and Holbrook,as well as many smaller cities.These districts average a $6,314 per-pupil expenditure with a range of $11,271 to $5,501. Typically,each receives $1,229 per year per pupil in new education reform aid, compared to the state average of $955.The range of state education reform aid is $2,476 to $233.Many students (33%)qualify for free or reduced lunch,a measure of poverty.The statewide average for Free/Reduced Lunch is 26%.

The Fifteen Cities are the 15 urban communities that are the most demographically disadvantaged in Massachusetts.This cluster includes cities such as Worcester, Boston,Fitchburg,and Lawrence.These districts average a $6,931 per-pupil expenditure with a range of to $8,487 to $6,004.Typically,each receives $2,504 per year per pupil in new education reform aid,compared to the state average of $955. The range of state education reform aid is $3,643 to $1,559.Many students (58%) qualify for free or reduced lunch,a measure of poverty.The statewide average for Free/Reduced Lunch is 26%.

There are good reasons to focus on the Fifteen Cities:

• These 15 communities educate 28%of our students.If the Massachusetts economy is to retain its vitality,and we are to offer quality education for all,we must find some way to be more effective educating the 300,000 or so students in these systems.

• These systems have large percentages of students failing MCAS.In fact,28%of the state ’s students attend school in the Fifteen Cities,but 46%of the students who failed the Grade 10 Math MCAS and 49%of the students who failed the Grade 10 ELA MCAS attend school in these 15 systems.

• These systems are remarkably similar in their demographic profiles.These 15 districts are large school systems that are home to 45%of the state ’s minority students and to most of the state's poor people.Since minority academic performance is a major concern of many people,reforming these systems is the key to boosting minority achievement.

Making the Grade on the 2001 MCAS

MCAS and the Class of 2003 Students who took last Spring's MCAS and are members of the Class of 2003 will have to pass MCAS – score Needs Improvement or better on both the Math and English Language Arts (ELA)tests – in order to graduate.Scores on the 2001 Grade 10 MCAS were much higher than scores on any previous administration,and,on a statewide basis,82%passed ELA and 75% passed Math.This represented a 16%gain on ELA and a 20%gain on Math pass rates compared to the previous year.

A report I worked on in September 2001 with MassInsight Education analyzed the 2000 MCAS on terms of pass rates for various student populations and found that thousands of Grade 10 students who did not pass the 2000 MCAS were within a few points of passing.The study also noted that many of the students who did not pass the 2000 MCAS were members of special populations -Special Education, Vocational Education,and Limited English Proficient students 9 .It is likely that the fact that the 2001 MCAS was the first one that counted motivated many students to take the test seriously.It also is likely that educators developed and implemented academic interventions that were successful in moving students who were close to passing up to passing.

Progress of Different Kinds of School Districts on the Grade 10 MCAS

While it is interesting to evaluate individual districts in terms of how well they score relative to the demography of the community -the purpose of this report - ultimately education reform is about year-to-year improvement.Districts,students, and teachers are expected to boost performance over time.

Demographic analysis can be a valuable tool to determine how well education reform is doing in meeting the goal of boosting students achievement.At day's end,in Massachusetts on the 2001 Grade 10 MCAS,many students in many districts of differing demography did remarkably well.Analyzing how students in different kinds of communities performed may give us valuable insight into how well education reform has worked so far and what we can expect in the future.

Clearly,students who are fortunate enough to go to school in demographically advantaged districts are much more likely to achieve high scores on any assessment of academic skills including SATs,Stanford 9s or MCAS.On the 2001 MCAS, Dover-Sherborn had 94%of its students in English and 82%of its students in Math at Advanced or Proficient on the assessment.Wayland reported 92%/85%in ELA/Math MCAS Advanced and Proficient.Lincoln-Sudbury,a hotbed of anti- MCAS sentiment in years when the test did not count,had 88%/92%of students in ELA and Math in Advanced or Proficient in 2001,the first year MCAS counted. Berlin-Boylston reported 90%/73%of students in Advanced and Proficient in ELA and Math respectively.Pentucket had 76%/76%in Advanced and Proficient in ELA and Math respectively.

While we celebrate the success of any student on any assessment,we are not surprised when students from advantaged circumstances perform well.What is more interesting is the performance of students from less advantaged communities. The 2001 MCAS,administered two years before the students who must pass it (the class of 2003)graduate,shows that a solid majority of students in Massachusetts who are scheduled to graduate in 2003 have already passed MCAS.While we want all students to learn more and do better,the immediate challenge for many educators,policymakers and citizens is to focus on moving the students who have not yet passed MCAS up to passing.Because demography closely mirrors achievement,most of the students who have yet to be successful at MCAS attend school in districts characterized by lower demography.

The results of the 2001 Grade 10 MCAS demonstrated that students in many districts that do not enjoy demographic advantage or even demographic neutrality performed well.Dennis-Yarmouth,Gill-Montague,and Ayer,three demographically challenged districts,had 62%/52%,60%/53%and 61%/42%of their students scoring Advanced and Proficient on the 2001 Grade 10 ELA/Math MCAS,well above the scores expected from such communities.

Grade 10 MCAS Performance Progress in Different Kinds of Community

Clearly,educators in Duxbury (97%/96%Pass in ELA 10/Math10 on the 2001 MCAS and 80%Advanced/Proficient in ELA 10 and 70%Advanced/Proficient in Math 10)face different challenges in boosting educational achievement than do teachers and administrators in Lynn (67%/63%Pass in ELA 10/Math 10 and 29% Advanced/Proficient in ELA 10 and 26%Advanced/Proficient in Math 10).

Yet,despite the very different MCAS performances all over Massachusetts,all schools want to do better.Given that,let us look at how different kinds of communities have done improving achievement during the MCAS years.The following looks at progress made during the past four years by different kinds of community.

While the overall goal of any education reform,including MCAS,is to increase the achievement of all students,strategies and tactics must reflect the specific needs of different kinds of communities.Many students in advantaged communities will strive to move up from Proficient to Advanced or from Needs Improvement to Proficient.In Middle Massachusetts and demographically challenged communities, the goal might be to move students up from Needs Improvement to Proficient or Advanced.In the 15 Cities,our urban core communities,moving students out of Fail is usually the initial goal.

The following assesses MCAS progress in terms of absolute scores from a base 1998/99 level to 2001.The base is the average of the 1998 and 1999 MCAS.I am using two years of data to get a better base line against which to measure progress. The first year of any thing or venture,be it a new car or a new educational assessment,tends to have some problems.By using the average of the 1998 and 1999 MCAS assessments,we have a more accurate starting point against which to assess subsequent progress.

I am focusing my analysis on the Grade 10 ELA and Math tests because of the keen public interest in the scores and because the 2001 MCAS is the first one that counts.

Advantaged Massachusetts

This cluster includes the 50 most demographically advantaged communities in Massachusetts.Approximately 158,000 students are educated in classrooms in these districts,17%of the total number of public school students.

On the 2001 Grade 10 MCAS,77%of the students tested scored Advanced or Proficient in ELA with 19%in Needs Improvement and 4%in Fail.On the Math Grade 10 assessment,72%scored Advanced or Proficient with 22%in Needs Improvement and 7%in Fail.While work needs to be done to help the students who did not pass,a better mark of success would be tracking the progress of students in moving up to Advanced or Proficient over time.

Strong performers in absolute gains on the Grade 10 2001 MCAS compared to the base 1998/99 average include:

• Swampscott with 17%fewer students failing ELA than in the base years (1998/99 average);Hopkinton with 15.5%fewer students in Fail;Longmeadow and Mendon-Upton with 13%;and Burlington with 12%fewer students in Fail in 2001 compared to the 1998/99 base.

• Easton with 34.5%fewer students failing Math than in the base years (1998/99 average);Burlington with 34%fewer students in Fail;North Reading with 33.5%;Swampscott with 31.5%;and Natick with 30%fewer students in Fail in 2001 compared to the 1998/99 base.

• Swampscott had a 31.5%gain in the percentage of students in Advanced or Proficient in ELA ;Longmeadow had a gain of 27.5%;Mendon-Upton 26%; Hopkinton 25.5%;and Lincoln-Sudbury 24%.

• Westford had a 39.5%gain in the percentage of students in Advanced or Proficient in Math ;Easton had a gain of 37.5%;Longmeadow 37%;Hopkinton 34%;and Swampscott,Groton-Dunstable,and Burlington with 33.5%.

Middle Massachusetts

This group includes the 108 communities that represent the middle demography of Massachusetts.About 326,000 students attend school in these communities,35%of the total.

On the 2001 Grade 10 MCAS,61%of the students tested scored Advanced or Proficient in ELA with 29%in Needs Improvement and 10%in Fail.On the Math Grade 10 assessment,54%scored Advanced or Proficient with 31%in Needs Improvement and 15%in Fail.While work needs to be done to help the students who did not pass,another mark of progress would be tracking the success of students in moving up to Advanced or Proficient over time.

Strong performers in absolute gains on the Grade 10 2001 MCAS compared to the base 1998/99 average include:

• Mashpee with 21.5%fewer students failing ELA than in the base years (1998/99 average);Somerset with 20.5%fewer students in Fail;Quabbin with 20%; Abington with 19.5%;Dighton-Rehoboth with 18.5%;and Beverly, Middleborough,and Millis with 18%fewer students in Fail in 2001 compared to the 1998/99 base.

• Granby with 46%fewer students failing Math than in the base years (1998/99 average);Sutton with 43.5%fewer students in Fail;Mashpee with 41%;Somerset with 40%;and Abington and East Bridgewater with 39%fewer students in Fail in 2001 compared to the 1998/99 base.

• Gateway had a 39%gain in the percentage of students in Advanced or Proficient in ELA ;Granby had a gain of 35.5%;Abington 32.5%;Somerset and Northampton,30.5%;East Bridgewater,29.5%;and Mashpee and Quabbin, 28.5%.

• Sutton had a 46%gain in the percentage of students in Advanced or Proficient in Math ;Foxborough had a gain of 41.5%;Somerset,Danvers,Newburyport, and Braintree,40.5%;and Ashburnham-Westminster,Tyngsborough,and Tewksbury with a 40%gain.

Challenged Massachusetts

This group of 50 districts represents communities that are broadly middle-class yet are demographically challenged in some ways.Property values,incomes and education levels are below state average,and schools must work hard to ensure that all students learn.These systems educate approximately182,000 students,20%of the total.

On the 2001 Grade 10 MCAS,48%of the students tested scored Advanced or Proficient in ELA with 35%in Needs Improvement and 17%in Fail.On the Math Grade 10 assessment,40%scored Advanced or Proficient with 36%in Needs Improvement and 24%in Fail.While substantial work needs to be done to help the students who did not pass,another mark of progress would be tracking the success of students in moving up to Advanced or Proficient over time.

Strong performers in absolute gains on the Grade 10 2001 MCAS compared to the base 1998/99 average include:

• Narragansett with 24.5%fewer students failing ELA than in the base years (1998/99 average);Taunton with 22.5%fewer students in Fail;Southern Berkshire and Easthampton with 20.5%;Fairhaven with 20%;and Attleboro and Ludlow with 17.5%fewer students in Fail in 2001 compared to the 1998/99 base.

• Easthampton with 47%fewer students failing Math than in the base years (1998/99 average);Medford with 38.5%fewer students in Fail;Taunton with 38%;Quincy with 37.5%;Clinton with 37%;and North Brookfield and Haverhill with 36.5%fewer students in Fail in 2001 compared to the 1998/99 base.

• Southbridge had a 39.5%gain in the percentage of students in Advanced or Proficient in ELA ;Easthampton had a gain of 28%;Gardner and Ralph Mahar 27%;and Clinton 26%.

• Southbridge had a 38%gain in the percentage of students in Advanced or Proficient in Math ;Clinton had a gain of 37%;Gill-Montague 34%; Easthampton 33%;and Ludlow and Waltham,30%.

The 15 Cities

The 15 Cities are the most urban communities in Massachusetts.They are generally large in population with many residents having low incomes and limited educational credentials.The 15 Cities are home to most of the Commonwealth's immigrants and minority citizens.These communities educate approximately 256,000 students,28%of the total.

On the 2001 Grade 10 MCAS,30%of the students tested scored Advanced or Proficient in ELA with 36%in Needs Improvement and 34%in Fail.On the Math Grade 10 assessment,24%scored Advanced or Proficient with 33%in Needs Improvement and 43%in Fail.While the primary effort in these districts is to move students up to passing,another mark of progress would be tracking the success of students in moving up to Advanced or Proficient over time.

Strong performers in absolute gains on the Grade 10 2001 MCAS compared to the base 1998/99 average include:

• Brockton with 19.5%fewer students failing ELA than in the base years (1998/99 average);New Bedford with 16.5%fewer students in Fail;Boston with 16%; Chicopee with 15.5%;and Somerville with 15%fewer students in Fail in 2001 compared to the 1998/99 base.

• Brockton with 41%fewer students failing Math than in the base years (1998/99 average);Somerville with 38%fewer students in Fail;Fall River and Lynn with 32.5%;and New Bedford with 31%fewer students in Fail in 2001 compared to the 1998/99 base.

• Brockton had a 19%gain in the percentage of students in Advanced or Proficient in ELA ;Somerville and Lowell each had a gain of 14.5%;New Bedford 14%;and Chicopee 13.5%.

• Brockton had a 21%gain in the percentage of students in Advanced or Proficient in Math;Somerville and Revere each had a gain of 20.5%;New Bedford 19%;and Lowell,17%.

Please see Appendix B for a listing of other top performers in different kinds of districts in improving MCAS results over four years.

6 Please note that in previous reports I set the 15 cities as a subset of the 50 Challenged Massachusetts communities.For this report,the 15 Cities are an independent cluster.Challenged Massachusetts includes the most disadvantaged 50 communities that are demographically above the 15 major urban areas that comprise the 15 Cities. 7 The data here is for Grade 10.I am using Grade 10 (Class of 2003)because these students are expected to pass MCAS in order to graduate.

7 The data here is for Grade 10.I am using Grade 10 (Class of 2003)because these students are expected to pass MCAS in order to graduate.

8 The amount of new education reform aid that is distributed as part of the Education Reform Act of 1993 is calculated in the following manner.I took the amount of Fiscal Year 1994 Chapter 70 funding (state education aid)for each district and divided it by the number of pupils in that district.I then took the amount of FY 1999 Chapter 70 district funding and divided that figure by the number of students in the district.The difference between the two produces a per-pupil figure that represents the new education reform aid.Chapter 70 data is available from the Massachusetts Department of Revenue.

9 See A New Commitment:Effective Remediation Strategies for High School Students ,Fall 2001,available at www.massinsight.com/meri/pdf_files/A%20New%20Commitment.pdf