Phi Delta Kappan, Oct 1999 v81 i2 p147 The Ninth Bracey Report On the Condition Of Public Education. Gerald W. Bracey. Full Text: COPYRIGHT 1999 Phi Delta Kappa, Inc. "Testing madness," public/private school comparisons, charter schools, and media spin dominate Mr. Bracey's annual assessment of the condition of public education in the U.S. THIS YEAR'S report begins with a brief chronicle of the testing madness that seems to have gripped us. Twenty years ago, while serving as testing director for the Commonwealth of Virginia, I thought the nation could get no nuttier about testing. But I was wrong, oh so wrong. Back then, the entire nation was consumed with that peculiar pathology known as minimum competency testing. Virginia went further, with competency tests for teachers and basic skills tests for students in the primary grades. By 1981 the infection had become so inflamed that the U.S. Department of Education held a three-day "trial" on the use of minimum competency tests. The "jury" consisted of the members of the audience. The three days were later condensed into three hours of videotape that aired on the Public Broadcasting Service (PBS). Then, my unofficial measure of the growing importance of testing was the change in how we, the consumers of tests, were treated. When I first attended the annual conference on large-scale assessment in 1977 (sponsored then by the Education Commission of the States and now by the Council of Chief State School Officers), the test publishers held "social hours," during which they served us cheap beer and wine, along with cheese and crackers, from 5 p.m. until 7 p.m. Five years later, the social hours had become hospitality suites that opened at noon and closed at midnight. Fancy hors d'oeuvres had replaced the cheese and crackers. The Psychological Corporation was famous for its mountain-sized displays of jumbo shrimp and oysters on the half-shell. The hosts now poured Chivas Regal, Jack Daniels, Stolichnaya, and similar high-priced liquors. CTB/McGraw-Hill put some 80 of us onto buses and drove us to dinner at the Flagstaff House, a very expensive restaurant in the mountains behind the conference site in Boulder, Colorado. Some years later, I was a speaker at the conference, along with Robert Huelskamp, one of the co-authors of the Sandia Report. It was Huelskamp's first time at the conference, and he was both amazed and appalled at the cozy relationship that existed between those who produced tests and those who bought them. Then, it was mostly the students who were at risk. Now, it's everyone. Principals can be sacked or transferred. Superintendents have bonus clauses for increases in test scores. Teachers can be dismissed. Whole systems can lose the power to govern themselves. In California, Gov. Gray Davis announced that he would not run again unless test scores went up. When the reading scores of students with limited proficiency in English showed double-digit improvements, my e-mail box was deluged with flashes from the phonics folks and from Ron Unz, the sponsor of California's anti-bilingual-education Proposition 227. Unz boasted that the results were "the most rapidly successful education reform program in history." The electrons stopped sizzling, though, when it was announced that the large gains were largely the result of 300,000 misclassified students; youngsters who were fluent in English had inadvertently been put into the limited-proficiency category. As this is written, reaction and resistance to the growing madness are weak and sporadic but perhaps building. Students in a few states have started refusing to take the tests, teachers have started refusing to give them, and parents have started refusing to send their children to school on testing day. In Virginia, SOL is supposed to stand for Standards of Learning. But after the 98% failure rate, some people started calling it "Standards of Lunacy." A number of slogans have been formulated: ERASE the CASE (Chicago), TOSS the TAAS (Texas), BLEEP the MEAP (Michigan), and so forth. In Chicago, teacher George Schmidt published the questions of that city's test and was promptly fired. Schmidt is suing. Chicago has established "transition centers" for students who score low on the test. Schmidt claims that these are actually "gulags" into which the students are placed in order to remove them from the database so scores will continue to rise. Paul Vallas, Chicago's superintendent of schools, has had a standing fight with local magazine Catalyst over whether or not test scores are the sole criterion for promotion and retention. Vallas says there are multiple criteria. Most newspaper stories, though, give the distinct impression that only the test scores count. Interestingly, neither Riverside Publishing nor any of the national organizations responsible for appropriate test use have shown much interest in fingering Chicago for inappropriate test use. Some highly predictable events have occurred amid the mania. In Texas, an assistant superintendent was fired for tampering with tests. Noting that the offense was only a misdemeanor, some Texas legislators proposed making it a felony. A similar occurrence took place in Arizona. In Henrico County, Virginia, a principal resigned for the same reason. Texans noticed that the proportion of students passing the TAAS (Texas Assessment of Academic Skills) increased, but so did the number of students excluded from the testing. The states showing the greatest gains in reading on the National Assessment of Educational Progress (NAEP) from 1994 to 1998 also had the biggest increases in excluded students. New York City's mayor, Rudolph Giuliani, campaigned on a gain of three percentile ranks in reading scores - but got clobbered when reading scores fell the next year. "Let me guess," Walter Haney, a Boston College professor and testing expert, told a New York Times reporter. "They changed the test." Indeed, they had changed the test. The results reminded the Times reporter, Anemona Hartocollis, about an earlier episode of madness. During that one, John Jacob Cannell, a pediatrician in Beckley, West Virginia, noticed that test scores in his impoverished area were above average. At the time, Cannell said that he asked himself, "If we're above average, who the hell is below?" Collecting data as best he could, Cannell discovered that the answer was essentially no one. Although Cannell had made some technical errors, Robert Linn's methodologically sound studies pretty much confirmed his conclusion. It became known as testing's Lake Wobegon Effect, named for Garrison Keillor's mythical Minnesota town where "all the children are above average." Tests and Accountability In awarding a Golden Apple to Richard Rothstein this year (see page 166), I note that Rothstein's accountability system requires a district to spell out in great detail the outcomes that the district thinks are important and then to develop a composite index based on all those indicators. The principal indicator used for accountability is change in the composite index. This prevents districts from focusing on one indicator in order to look good. But most places are not yet using a composite index. More are using techniques like the Tennessee Value-Added Accountability System (TVAAS). By testing every student every year, TVAAS is able to track progress on the norm-referenced, multiple-choice items provided by CTB/McGraw-Hill. The principal developer, William Sanders of the University of Tennessee, has found that both schools and individual teachers can be categorized in terms of their effectiveness in producing increases or decreases in test scores. Sanders and his system are "hot," meriting a special feature in Education Week, headlined "Sanders 101."1 Sanders divides teachers into five categories of effectiveness according to how much they change test scores. He has found that the impact of test-effective third-grade teachers was still visible in fifth-grade students, independent of the test effectiveness of the fifth-grade teacher. Perhaps more interesting is the finding that the cumulative impact of a sequence of highly test-effective or highly test-ineffective teachers is large. Sanders examined an average district and a high-scoring district and found that students who had test-effective teachers three years in a row scored at the 83rd and 96th percentiles respectively; students who had test-ineffective teachers three years in a row scored at the 29th and 44th percentiles.2 Sanders does not provide information on where the students started from, so we don't know how much gain or how much loss was involved. We do know that, in the two systems, students who had three "average" teachers in a row scored at the 50th and 80th percentiles. At one point Sanders does say that groups were statistically equalized such that the various groups all had "an even start." We also do not know anything about tracking practices in the two systems, nor do we know how many students were in any of the groups. Offhand, a cluster of students in the 96th percentile seems a bit unlikely, since the 95th percentile is a common criterion for inclusion in gifted and talented programs. Note that at one level the system is circular. Effective teachers are defined as those who improve test scores. It would thus be a bit surprising if three such teachers in a row did not produce changes for the better in test scores. The important research has not been done: research that would tell us why or how some teachers change test scores while others don't. It would also provide a description of what test-ineffective teachers are actually doing. We might - or might not - be impressed with "effective" teachers. One might wonder whether parents, principals, or other teachers in the system, if asked to name "good" teachers, would come up with the same list. Without collateral evidence, we can't know the dynamics, and, given the kinds of activities that often produce gains in test scores, we might not want those teachers or activities in the system. Indeed, an indifference to test scores might be a better sign of good pedagogy. Consider, for instance, the Key School in Indianapolis, a magnet school based largely on Howard Gardner's theory of multiple intelligences. To develop the various intelligences, the Key School teaches all students a foreign language and a musical instrument. It provides daily art lessons and activities to stimulate inter- and intra-personal intelligence. Outside of Gardner's theory but in keeping with wise practice, the teachers spend each Wednesday afternoon in a planning and evaluation meeting. Parent volunteers take the students to the gymnasium/auditorium, where some group from the community explains what it does in the real world. Once when I was there, nurses and paramedics explained their jobs and the skills they needed. On another occasion, a quartet from the Indianapolis symphony talked about their instruments and their music and were blown away by the sophistication of the students' questions. But recall that, in most systems, instrumental music education does not begin before fourth or fifth grade and then only for a small number of students. In 1991, after the Key School had been in operation for four years, NEA Today put it on the cover with the question "Is This the Best Elementary School in the Country?" The Key School has also been the subject of adoring coverage by the television networks and PBS. Perhaps the best testimonial came from the school's graduates, who returned to the school to tell the teachers that they were "dying" in traditional middle schools. (The middle grades were added to Key School later.) Yet when the state of Indiana put a test-based accountability system in place, Key School teachers feared for their accredited lives. Key School test scores are not that impressive (admission to the school is by lottery, with the proviso that the ethnic makeup must match that of the district as a whole). Teaching a foreign language or music or art will not do anything to raise percentile ranks on tests of academic achievement. The languages of these disciplines are too specialized to be used in such tests. Standardized tests, especially in the lower grades, can use only those words that most students have been exposed to. Otherwise the items will not "behave" properly - behavior being defined here in statistical terms. Uncommon words might show up on the SAT, and knowing Spanish, the Key School's taught language, might help a student taking the SAT decipher an otherwise inscrutable word with a Latin root. But nothing about these skills helps scores on the Iowa Tests of Basic Skills (ITBS) or on any other elementary achievement tests. The Key School's dilemma: Should it abandon what the faculty considered good pedagogy to raise test scores, or should the teachers stick by their pedagogical guns and risk loss of accreditation? The Key School solved its dilemma by creating drills and worksheets covering the kinds of skills that do affect test scores and by cajoling parents into completing them with their children at home. Scores went up so much that the state department of education sent someone over to see if they were cheating. Sanders and other advocates for the TVAAS have argued that tests tap higher-order skills, relying on a couple of sentences that I certainly do wish Blaine Worthen and Vicki Spandel had never written: "The notion that multiple-choice tests can tap only recall is a myth. In fact, the best multiple-choice items can - and do - measure and evaluate ideas, products, or performances."3 People pushing standardized tests love these sentences, and they turn up in lots of essays that defend the tests. The problem with the statement is simply that it doesn't specify where one finds "the best multiple-choice items." A few years ago, I phoned Worthen to find out. His response was immediate: "They occur on the tests I give in my graduate statistics and research methods classes." Ahhh. I recall those kinds of items from my own graduate school days. The stem of the item would cover almost an entire page, and about four questions would be based on that stem. The whole two-hour test might contain only six to eight items. To answer correctly, you had to be thoroughly versed in, say, the differences between Hull's and Tolman's learning theories (meaning, of course, that you had to know both theories well in order to derive the differences) or in the assumptions that must be met for certain statistical analyses. These are the items Worthen and Spandel were talking about. Their comments do not apply to commercial achievement tests or even to items on the SAT. After all, on that test, students must answer a question a minute in the verbal section and one every 90 seconds in the math section. No doubt that is why the College Board's advice includes the directive "Keep moving." A student who thinks very much about the questions will be in trouble. But "moving" is antithetical to "thinking." For this reason, the questions look like these, taken from actual SATs: Rib Cage:Lung :: a) skull:brain, b) appendix:organ, c) sock:foot, d) skeleton:body, e) hair:scalp * * * If the product of five integers is negative, at most how many of the five can be negative? a) 1, b) 2, c) 3, d) 4, e) 5 Note that these are items from a college admissions test. Those from elementary and secondary achievement tests do not even reach this level of complexity. (The Iowa Tests of Educational Development in grades 9-12 are notable exceptions.) Thus a system that rests on items like those the TVAAS uses cannot measure much in the way of higher-order thinking skills. It would also appear to dampen curricular and instructional creativity. For a description of what higher-order thinking skills really look like, see Lauren Resnick's little book, Education and Learning to Think.4 There are several other interesting findings from Sanders' work. First, the gains he found appear to be independent of either ethnicity or socioeconomic status (SES). Sounds good, but from one perspective the lack of correlation is bad news. It means that the black/white score gap will not be narrowed, much less eliminated. We need to find a negative correlation between ethnicity or SES and change. That would mean that low-income or minority children were gaining. Second, the residual effect after a student has had a test-ineffective teacher can be seen two grades later. Sanders claims that there is little evidence for a compensatory effect if an ineffective teacher is followed by an effective teacher. That is, although a test-effective teacher can produce high gains in those students who previously had a test-ineffective teacher, she cannot make up the lost ground. One final point: the system is certainly a good jobs program for test publishers. It requires that all students be tested every year. Tennessee tests in five areas: reading, language arts, mathematics, science, and social studies. Indeed, it does seem necessary to test all areas lest teachers emphasize only those areas tested. One wonders, too, whether the same teachers who are test-effective in reading are test-effective in, say, science. It will be interesting to see what future research yields. Public Schools Compared To Private by Case Study It has been a banner period for Catholic schools in the public relations department. Investor's Business Daily hailed "The Magic of Catholic Schools." Some websites revived Diane Ravitch's Forbes article from three years back, "Why Do Catholic Schools Succeed?" And Nina Shokraii of the Heritage Foundation explained "Why Catholic Schools Spell Success for America's Inner-City Children."5 A study by Richard Rothstein, Martin Carnoy, and Luis Benveniste and an examination of NAEP data by type of school seriously call these hoorahs into question.6 Rothstein of the Economic Policy Institute, Carnoy of Stanford University, and Benveniste of the World Bank set out to conduct case studies of various kinds of schools - public, Catholic, and other private. They sought to explore six common hypotheses. * Private elementary school personnel are more accountable to parents than are public school personnel. * Private schools have more clearly defined expectations and outputs than public schools. * Private elementary schools teach good behavior and values better than public elementary schools. * Private schools' teacher selection and retention practices are more efficient than those of public schools. * Private schools achieve academic success with curricular materials (in standard subjects) that differ from curricular materials found in public schools. * Private school innovations stimulate improved practices at the public schools with which they compete. The title of their study is "What Can Public Schools Learn from the Private Non-Profit Education Sector?" Their answer is "Not much." But they go on to say that, "if phrased somewhat differently, Owhat can schools, public and private, learn from other schools, public and private,' our answer is Oquite a bit,' depending on which schools we study and what we are attempting to learn." Consider the accountability hypothesis. In this instance community income characteristics overwhelmed governance. Low-income schools - both public and private - often complained that they had trouble getting the parents involved. When parents did become involved, it was more likely to be with regard to dissatisfaction with a child's grade or a disciplinary action taken against the child than about some issue of curriculum or instruction. Low-income private schools succeeded more than low-income public schools because they could make parent involvement a condition for admission. Most such involvement, however, was nonacademic. Low- and lower-middle-income schools often attempted to get the parents more involved, to get them to take more responsibility for their children's academic proficiency and nonacademic behavior. In effect, these schools were trying to hold the parents accountable. At high-income schools, the hypothesis was not only not confirmed, it was turned on its head. One teacher in an affluent public school said that she typically receives one letter a week from the parent of every child with questions or suggestions about her teaching. Another said that, on the first day of school, 25 parents dropped in with specific suggestions on how to organize the curriculum. In these schools, many parents were highly educated professionals who felt they had a right to participate in their children's education. At one public school, the researchers found that "parental questioning of school and classroom management has become so extensive that the school established an ombudsman team that organized and researched parent complaints and published summaries in a weekly school bulletin." This isn't the way free-market theory says it's supposed to work. Because the "government schools" are a monopoly, they can afford to be unresponsive. Private schools, which must compete for students, are supposed to be more accountable. Affluent private schools, by contrast, were more successful at advising parents that curricular and pedagogical decisions were the exclusive prerogative of the school's professional staff. In fact, in one private school, accountability was limited to a simple piece of advice: caveat emptor. "The proprietors have maintained a clear policy of discouraging parental involvement to prevent interference in the school's operation. Until recently, and for most of the school's 28-year existence, there has been no parent association or other parent advisory group and no invitations for parents to assist in classrooms." As for clarity of goals and expectations, reality again proved more complex than theory. A common finding was that there were "multiple clarities" in both public and private schools. A faculty might be quite clear about the way it thought the school should proceed, but the central administration might have equally clear - and contradictory - ideas. In religious schools, religious and secular goals often clashed. One principal in a private school with twice as many applicants as seats wanted to emphasize academics in the admissions process. The parish priest, on the other hand, pressured the principal to admit low-scoring students whose parents were parishioners, saying, "We're not a successful school if our students get into Harvard but in the process drop the Catholic church. The principal reason for the school is to hand down the Catholic faith." Similarly, in one school, the faculty generally disdained the kind of skills that standardized tests measure. Their math curriculum was built around the standards from the National Council of Teachers of Mathematics. On the other hand, the school administered the CTBS (Comprehensive Tests of Basic Skills), which emphasize computation. Teachers reported that, while their goals were clear, they had to abandon them to some extent in order to prepare students for the tests. And so on. None of the six hypotheses was confirmed. In general, the results look more like those predicted by economist Byron Brown than those predicted by economist Milton Friedman. According to Rothstein, Carnoy, and Benveniste, Brown argues that schools must tend to look alike because of the uncertainty of the post-school years. Neither the school - be it public or private - nor the parents can know for sure what the future will bring for the children in terms of employment prospects. Therefore, the schools offer similar curricula and teach in similar ways. A school that is truly a "break-the-mold school, that deviates too much from the general perception of what a school is, increases the risk that the parents have made the wrong choice by sending their kids there." This kind of anxiety, of course, was precisely the motivation for the Eight-Year Study that commenced in 1932. Progressive schools wanted to "break the mold" but were afraid innovations would dampen their students' chances for college, since the admissions process was based on traditional criteria. The Progressive Education Association convinced colleges to give up their usual admissions requirements for graduates of the "break-the-mold" schools that the PEA had in mind. When they got to college, students from these schools did much better than students from traditional schools. However, it is easy to understand how individual schools, without the assurance afforded the PEA by the cooperating universities, might be anxious about deviating too far from tradition. Indeed, the researchers found that the schools that offered the most original curricula operated in an environment of affluent and/or well-educated parents. Such parents were least uncertain about the future of their children. In closing, the researchers mention Anthony Bryk, Valerie Lee, and Peter Holland's Catholic Schools and the Common Good, observing similarities and differences in what the authors of that book found in their study of Catholic schools. Although they don't use this terminology, Rothstein, Carnoy, and Benveniste note that, by limiting their study to Catholic schools, Bryk, Lee, and Holland committed the "base rates fallacy" of not looking at the whole population. Bryk, Lee, and Holland claimed that Catholic schools participate more than other schools in a "voluntary community" where parents "make life easier for the schools by insuring that students attend regularly, do their homework, and adhere to the school's behavioral standards."7 If Bryk, Lee, and Holland had looked at all kinds of schools, they might have seen what Rothstein, Carnoy, and Benveniste report: There were certainly parents with whom we spoke, and about whom the faculty we interviewed spoke, where such characterizations applied. But we did not observe these to be particularly more frequent in private or in Catholic schools as a whole. Rather, in this respect as in so many others, the social, cultural, and economic backgrounds of the parents and the community in which the school was located seemed to be the main determinant of variation, much more so than a school's public or private character or, within the latter group, whether it was religious or secular. Within particular communities, the similarities between schools and the problems they confronted overwhelmed the differences.8 Public Schools Compared To Private by NAEP Earlier studies showed that, when parental income and education are factored out, public and private schools do not differ on NAEP scores. People who use private schools tend to be wealthier and better educated than the populace at large. When these advantages are statistically removed, public and private schools look very similar. In the most recent NAEP report of science achievement (this analysis was not conducted for reading and math), such removal is not needed.9 This report shows a most interesting outcome. The data of interest are shown in Table 1. The largest differences occur at the lowest grade and in the lowest ranks. The differences between public and private schools become smaller as one moves up the performance scale. The difference is largest (23 points) at the 10th percentile for fourth-graders. By the time one gets to the 90th percentile, the difference has dropped to only nine points. It is reasonable to speculate that the low-end differences are the result of the selection of students at private schools. The differences also diminish as one goes up the grade ladder. At the fourth grade, the difference at the 10th percentile is 23 points, but by the 12th grade it is only 12 points. At the fourth grade, there is a 14-point difference at the 50th percentile, but by 12th grade it has shrunk to seven points. By the time one reaches the 12th grade and the 75th and 90th percentiles, the differences between public and private schools have disappeared. The NAEP mathematics and reading assessments do not give the various percentile ranks, just average (i.e., median) scores. The average score difference between public and private schools falls from 15 points at grade 4 to 11 points at grade 12. In reading, the gap actually increases slightly between fourth and eighth grades, from 18 points to 20 points; then it falls back to 14 points at grade 12. These changes are small, but consistent: they occurred in the 1994 and 1992 assessments as well. NAEP Trends and Simpson's Paradox Simpson's Paradox first came to the attention of many educational researchers via the Sandia Report. That report showed that the overall average SAT score fell between 1975 and 1990. Yet the scores for white students were stable, and those for black, Hispanic, and Asian students rose. That's the paradox: How can the scores for the subgroups rise and the scores for the total group fall? The answer is simple. At the two times when the measurements were taken, the subgroups constituted different proportions of the total group. Simpson's Paradox is likely to be relevant in any national comparison of test scores over time, and it can be relevant to state, district, or even school comparisons when the ethnic mix of the students is changing. Here's an oversimplified example. Assume that at Time 1 white students make up 90% of the population, and the average SAT score of white students is 500, while the average SAT score of minority students is 400. The overall average is 490. Assume that at Time 2, the average SAT score of white students is 510, and the average score of minority students is 430. However, white students represent just 70% of the population at Time 2. White students' scores have improved by 10 points, and minority students' scores have gained 30 points. Yet the overall average is lower - just 486. The scores of minorities are improving, but they are still relatively low. When they constitute a larger portion of the total, they have the effect of depressing the overall average, thus masking the good news that scores for everyone are rising. This is what we find looking at NAEP trends (see Table 2). In the interest of saving space, I show only the highest, median, and lowest ranks and only for science. Interested readers can find more detailed summaries in NAEP 1996 Trends of Academic Progress.10 Reading and math results closely resemble those for science. The overall trends closely mimic trends for white students because white students constitute by far the largest group in the sample. For example, the score for 13-year-old white students at the 95th percentile shows no change, and neither does the 95th percentile score for all students. Yet the 95th percentile score rises 10 points for black students and 11 points for Hispanic students. Similarly, the average score for 9-year-old white students rises by 10 points, and the overall average rises by nine points. But the increase for Hispanic students is 17 points and for blacks, 28 points. Critics are likely to say that, while the scores of minorities may be up, scores are still awful for everyone. In science, for example, only 18% of fourth-graders scored at the proficient level, and only 2% scored at the advanced level. (Grade levels are used for NAEP cross-sectional studies; ages are used for trend analyses.) A proper rejoinder to this contention is to point out that NAEP proficiency levels have met with scorn and derision from the scholarly community. Studies have repeatedly found the proficiency levels misleading, and they are not corroborated by other data. For instance, those same fourth-graders who showed so poorly on the NAEP proficiency levels finished third among 26 nations in TIMSS. Similar laments were heard about young American readers, and they finished second in the most recent international comparison of reading. In the spring of 1999, the National Research Council's Committee on the Evaluation of National and State Assessments of Educational Progress noted that "the current process for setting NAEP achievement levels is fundamentally flawed. . . . [It] should be replaced."11 Vouchers: Chile Chills Out It has been observed repeatedly that, even if the voucher programs had produced gains in achievement in places like Milwaukee and Cleveland, these are such hothouse experiments that the results could not be generalized to any larger state or national program. It appears that we are about to get our first large-scale voucher system with the passage of legislation in Florida offering vouchers to children attending "failing" schools, failure being determined by the state. It is not clear at this writing how large the program will become or how it will be evaluated. Private schools, for instance, do not have to test the public school students they receive. Still, the program is something to watch. (For the state of this program as of midsummer, see the report by Stanley Elam in the September 1999 Kappan.) In the meantime, the failure of vouchers in Chile offers some sobering data for advocates of unfettered choice. As one of the researchers who evaluated the program said, "Markets should work the same way in Chile as in the United States."12 In 1980, Chile's military government transferred control of the nation's schools from the Ministry of Education to localities. At the same time, drawing on the writings of Milton Friedman, it established a method of funding schools directly tied to enrollment. Private schools were eligible to receive the payments if they did not charge tuition. By 1998, almost 40% of enrollments were in voucher schools. During this period, the proportion of students in elite tuition-charging schools varied between 5% and 9%. Most of the migration was from public schools to new private voucher schools. Chile tests students in the fourth grade in even years and in the eighth grade in odd years. The researchers, Patrick McEwan and Martin Carnoy, concentrated on fourth-grade data. Then, using two sources that collect demographic information, McEwan and Carnoy were able to gather data about the educational levels and socioeconomic status of the parents who were sending their children to particular schools. Friedman is not going to like the results. Vouchers in Chile have not produced the educational results proponents claim for them nor what the poor might have expected. For example, our analysis of nationwide school test score results from 1983 to 1997 shows that pupils in most of Chile's private subsidized (voucher) schools do not outperform public school pupils once socio-economic background differences are accounted for. Instead, vouchers redistribute pupils with better-educated parents from public to private schools. Nor has competition made public schools better. School improvement in Chile came mainly from a centralized, reformist bureaucracy intent on improving the least successful schools, not from free-market competition.13 The scores of public schools have improved, an outcome that, at first glance, appears in line with free-market theory. Unfortunately for the theory, the scores rose more in municipalities that had only public schools. That is, scores rose most where there was no competition. Why do families choose private schools? It appears that better-educated, more affluent parents want their children to attend school with other children from better-educated, more affluent families. Even though public schools have smaller classes than the new voucher schools, parents appear to want their children to go to school with the "right" kind of classmates. Thus, as predicted by voucher critics, the private schools have drawn away - skimmed off - the wealthier students from better-educated families. Principals at private voucher schools also like to emphasize their schools' ability to get their graduates into better secondary schools and low-cost public universities. Competition to enter both is fierce. A Dispatch from The Privatization Front "There is a vast left-wing conspiracy to deprive poor children of a good education. Rarely has such a grand army as our own held the high moral ground for so long and advanced so little." Thus spake Lamar Alexander, former secretary of education and Presidential candidate, at a September 1998 conference - no, revival meeting - on the wonders of vouchers and privatization. It was at this conference, held too late for last year's Bracey Report, that Arizona State Superintendent Lisa Graham Keegan earned her "Benedict Arnold Award" (see page 155), but her performance was lost among the dazzling hyperbole of the two-day event. William Bennett keynoted after the opening dinner (chicken, drinks at $6.50 a pop on top of the $249 registration fee). Bennett claimed that there was a school in Chicago with a 100% dropout rate and that 50% of Chicago schoolchildren finished in the 1st percentile on achievement tests. The author of The Book of Virtues set the tone for the conference to follow: facts would be in short supply for the duration. "The Berlin Wall of education monopoly" was a phrase much heard during the two-day conference. This referred to the public schools in general (usually called "government schools" or "socialist schools") and to the National Education Association in particular. "People OAgainst' the American Way" was also a frequent target of animus. The NEA was seen as analogous to the phone company before the breakup. NEA wishes. The conference got far enough afield to discuss "paycheck protection" legislation, an anti-union tactic much in favor these days with those on the Right. I call the conference a revival meeting because of all the emotional (and usually irrelevant) testimonials and staged applause, but the religious aspects went beyond such Protestant events. No doubt the Pope himself noticed a mysterious warming in his heart during this period. Much of the conference was a paean to Catholic schools. Former University of Oklahoma quarterback and current U.S. Rep. J. C. Watts (R-Okla.) declared that Catholic schools will "unleash the caged eagles in the inner cities. Catholic schools are taking the worst and making them the best." William Leininger of CEO America discussed that foundation's plan to provide vouchers to the Edgewood, Texas, school district in order to bring market forces to bear on Edgewood, an overwhelmingly poor, Hispanic area of San Antonio. An Edgewood mom brought in for a testimonial gave evidence that forces other than the market were at work. "I so, so wanted a religious education for my son," she told the group. Leininger declared that Edgewood had responded to the challenge by shaping up. It was noted from the floor, by Mike Antonucci of the Education Intelligence Agency, that our conference packet contained an article indicating that Edgewood had shaped up before CEO America announced its scholarship program: "[Edgewood] opened magnet schools for math and technology and started its first-ever advanced placement classes. Elementary schools were rebuilt, their teaching revised. A high school for troubled youngsters was started." Edgewood nevertheless lags behind the state average, the article admitted, "but 21U2 times as many eighth-graders passed their math exams last year as in 1993. Dropout rates have been cut in half, SATs are up 134 points." The article went on to observe that the Catholic schools had shown no interest in the 12% of the district's students who qualify for special education. Nor had the state shown any interest in the quality of the private schools involved in the CEO America program. The article also questioned the quality of the schools CEO America was supporting. One such school was The Sword of the Lord School. This school had no phone, and at its stated address "there is only a ramshackle cottage with trash cans in the front, old cars in the back, and a sticker reading OProperty of Jesus Christ' across the door." Given that this article appeared in the Wall Street Journal, it seems unlikely that it was part of Lamar Alexander's "vast left-wing conspiracy."14 Dick DeVos, president of Amway, lamented the terrible effect our education system was having on the work force. At the time, productivity in the U.S. was growing at a mere 3% a year. Currently it is closer to 4%. Apparently, DeVos hadn't noticed. As I said, the conference proceedings were largely unimpeded by facts. DeVos also provided a testimonial letter from a mother who declared that she had had a terrible education. Her teachers didn't care about her and looked down on her. She quit school after ninth grade and was so happy that her son would attend a Catholic school. The elegance, clarity of tone, and error-free prose of the letter would put to shame most letters that appear in such periodicals of ideas as the Atlantic, the Nation, or the National Review. DeVos did not bother to explain how someone with such a short, terrible education had learned to write so beautifully. Another testimonial was offered by a Washington, D.C., mom, Virginia Walden, who declared that sending her child to a Catholic school with a voucher "worked miracles on him." Walden works for Jeanne Allen's ultra-right-wing Center for Education Reform. In the realm of testimonials, no one topped Bret Schundler, mayor of Jersey City, New Jersey, who regaled the audience with anecdotes. He wrapped up with one about how a woman, dying of cancer, terminated her chemotherapy and left the hospital to come talk with him. Not only was she terminally ill, but her husband had just died of a heart attack, and, while the diocese had waived some fees, she was still short of the amount necessary to cover her share of the tuition at her son's Catholic school. Schundler wrote a check then and there. I have no idea what this story was supposed to prove. Away from the testimonial beat, Creators Syndicate columnist Mona Charen didn't plump for vouchers. Instead, she settled for deriding the current education system. She emphasized repeatedly that she lives in Fairfax County, whose school system is reputedly among the best in the nation. "But did you know," she asked the audience, "that students in Fairfax attend school only 41U2 days a week? The teachers' union said they needed that other half-day for teacher training. One would have thought," Charen sniffed, "that they were already trained when they were hired." From this Fairfax farce, she said, we can see that standards have dropped. Don't you wonder if she applies the same standards for training to her doctor and dentist? I belabor the proceedings of this conference to show what is happening in the name of choice and privatization. It might seem loony, and it might be that there were more non sequiturs emitted during this conference than in any other similar time period. But the looniness is well organized and extremely well funded, and it is therefore dangerous. The people involved have clout and access to the media. In addition to those already named, the speakers included former U.S. Attorney General Edwin Meese, former Indiana Sen. Dan Coats, and Reader's Digest editor Michael Barone. The only real cerebral note at the conference was sounded by Clint Bolick, president of the Institute for Justice. Bolick had just come from successfully arguing before the Wisconsin Supreme Court that church-related schools should be eligible for the vouchers in the Milwaukee program. He said that it was the only time in his experience that both the plaintiffs and the defendants had asked the U.S. Supreme Court to hear a case. The U.S. Supreme Court later refused. Bolick argued that school choice was the civil rights issue of the moment. He maintained that it was impossible for a Supreme Court not to apply the same logic to choice as it did to segregation in Brown v. Board of Education. The issue of choice is all about the expansion of educational opportunity. Of course, with court-ordered busing being terminated in many cities these days, it is not certain that today's High Court would reach the same conclusion as the Warren Court did in 1954. The Class-Size Controversy In discussions of vouchers, an important point is often overlooked by voucher advocates: even if vouchers work in the small-scale experiments conducted to date (and that is by no means clear), these results cannot be generalized to larger-scale programs, because so many additional forces would come into play. This applies to any experiment writ large and can be seen in the large-scale experiment in class-size reduction (CSR) now in progress in California. Faced with low achievement, the causes of which are very much in debate, and with classes averaging about 30 students, California invested $1.5 billion a year to get class size down to no more than 20 in the primary grades. The inspiration for the project, of course, was Tennessee's Project STAR. But from the outset there were significant differences between STAR and the California program. First, while students were randomly assigned within buildings to each of the three STAR treatments, the schools themselves volunteered for the program, and only about 10,000 students were involved. This probably meant that few buildings suffered disruptions. From the outset, though, stories came out of California alleging that classes were being held in closets and rest rooms because of the lack of space. These problems and other outcomes, including some positive outcomes, have been reported by the California Class Size Reduction Consortium, a group of six research- and policy-oriented think tanks. The Consortium found that the program had been put in place with remarkable speed - so fast, in fact, that almost 25% of the parents had not heard about it even at the end of the second year. Space was already at a premium in California schools because of rising enrollments. Districts received a flat rate of $650 for each child in a small class in the first year of the program, $800 the second year. Ironically, but not surprisingly, those with the least available space and those with larger classes - namely, schools in poor neighborhoods - were least likely to move fast on the program. This means that poor schools received proportionately less money to implement the program and, at the same time, needed more money because they needed more teachers. To meet the mandate, these schools pulled money away from facilities maintenance, administrative services, libraries, computer programs, and, to a lesser extent, from arts, professional development, special education, and child care. With smaller classes, more teachers were required, and the teacher force grew by 23,500 people in the first two years. The number of not fully credentialed teachers soared from 1% to 12%; those with only a bachelor's degree increased from 17% to 23%. The proportion of novice teachers increased from 16% of the teaching force in 1995-96 to 28% in 1997-98. Naturally, these changes did not affect all schools equally. Some teachers jumped from inner-city to suburban districts, and the poor districts were less able to compete for the new teachers entering the work force. The quartile of districts with the lowest poverty rates saw the proportion of uncredentialed teachers rise from virtually zero to 5%. However, in the quartile with the highest poverty rates (30% or more low-income students), the uncredentialed rate zoomed from 2% to 20%. In spite of all these problems, test scores rose, though not by much. The effect sizes ranged from +0.05 to +0.10. Is this cause for joy or disappointment? Given the speed and difficulty of implementing the program, a conclusion probably depends more on the initial bias of the person rendering the judgment than on anything else. Unfortunately, the rep ort does not break out results by the SES of schools. One might predict that effect sizes would be large in the suburbs, possibly negative in the inner cities. The Consortium urges "caution in making too strong a judgment about the effect of CSR on achievement at this early point in time. No one has ever implemented a CSR reform on this scale before, and it is difficult to establish criteria for success at this juncture."15 Stay tuned. Elsewhere, the Wisconsin Student Achievement Guarantees in Education (SAGE) program reduced student/teacher ratios to 15 to 1 in 30 schools. SAGE has been in operation for two years now. According to Alex Molnar of the University of Wisconsin, Milwaukee, the results mimic those of Project STAR: students in small classes gain more and sustain those gains.16 In the small classes, the black/ white test score gap was reduced by 19%, while in comparison schools, it grew by 58%. In a new wrinkle, some of the classes contained 30 pupils but two teachers. Outcomes for these classes did not differ significantly from those for students in a true 15-to-1 classroom. This suggests that the benefits of smaller classes can be obtained without the huge need for separate space experienced in California. Finally, Cecilia Rouse of Princeton University has noted that the voucher students in Milwaukee apparently had smaller classes than any of the other groups studied in Milwaukee.17 Thus, if the voucher group shows any improvement - and it's still "if" - then the benefit could stem at least partly from the small classes, not from some marketplace miracle of vouchers. We should note that Rouse was able to replicate Paul Peterson's finding of a gain in math but not in reading. She also found the positive outcome for vouchers to be ephemeral, disappearing when she used scores for the same students for two or more years. (The summer 1999 issue of Educational Evaluation and Policy Analysis devotes all 248 pages to the class-size controversy. It arrived too late for inclusion in this report and will be described in a future Research column.) Charters For many people, "charter" equals "hope." That faith is reflected in the subtitle of Joe Nathan's 1996 book, Charter Schools: Creating Hope and Opportunity for American Education.18 Others have been less sanguine. Chester Finn, Jr., former assistant secretary of education, although a charter advocate, lamented in the same year that no states had systems in place to evaluate the achievement of charters.19 While Finn allowed that program evaluation systems were often weak, he stressed that such evaluation was critically important in the case of charters because they promise to raise achievement in return for increased freedom from rules and regulations. Others who have looked at the inadequacy of evaluation were even more skeptical. In Giving Kids the Business, Alex Molnar wrote, "Charter schools will fail, fraud will be uncovered, and tax dollars will be wasted. But just as certainly glowing testimony will be paid to the dedication of selfless teachers and administrators of some OChartermetoo' school who transformed the lives of their students and proved the success of charter school reform."20 It looks as if both Finn and Molnar were right. Venality has turned up most obviously in Arizona and Texas. Arizona's charter laws were written by the current state superintendent (see "The Benedict Arnold Award," page 155), and she essentially shut down her evaluation team rather than find fault. The charter grantors in Texas were pressured by staff from Gov. George W. Bush's office to grant charters. A set of articles by Stuart Eskenazi in the Houston Press in July 1999 described the problems to date.21 One charter school's supposed CEO was listed as such because her name had been forged on the application. Another charter inhabited a decrepit, two-story gray stucco office building that sat woefully along a busy commercial street. The City of Arlington [Texas] had declared the second level of the vacant building unsuitable for habitation, so the school set up shop in the two large rooms on the ground floor. The building had no heat. The classrooms had no desks, no chairs, no textbooks, no chalkboards, no trash cans, no gymnasium, no lunchroom, no vending machines, no functioning toilet. It was called the Renaissance Charter School. Things came to light when frustrated teachers gave students an assignment (which they could write only if they had brought paper and pencil from home): "Write a letter to the Texas Education Agency describing conditions here." Wrote one student, "If you name it, we don't have it." Until the letters arrived, state officials were under the impression that Renaissance was a model school. Eskenazi observes that "[Gov.] Bush led an embarrassingly large entourage of reporters and photographers inside a Massachusetts charter school last month. And two weeks later, he visited a Los Angeles charter campus to extol the virtues of those schools. Texans, however, might have been better off if the governor had been paying more attention to charter schools that were failing in his own state." Outside of the realm of actual malfeasance, five evaluations in two of the most active charter states, California and Michigan, found that parents and teachers both reported high satisfaction with their schools but that test scores or other indices were either not available or were unimpressive.22 In some cases, the absence of test data makes perfect sense. As with the Key School mentioned above, some charter schools are engaged in activities for which standardized tests make little sense. On the other hand, one wonders why those proposing the school and those proposing the charter did not work together to establish what would be appropriate as evidence of success or lack thereof. There appears to have been a lot of naivet? on all sides. Some people with their hearts in the right place who wanted to help students discovered that running a school was very hard work and called for a lot of skills they didn't have, such as creating and monitoring budgets and handling the varying demands of differing clienteles. This problem has led to a disturbing trend, at least in Michigan. (It might be a problem elsewhere, too, but only the Michigan report addresses the issue.) One school founder described the operation of a school, even her very small school, as overwhelming. "It was a real burnout situation," she said, so she turned the keys over to a private, for-profit company.23 This is disturbing because, while what one Michigan evaluation called "mom-and-pop" charter schools often offered education based on their unique visions, the private firms create "cookie-cutter" schools: they are franchises with prepackaged, rigid curricula. In one year, the proportion of cookie-cutter schools rose from 50% to 70%. Most of the reports come up short on evidence of outcomes that could be used for accountability purposes. Amy Stuart Wells, in the UCLA case study of some California charters, concludes, "It is not clear that charter schools are being held responsible for anything more than the public schools are held to, although in areas where they have more control over resources, they sometimes have to report their use of resources in more detail." Translation: charter schools lose their charters because they mismanage money, not because they fail to educate students. The WestEd report on Los Angeles charters does present specific information about achievement. WestEd examined the percentage of students below the 50th percentile at the start of the charter who moved above the 50th percentile at the end of one and two years of charter operation. It also looked at the percentage of students who started out above the 50th percentile and remained there after one and two years. Across CTBS scores on reading, mathematics, and language arts, charter school students were more likely to move from below the 50th percentile to above it than were students in comparison schools. For the most part, they were also more likely to remain above the 50th percentile if they started there. However, WestEd does not take note of the fact that a higher proportion of charter school students were below the 50th percentile at the end of two years than at the end of one. The comparison schools showed this pattern - but it was less pronounced than in the charter schools - in reading and language arts, and the proportion of students below the 50th percentile at the end of two years in comparison schools actually fell in math. This is not how it is supposed to work in theory. The data are insufficiently detailed to determine whether this last finding actually contradicts the others. The two Michigan evaluations yield apparently contradictory results, but, since they are evaluations of different regions and used different methodologies, it is difficult to know for certain. One study looked at southeastern Michigan, and more than 50% of its schools were in Detroit. It attempted to find schools that were demographically similar to the charter schools (known locally as PSAs - Public School Academies) and found that 71% of PSAs had larger gains on the mathematics test of the Michigan Education Assessment Program (MEAP) than did the regular public schools. In reading, 50% of the gains by PSAs were larger than gains for comparison schools. The other evaluation examined PSAs in the rest of Michigan. It found that the PSAs had lower MEAP scores and smaller gains than comparison schools. Although the PSAs were said to have a higher proportion of at-risk students, no attempt was made to control statistically for this in the comparisons. Although an "educational vision" is the most common reason given for starting a new charter school, without paying more attention to the specifics of the school's program, it will be difficult to know what causes any accomplishments, if they occur. According to the third-year report of the U.S. Department of Education's study of charters, 74.3% of new charters and 66.2% of all charters (including those converted from preexisting private or public schools) have fewer than 200 students.24 Fully 96.8% of new charters and 88.8% of all charters have fewer than 600 students. Class sizes have not been analyzed in detail but are reported to be small. Since small schools are alleged to be more effective and small classes are known to be, improved outcomes in charter schools might be due as much to class-size effects as to anything else. Who is attending the charters is of some interest. The U.S. Department of Education report contains a breakout of charter enrollment by ethnicity, but this is not very useful because it is aggregated at too high a level. State-level aggregations are not much better, lumping high minority areas together with almost all-white areas to attain some meaningless "average" proportion. One of the Michigan evaluations and an evaluation in Arizona found that charters become progressively whiter over time and are whiter than the nearest public school. Western Michigan University's evaluation notes that, "in relation to host districts, the charters as a whole have fewer minorities. Thus there is support for those who argue that the charter schools are skimming and increasing segregation."25 In his doctoral dissertation at Arizona State University, Casey Cobb found that, even at the state level, the proportion of whites in charter schools grew over time, from 64.8% in 1995, to 67% in 1996, to 68% in 1997. Cobb also analyzed theproportion of white students in charter schools and the proportion in the physically closest public schools. This is not a perfect technique, since charter schools often draw from wider areas than neighborhood public schools. Still, the findings are dramatic. One public school enrolled 12% white students, but the nearby charter was 82% white. For 17 of the pairs, the difference in the proportion of whites in public schools and in charters exceeded 20%. Whites were a majority in all the charter schools, but whites were a majority in only three of the public schools.26 Does the market work? That is, do the public schools respond to the charters? Mostly not. Wells' survey and an eight-state survey by Eric Rofes, then at PACE, found the larger systems mostly unaffected by charters. Said Rofes, "The majority of districts had gone about business-as-usual and responded to charters slowly and in small ways." The Western Michigan evaluation found substantially more response in that state. Interestingly, so did Rofes. He noted that the Lansing district initially thought of charters as a "distraction" but then vowed to "out-charter the charters" when the district started losing students. What accounts for these regional differences is not clear. If the public schools are promising to "out-charter" the charters, then, at some level, the market is working. However, the changes that the public schools made had mostly to do with such things as monitoring the playground more closely or making the crosswalks safer. Indeed, one of the things that apparently surprised the Western Michigan evaluators was that they couldn't find a single innovation that was not already part of the repertoire of the public schools. Their concluding statement on the matter captures the flavor of other charter school evaluations as well: In summary, there are many opportunities for charter schools to learn about innovative practices. Since all of these schools are newly developed with the exception of the relatively few converted private or parochial schools, one might expect that innovative practices would be frequent and widespread. However, such is not the case. We found unpredictably few innovations, which would not suggest that transportability is an immediate expectation. In fact, we found the charter schools to be remarkably similar to the regular public schools.27 The Western Michigan evaluators do hypothesize that more innovation might be seen in the future. Michigan charter schools, like most, are quite new, and, as I noted above, many are bogged down in the job of just running smoothly. For those that survive and become stable, innovation and transportability of changes will become more of a possibility. Recall that in Maslow's hierarchy, survival needs come first. On the other hand, neither Wells nor Rofes found the charters serving as laboratories for the rest of the district or region. This absence of concern over what the Western Michigan evaluators called "transportability" is an important finding, though, because such generalization of best practices is one of the major arguments of many charter school advocates. Charter enthusiast Ted Kolderie put it this way: Too often, those asking "what's happening?" [in regard to the impact of a charter school law] look only at the schools created and the students enrolled: the first-order effects of a law. There are also second-order effects: changes/responses in the mainline system when laws are enacted and schools created. An evaluation needs to look for these. . . . Despite what the words seem to imply, "charter schools" is not basically about the schools. For the teachers who found them and the students who enroll in them, true, it is the schools that are important. But for others, from the beginning, "charter schools" has been about system reform . . . a way for the state to cause the district system to improve. The schools are instrumental.28 The "second-order" effects are thus far invisible. This should not surprise anyone acquainted with the history of and research on the dissemination or even the implementation of innovations. The RAND Change Agent Study found that few changes get institutionalized. And I imagine that a study of the lasting impact of the Joint Dissemination Review Panel would have a hard time finding many long-term changes brought into being by the innovations it advocated. Vouchers In thinking about this report early in the spring, I anticipated a long section on vouchers. Now I've decided it would be better as part of next year's report. While a lot is happening, it is just happening now. And given the extensive treatment of voucher programs in the September Kappan, I will be brief. Florida's new law seems to have only one sure winner: the state. The public schools lose the money attached to the student. The private schools get tuition, but if that is less than the value of the voucher, the state keeps the rest. Developments are percolating in other states, and several privately sponsored voucher programs have gotten under way. Maybe by next year there will be sufficient data to report something. I will just mention that the cover story of the July 1999 issue of the Atlantic touted a "Bold New Plan to Fix City Schools," which so far hasn't generated any enthusiasm, perhaps because it looked to me more like a bold plan to sell more magazines.29 The major wrinkle in the new plan, by one Matthew Miller, was to pick maybe three urban areas willing to commit to restructuring their schools. Along with this promise would come an across-the-board 20% increase in spending on schools. In addition, all students would be eligible for vouchers worth one-half of the per-pupil expenditure. Given the limited places available in Catholic and other private schools, the city schools would be ahead, moneywise. Over time, entrepreneurs would build schools that would compete for the vouchers, and the resulting market pressures would force the public schools to improve. There are a couple of problems with this rosy scenario. First, where will the money come from? Miller suggests federal surpluses as the source, and one can already hear Alan Greenspan moaning. This is an odd and probably expedient move for Miller to make. He opens with an excellent summary of why a school-funding scheme based mostly on property taxes is basically "socialism for the rich," in John Coons' wonderful phrase. Why not follow through with an income tax scheme? That would leave no reason to write an article. Second, although Miller glibly says the market will cause schools to be built, he doesn't say how the market will produce the teachers. Let us consider some of the problems in teacher supply. The day this is being written, the front page of the Washington Post features a story on the desperate search for teachers in the region.30 My discussion of class size, above, indicated that the California initiative has dramatically increased the teacher shortage. What's more, I know of some teachers who are being driven into retirement by the increasingly punitive standards movement. They're competent but demoralized. A number of for-profit school outfits pay teachers a lot less than public school teachers get but offer them the solace of an affluent, well-behaved student body in return. (This strategy works best in times when suburban teaching positions are in short supply.) Nobel, Inc., which puts schools into upscale niches, found itself scrambling when the California class-size reduction program opened up many new, better-paying positions in white-collar neighborhoods. All of this does not augur well for finding new classroom teachers. Under the circumstances, one can only imagine that entrepreneurs will look to places other than schools to spend their venture capital. The Neurotic Need To Believe the Worst After a decade of working in the arena of public school performance, I have concluded that many Americans have a neurotic need, almost a compulsion, to believe the worst about their schools. The neurosis goes far beyond Peter Schrag's mordant observation that good news about public schools serves no one's political school reform agenda.31 The need manifests itself in at least two forms: media imbalances and Internet fulminations. Here's just one media example. In November 1998, while otherwise enjoying a holiday abroad, I picked up a copy of the International Herald Tribune, once published by the New York Herald Tribune, now jointly published by the Washington Post and the New York Times. Half of page 2 was devoted to a story from the front page of the previous day's New York Times detailing how a few other nations had surpassed the U.S. in high school graduation rate.32 Times reporter Ethan Bronner cited the otherwise sensible Gerald Graff, a professor of English and education at the University of Chicago, who said, "I think we should be quite alarmed by this." This was in stark contrast to the view of Andreas Schleicher, a top official of the Organisation for Economic Co-operation and Development (OECD). Schleicher argued that "the U.S. has lost its supremacy as the premier educator, not because the United States is doing worse but because there are so many countries who have become better."33 I dare say that, given the steady drumbeat of crisis and failure over the last 20 years, scarcely anyone reading the Associated Press story had the slightest inkling that we ever were the world's "premier educator." Not content merely to report this story, Bronner went back to that nasty piece of propaganda, A Nation at Risk, to dredge up horror stories about the low quality of American education. Bronner called it "an alarming report," implicitly validating its (erroneous) conclusions. Bronner also totally accepted the discredited TIMSS Final Year Study as valid and alluded to an international study of reading skills in adults, about which he erroneously claimed "comparisons of adult literacy rates in the study show Americans to be among the industrialized world's least literate populations." (This study was discussed at length in the Seventh Bracey Report.) The purported deficiencies of American schools are even big news worldwide. Xinhua, the official news agency of the People's Republic of China, also carried the story, with an interesting explanation of the results. Other countries have high unemployment rates, and so young people feel a lot of pressure to do well. Given the low unemployment rates in the U.S., our students can take it easy. Xinhua also balanced the high school graduation rate with the fact that we have the second-highest college graduation rate, 35%, at the typical graduation age.34 Bronner spun this part of the story, noting that the U.S. had the highest college entry rates and also the highest college "dropout" rates. American schools appear to get better treatment from the Communist press than from domestic reporters. Nor is it clear that Bronner's or any of the other stories are even accurate in their principal charge that other nations' graduation rates have surpassed ours. Bronner writes that the study puts the U.S. graduation rate at 72%. This is far lower than any other estimate. For example, the Digest of Education Statistics 1998 puts the figure at 82.1% for persons 25 years old or older. That group, of course, would include many people from earlier years when the graduation rate was lower. A person who dropped out of the senior year of high school in 1950 would be 67 today and would still be included in the figure. The graduation rate in 1950 was 60%. Elsewhere, I have called into question the accuracy of OECD figures, and I do so once again. I also repeat my call for education reporters to be just a bit more skeptical of the figures they're fed and to try and verify them with other sources. In stark contrast to the long lament over graduation rates, the New York Times buried the most recent NAEP reading results deep inside the front section. So did the Washington Post. And USA Today, usually the most balanced and forthcoming paper about education in the nation, entombed its coverage on page D-9. The scores were up across the board. Much was made of the dive NAEP reading scores took in 1994. No fanfare attended the upward swing. In one perverse way, it is just as well that little was made of the 1998 results. I'm convinced that these scores are simply a return to normalcy and that the downturn in 1994 was a technical glitch in equating that year's test to earlier forms. That won't be much solace now to people in Virginia and California, where the apparent slide was used as a lever for adopting ill-considered reforms. Columbine The Eighth Bracey Report opened with the tragic shootings in Jonesboro. Now, with a sad symmetry, I will close the Ninth Bracey Report with a word about Columbine High School - although it is difficult to think of anything new to say about it. All the media shortcomings discussed in connection with Jonesboro were glaringly present in reports about the Columbine shootings. Indeed, I thought of appropriating the old Esquire "Wretched Excess" Dubious Achievement Award to describe it. Perhaps it was the scale of the tragedy that sent people off the deep end. Overlooked in all the talk about violence in the schools was the fact that, before Columbine, the nation was heading for a recent record-low incidence of school-related killings. School violence in general has been declining. In any case, in a press release, Steve Dasbach, national director of the Libertarian Party, suggested that there was something inherent in "government" schools that promoted violence. "After all, high schools should provide an education, not a death sentence. . . . All the shootings happened at public schools, which prompts Libertarians to ask: What's wrong with government schools?" Some Republican members of the U.S. House of Representatives called for posting the Ten Commandments in schools and argued that there was a correlation between the influence of religion and obedience to the law. As Steve Chapman pointed out in Slate magazine, though, correlation is not causation.35 Chapman also observed that the places where the recent shootings had occurred - Pearl, Mississippi; Paducah, Kentucky; Jonesboro, Arkansas; Edinboro, Pennsylvania; and Springfield, Ohio - were not hotbeds of secular humanism or atheism. "Littleton was fertile ground for evangelical churches," he wrote. When many Americans think of "evangelicals," they tend to think of fundamentalist Protestants, but in the broadest meaning of the word this group also includes Mormons, who are not generally thought of in terms of violent tendencies. The Church of Jesus Christ of Latter-Day Saints erected a temple in Littleton in the late 1980s, and the area has a large Mormon population. The one fresh perspective on the situation was offered by two African American columnists for the Washington Post, Courtland Milloy and William Raspberry.36 Milloy wrote that interpretations of the "same" events by blacks and whites were often so different that they seemed to come from parallel universes. A lot of blacks looked at Columbine and said, "I'm so glad those killers weren't black. You know we'd all be in trouble if they were." Milloy contrasted the polite handling of the parents of the Columbine killers with the handling of the case of Susan Smith, wherein black men were being detained in six states before Smith confessed to drowning her two sons and confabulating a black man to blame the murders on. Milloy continued by observing that Harris and Klebold were members of a "clique"; black kids would have been labeled as members of a "gang." If black kids wore black raincoats to class, tattooed themselves with swastikas, and made disturbing videos, they'd be kicked out in a flash, according to the black parallel universe. Earlier, when gun violence was being portrayed as unique to urban areas, sociologists blamed parents who failed to instill proper values in their children. Police promised more arrests, prosecutors promised more convictions, and judges promised more hard time for teenagers. "Now," said Milloy, "in the aftermath of Columbine, the finger is being pointed at Oa culture of alienation,' and there is talk of improving school curriculums, controlling guns, regulating the Internet, and installing V-chips." The portrayal of Klebold and Harris as "outcasts" really stung in the black parallel universe, which is often more cast out than Harris and Klebold ever could be. Milloy quoted sociologist Orlando Patterson: "If the terrorist act of white, middle-class teenagers creates an orgy of national soul-searching, then surely the next time a heinous crime is committed by underclass African American or Latino kids, we should engage in the same kind of national self-examination." Milloy reported that some in the parallel universe think that as the United States becomes less and less white, more and more whites are going to go insane. I don't know if he interpreted the racist shooting rampage in Illinois and Indiana a few weeks later as evidence in support of that hypothesis. Raspberry echoed Milloy and closed his column by urging, "What we need to do, while insisting on the truth of our own reality from our own parallel universes, is to learn to accept that ours is not the only reality - to understand that others can reach different conclusions without being any less decent and thoughtful than we are. The survival of our democracy depends on it." 1. Jeff Archer, "Sanders 101," Education Week, 5 May 1999, pp. 26-29. 2. William L. Sanders, "Cumulative and Residual Effects of Teachers on Future Student Academic Achievement," available from http://www.mdk12.or/practices/ensure/tva/tva_2.html. For an overview, see William L. Sanders, "Value-Added Assessment," School Administrator, December 1998, pp. 24-29. For more details, see "Graphical Summary of Educational Findings from TVAAS," available from http://www.shearonforschools.com/summary/GRAPH-SUM.HTML. 3. Blaine Worthen and Vicki Spanel, "Putting the Standardized Test Debate in Perspective," Educational Leadership, May 1991, p. 67. 4. Lauren Resnick, Education and Learning to Think (Washington, D.C.: National Research Council, 1987). 5. Michael Chapman, "The Magic of Catholic Schools," Investor's Business Daily, 25 January 1999, p. 1; Diane Ravitch, "Why Do Catholic Schools Succeed?," Forbes, 7 October 1996, p. 81; and Nina Shokraii, "Why Catholic Schools Mean Success for America's Inner-City Children," Heritage Foundation Backgrounder #1128, 30 June 1997. Shokraii's article is also available from the Heritage Foundation website at http://www.heritage.org. 6. Richard Rothstein, Martin Carnoy, and Luis Benveniste, "What Can Public Schools Learn from the Private, Non-Profit Education Sector?," unpublished paper, Economic Policy Institute, Washington, D.C., 1999. 7. Anthony S. Bryk, Valerie E. Lee, and Peter B. Holland, Catholic Schools and the Common Good (Cambridge, Mass.: Harvard University Press, 1993), p. 305. 8. Rothstein, Carnoy, and Benveniste, pp. 66-67. 9. NAEP 1996 Science Report Card for the Nation and States (Washington, D.C.: National Center for Education Statistics, May 1997). 10. Jay R. Campbell, Kristin L. Voelkl, and Patricia L. Donahue, NAEP 1996 Trends in Academic Progress (Washington, D.C.: National Center for Education Statistics, September 1997). 11. National Research Council, Grading the Nation's Report Card: Evaluating NAEP and Transforming the Assessment of Educational Progress (Washington, D.C.: National Academy Press, 1999), Executive Summary, p. 11. 12. Patrick J. McEwan and Martin Carnoy, "The Effectiveness and Efficiency of Private Schools in Chile's Voucher System," unpublished paper, Stanford University, 1999. 13. Martin Carnoy and Patrick McEwan, "Private-Public School Differences in Chile's Full Choice, Marketized Education System," unpublished paper, Stanford University, 1999. 14. June Kronholz, "Class Action: A Poor School District Learns to Cope in a Test Tube," Wall Street Journal, 11 September 1998, p. 1. 15. "Class Size Reduction in California: Early Evaluation Findings, 1996-1998," available from the Class Size Consortium, http://www.classize.org, June 1999. 16. Alex Molnar, Class Size and Education Vouchers: An Update (Harrisburg, Pa.: Keystone Research Center, June 1999). 17. Cecilia E. Rouse, "Schools and Student Achievement: More Evidence from the Milwaukee Parental Choice Program," Federal Reserve Bank of New York Economic Policy Review, March 1998. The paper is available from the Federal Reserve Bank of New York website, www.ny.frb.org. 18. Joe Nathan, Charter Schools: Creating Hope and Opportunity for American Education (San Francisco: Jossey-Bass, 1996). 19. Chester E. Finn, Jr., Louann Bierlein, and Bruno V. Manno, "Charter Schools in Action: A First Look," Hudson Institute Briefing Paper, January 1996. 20. Alex Molnar, Giving Kids the Business (Boulder, Colo.: Westview Press, 1996), p. 167. 21. Stuart Eskenazi, "Learning Curves," Houston Press, 22-28 July 1999. The set of articles is available from the Houston Press website at www.houstonpress.com/issues/1999-07-22/feature.html. 22. Eric Rofes, How Are School Districts Responding to Charter Laws and Charter Schools? (Berkeley: Policy Analysis for California Education, University of California, April 1998); Amy Stuart Wells et al., Beyond the Rhetoric of Charter School Reform: A Study of Ten California School Districts (Los Angeles: UCLA, 1998); The Findings and Implications of Increased Flexibility and Accountability: An Evaluation of Charter Schools in the Los Angeles Unified School District (San Francisco: WestEd, 1998); Michigan's Charter School Initiative: From Theory to Practice (Lansing: Public Sector Consultants, February 1999); and Jerry Horn and Garon Miron, Evaluation of the Michigan Public School Academy Initiative, Final Report (Kalamazoo: The Evaluation Center, Western Michigan University, January 1999). 23. Lynn Schnaiberg, "Firms Hoping to Turn Profit from Charters," Education Week, 10 December 1997, p. 14. 24. The State of Charter Schools: Third Year Reports (Washington, D.C.: U.S. Department of Education, May 1999). 25. Horn and Miron, p. 17. 26. Casey Cobb and Gene V Glass, "Ethnic Segregation in Arizona Charter Schools," Education Policy Analysis Archives, 14 January 1999. Available from http://epaa.asu.edu/v7n3.html. 27. Horn and Miron, p. 77. 28. Ted Kolderie, The Charter Idea: Update and Prospects, Fall '95, Public Services Redesigns Project (St. Paul: Center for Policy Studies, 1995). 29. Matthew Miller, "A Bold New Plan to Fix City Schools," Atlantic, July 1999, pp. 15-31. 30. David Nakamura, "Struggling Schools Are Desperate for Experienced Teachers," Washington Post, 29 July 1999, p. A-1. 31. Peter Schrag, "The Near-Myth of Our Failing Schools," Atlantic, October 1997, pp. 72-80. 32. Ethan Bronner, "Long a Leader, U.S. Now Lags in High School Graduation Rate," New York Times, 24 November 1998, p. A-1. 33. "U.S. Outpaced in Graduation Rates," Washington Post, 24 November 1998, p. A-15. 34. Xinhua News Agency, "U.S. Falls Behind Others in High School Graduation Rate," 23 November 1998. Available from www.northernlight.com. 35. Steve Chapman, "Praise the Lord, Pass the Ammo," posted 30 June 1999, Slate, available from www.slate.com/Features/godmurder/godmurder.asp. 36. Courtland Milloy, "A Look at Tragedy in Black and White," Washington Post, 2 May 1999, p. C-3; and William Raspberry, "If They'd Been Black," Washington Post, 7 May 1999, p. A-39. The Awards One section of this year's report is titled "The Neurotic Need to Believe the Worst" and probably needs no further explanation. It might, however, be the reason why, once again, we have more Rotten Apples than Golden ones. Still, a couple of the Goldies represent real accomplishments. For this year's awards, see the sidebars sprinkled throughout this report. Rotten Apple The Who Cares About Facts? Award goes to Willard Daggett, consultant extraordinaire. In the Fifth Bracey Report, I took Daggett to task for inventing research, for inventing research that explained the invented research, and for inventing the organization that conducted the second invented research. When Daggett spoke to faculty members and high school students at North High School in Grosse Pointe, Michigan, in late 1998, a number of his statements raised the eyebrows of at least one teacher. That teacher checked a few claims, found them spurious, and dropped me a letter. The claims were so stunning that I asked for a copy of the video of the Daggett talk. I got one. For the whole hour, Daggett spewed factual errors at a Gatling gun pace. Here are just a few. Many other statements look to be pure fantasy, but I didn't take the time to track them down. Claim: Fifty-four percent of the students in Arizona are in charter schools. Reality: Despite State Superintendent Lisa (I'm-trying-to-kill-education-as-we-know-it) Keegan's extraordinary efforts to make this true, only 2% of Arizona students attended charter schools when Daggett made the claim. The proportion has now edged over 3%. (For more on Keegan, see "The Benedict Arnold Award," page 155.) Claim: Twenty-one percent of the students in California are in charter schools. Reality: All the charter schools in the nation wouldn't hold so many students. At the time Daggett spoke, California had recently raised the limit on charters from 100 to 250. Among the 7,500 public schools in California, there were 156 (mostly small) charter schools enrolling about 68,000 students, roughly 1% of California students.1 Claim: Harvard University brought 2,000 high school valedictorians and salutatorians back to their high schools a year after graduation and gave them final exams in math, science, and social studies. Eighty-eight percent of them flunked at least two tests. Reality: First, try to imagine the logistics of rounding up 2,000 graduates from 1,000 high schools. Then try to imagine a single social studies test that would be appropriate for all 2,000. Despite the efforts of Jerome Murphy, dean of the Harvard Graduate School of Education, and of one of Howard Gardner's graduate assistants, no one has been able to locate this research. Claim: The U.S. is the only industrialized nation that teaches biology and chemistry as separate courses. Reality: A query to Senta Raizen, director of the National Center for Improving Science Education, produced the response that no countries integrate these disciplines at the high school level. The scientists I spoke with said that such integration would be extremely difficult. Claim: Twenty-nine nations require one year of applied physics for graduation from high school, 27 nations require two, and 23 require three. Reality: A survey of the "Structure of Upper-Secondary Education Systems and Characteristics of Students Tested" section in Mathematics and Science Achievement in the Final Year of Secondary School, the final report of the Third International Mathematics and Science Study (TIMSS), indicates that most students don't take any physics. Nor do they take the courses in probability, statistics, measurement systems, and technical reading that Daggett claims these nations also require before they will release students from secondary school. Claim: Ninety percent of the graduates of Grosse Pointe will work for firms that employ fewer than 10 people. Reality: Then Michigan had better start scaling back soon. While doing some demographic research, I accidentally encountered in the 1998 City and County Data Book the information that currently 34% of all Michigan firms employ more than 20 people. 1. Amy Stuart Wells et al., Beyond the Rhetoric of Charter School Reform: A Study of Ten California School Districts (Los Angeles: UCLA, 1998). Rotten Apple The Stop Pointing at Our Emperor Award goes to Susan Klein, Susan Allan, and Caryn Wells, administrators in the Grosse Pointe public schools. After Allan sent one e-mail in response to my comments on the Daggett presentation, these school officials apparently decided that the emperor did indeed have new clothes and so declined further comment or action. I offered to set the record straight for $20,000 - twice the $10,000 that Daggett reportedly carried away from Grosse Pointe for sharing his fantasies. The administrators continued to show the Daggett tape as a "professional development" activity, and Wells, the principal at North High School, told the faculty that I was "stalking" her. The response of these administrators does raise questions. How can education make progress if educators' efforts are informed by statements that are demonstrably not true? How can the public trust educators if they continue to fall for such foolishness? Rotten Apple The Who Cares About Ethics? Award goes to a number of national education organizations. I sent a summary of the Daggett confabulations to several organizations. No serious response issued from any - except for the National Education Association, which apparently sent the info to state affiliates. Joe Schneider, associate executive director of the American Association of School Administrators, responded that there had always been people who made up data and that there always would be, so ho hum or, at best, caveat emptor. I think we should aim higher. When a scientist at Lawrence Livermore Laboratories faked data and was found out, the San Francisco Chronicle broke the story on the front page.1 The New York Times deemed the story important enough for its own page-one account.2 The Los Angeles Times led with it in its State News section, while the Washington Post, obsessed with the death of John F. Kennedy, Jr., and concerned with more local issues, relegated it to page 12 of the front section. The apparent fraud (the researcher resigned but denied any wrongdoing) was discovered by the Office of Research Integrity, part of the U.S. Department of Health and Human Services. I think it's high time that such an office be established in the U.S. Department of Education. 1. Tanya Schevitz, "U.S. Says Fake Data Tied Cancer, Power Lines," San Francisco Chronicle, 23 July 1999, p. 1. 2. William J. Broad, "Data Tying Cancer to Electric Power Found to Be False," New York Times, 24 July 1999, p. 1. Rotten Apple The We'll Know Our Standards Are High Enough When Everyone Flunks Award goes to the Virginia State Board of Education. This august group of worthies almost accomplished for the Old Dominion what Washington State software engineer Arthur Hu says is only dreamed of in the Pacific Northwest. Hu claims that Marc Tucker is now huckstering Washington with a set of tests that everyone will fail. When they do, Tucker can sell his education reform program for oodles. As evidence, Hu cites seventh-grade math problems that call for skills taught in most math texts well into the ninth grade and fourth-grade items like the following that require skills that show up in seventh-grade texts: Lisa put some fruit in a large bowl. The bowl had twice as many apples as oranges, and half as many pears as oranges. Altogether there were 14 pieces of fruit in the bowl. How many apples did Lisa put in the bowl? How many oranges? How many pears? In Virginia, the state board of education successfully established such tests for grades 3, 5, 8, and 11. Ninety-eight percent of Virginia's schools failed. Yes, 98%. If a school doesn't meet the board's standards, it will lose its accreditation. If the students don't meet them, they don't graduate. How reasonable is it to build tests that virtually everyone fails? Follow me along a stream of data: * Students in only six of the 41 countries in TIMSS scored higher than students in Iowa in math, and students in only one country did better in science (see the November 1998 Research column for the data). * Students in Iowa score between the 62nd and 68th percentiles on domestic achievement tests, depending on grade and test. (This information was supplied by H. D. Hoover, director of the Iowa Testing Programs.) * Students in Fairfax County's 202 schools score between the 64th and 83rd percentiles on domestic tests. Most of their scores are in the upper 70s - much higher than "world-class" Iowa. * Only 13 of Fairfax County's 202 high-scoring schools passed the Virginia test. The state board seemed very satisfied. But then Virginians seem to have had punitive attitudes about their schools for a long time. Maybe the last Virginian to reward success instead of punishing failure was Thomas Jefferson, who established a system that allowed poor children to go to school for free all the way through college as long as they did well enough. (Rich children could go as long as their parents forked out tuition, no matter what their performance looked like.) The board went through the motions of establishing cutoff scores, using the fairly well-accepted Modified Angoff Method. The board then accepted only the most extreme recommendations of the judges. In some cases, the board actually set cutoff scores higher than any score recommended by a judge. When the tests were established, several board members were ardent advocates of vouchers. The test results have led some to suspect that the cutoff scores were deliberately designed to make Virginia's public schools look bad. According to this theory, the hope is that citizens will be made more concerned about the schools and so more ready to permit the transfer of public funds to private schools through vouchers or other schemes. Although scores rose on the second round of testing, these results serve to invalidate the tests even further. Some people dismissed the first round, saying that since it didn't count, people didn't take it seriously. Well, they took it seriously after the results. The first year's results were the two-by-four that got the donkey's attention. In the second round, though, only 23% of Fairfax County's 202 schools passed. TABLE 1 NAEP 1996 Science Results Percentile Rank 10th 25th 50th 75th 90th Grade 4 Public 103 127 151 172 188 Catholic 126 145 165 182 197 Other Private 124 145 164 182 197 Grade 8 Public 102 126 151 172 191 Catholic 123 143 164 182 199 Other Private 118 142 164 183 199 Grade 12 Public 103 126 151 174 192 Catholic 115 136 156 175 191 Other Private 111 137 158 177 193 Rotten Apple The Benedict Arnold Award goes to Lisa Graham Keegan, state superintendent of public instruction in Arizona. As a state legislator, Keegan wrote voucher laws. When her voucher efforts fell short, she came back with charter school legislation that passed. Then she hopped over to become state superintendent. Talk about foxes guarding hen houses! In a speech to a Washington, D.C., conference on privatization, Keegan recounted an encounter with a woman who approached her and said, "You're killing public education as we know it." Said Keegan to the conference, "I told her, OEvery night I pray that it is so.'" TABLE 2. NAEP Science Trends Percentile White Black Hispanic All Rank 1977 1996 1977 1996 1977 1996 1977 1996 9-Year-Olds 5th 163 173 107 139 125 142 144 159 50th 230 240 174 202 191 208 222 231 95th 295 303 244 264 261 268 291 298 13-Year-Olds 5th 191 209 144 168 147 174 174 191 50th 256 267 207 225 213 231 249 258 95th 321 321 275 285 282 293 317 317 17-Year-Olds 5th 231 237 172 192 194 196 213 217 50th 298 309 240 259 262 270 291 298 95th 365 371 310 327 331 340 362 365 Source: Christine Y. O'Sullivan, Clyde M. Reese, and John Mazzeo, NAEP 1996 Trends in Academic Progress (Washington, D.C.: National Center for Education Statistics, Report No. NCES 97-98, September 1997). Rotten Apple The Ideology Is More Fun Than Science Award goes to Paul Peterson of Harvard University. At a March 1999 luncheon at the National Press Club in Washington, D.C., Peterson introduced speaker Andrew Coulson, whose book Market Education: The Unknown History was the occasion for the food.1 Citing TIMSS, Peterson claimed that "the longer children remain in school today, the farther behind they fall" in relation to students in other nations. Peterson has apparently abandoned disinterested research in favor of advocacy. In a 1990 publication, he declared that voucher advocates "are a small band of Jedi attackers, using their intellectual powers to fight the unified might of Death Star forces led by Darth Vader, whose intellectual capacity has been corrupted by the urge for complete hegemony."2 As Jedi Peterson waved his light saber at the data from the Milwaukee voucher program, people looked on quizzically. Initially, no one could figure out what methods he had used, and to date no one has been able to replicate his findings fully. It is more than passing strange that Peterson, who went over the Milwaukee data with such a fine-toothed analytic comb, would accept uncritically the discredited TIMSS Final Year Study.3 The food was pretty good, though. 1. Andrew J. Coulson, Market Education: The Unknown History (New Brunswick, N.J.: Transaction Publishers, 1999). 2. Paul E. Peterson, "Money and Competition in American Schools," in William H. Clune and John F. Witte, eds., Choice and Control in American Education (London: Falmer Press, 1990), p. 73. 3. Gerald W. Bracey, "TIMSS, Rhymes with ODims,' as in OWitted,'" Phi Delta Kappan, May 1998, pp. 686-87; idem, "Tinkering with TIMSS," Phi Delta Kappan, September 1998, pp. 32-36; and Iris Rotberg, "Interpretations of International Test Score Comparisons," Science, 15 May 1998, pp. 1030-31. Rotten Apple The Logical Path Not Taken Award goes to Andrew Coulson, the speaker Paul Peterson introduced. A member of the audience asked Coulson if, in the course of his research, he had found any case in which a state-controlled school system had become a market system. Coulson said he had not. In all cases, he averred, nations that have started with market systems have fallen under the sway of honeyed promises from those touting state control. No nation has ever reverted to a market system. How interesting. Considering how cheerfully various nations have overthrown governments and even entire forms of government, does it not seem that people would readily rid themselves of a single, not very powerful government agency - state-sponsored schools - if they chose to? Are those who advocate state control of schools so silver-tongued that they have been able to pull so much wool over so many eyes in so many nations for so many years? One might think that an institution with as much tenacity as the public schools is fulfilling some function that the people actually want. But then, if that train of logic were followed, Market Education might be a long-discarded idea, not the title of Coulson's book. EDDRA: The Education Disinformation Detection and Reporting Agency As part of the "Who Cares About Ethics? Award" (page 152), I call for the establishment of an Office of Research Integrity in the Department of Education similar to the one operated by the Department of Health and Human Services. It is a serious call, and I hope others will follow up. In the meantime, I have established a website for the purpose of debunking disinformation. The immediate impetus for the site was a story in the Washington Post indicating that students schooled at home score well on tests.1 I saw the story around 6:30 a.m., and by 8:30 I had the study in hand and knew it to be largely a public relations stunt by the Home School Legal Defense Association. The study itself was properly conducted. No integrity problem there. The problem lay in the demographics of the sample. The educational levels and affluence of the home-schooling families were such that one could safely predict that their children would score well on tests in any school setting. Indeed, one could take the affluence, the families' levels of education, and the power of one-on-one tutoring and make the case that these kids were underachievers. At the time, there was no place where I could put these observations. Thus the EDDRA website. It can be reached at http://www.america-tomorrow.com/ bracey. The site is free, but people need to register in order to get the periodic updates and any breaking news. - GWB 1. Jay Mathews, "A Home Run for Home Schooling," Washington Post, 24 March 1999, p. A-11. Mathews' story did not display the hyperbole of the headline. Rotten Apple The Unquestioned Acceptance Without Digging Award goes to Thomas Boysen of the Milken Foundation (former commissioner of education in Kentucky) and Thomas Sobol, a professor of education at Columbia (and former commissioner of education in New York). In the 10 March 1999 issue of Education Week, Boysen and Sobol wrote, "The public must also be prepared to live temporarily with distressingly low test scores of the sort that occurred recently in Virginia and Massachusetts."1 One might think that former commissioners of education might be a little skeptical of such reports. The Massachusetts test was a test for prospective teachers, which 59% of the candidates failed. It has been called very much into question.2 While the department of education in Massachusetts and the test developer, National Evaluation Systems, have refused to supply any reliability or validity data, Boston College researchers estimated the reliability of the tests to be, well, sorta low: .27 for reading; .36 for writing. Even when the researchers removed some outlier scores, they were unable to get the coefficients above .50. Remember, when we're talking reliability, we're aiming for .90 or better. The Boston College estimates are probably too small because of a statistical phenomenon called restriction of range, so why won't the state department release any reliability data? What happens when these high failure rates collide with the teacher shortage has not yet received much attention. The Virginia scores, by the way, were not so "distressingly low."3 1. Thomas C. Boysen and Thomas Sobol, "Educational Accountability: The Next Steps," Education Week, 10 March 1999, p. 52. 2. Walter Haney et al., More Error Than Truth? An Independent Examination of the Massachusetts Teacher Tests (Chestnut Hill, Mass.: Boston College, February 1999). 3. Gerald W. Bracey, "Weigh Virginia Results on a Global Scale," Education Week, 3 February 1999, p. 37. Rotten Apple The 1999 We Don't Have to Check for Accuracy Because We All Know It's True Award goes to Stephan Thernstrom, Winthrop Professor of History at Harvard University. In the introduction to the fall 1998 edition of The Concord Review, Thernstrom, co-author with his wife, Abigail Thernstrom, of America in Black and White, presented the following gloomy view of our schools. It is hard not to feel discouraged about the state of our elementary and secondary schools these days. We spend more and more money on education each year, but what we have to show for the investment is very much open to question. For more than a quarter century now, the National Assessment of Educational Progress [NAEP] has been monitoring the achievement of representative samples of American students at the ages of nine, thirteen, and seventeen. The results are disheartening. At the end of their high school education, our students today are not significantly better at reading, writing,