Holding Back to Get Ahead

Researcher Joanne Golann says that no-excuses charters are teaching low-income students to defer to authority and hold back their opinions—the opposite of what they’ll need to succeed in college and life.

walking in lineEduShyster: You’re one of the first researchers to spend an extended period of time inside a no-excuses charter school. Here’s where I ask you to sum up close to two years of research into a single sentence. OK—you can have a paragraph.

Joanne Golann: I found that in trying to prepare students for college, the school failed to teach students the skills and behaviors to help them succeed in college. In a tightly regulated environment, students learned to monitor themselves, hold back their opinions, and defer to authority. These are very different skills than the ones middle-class kids learn—to take initiative, be assertive, and negotiate with authority. Colleges expect students to take charge of their learning and to advocate for themselves. One of the students I talk about in the article learned to restrain herself to get through, to hold herself back and not speak her mind. She ended up winning the most-improved student award in 8th grade for her changed behavior.

EduShyster: To conduct the study, you were basically *embedded* at a school you call Dream Academy, an acclaimed charter school in a Northeastern city. Explain how you did your research.

Golann: For a period of fifteen months between 2012 and 2013, I was at the school, a no-excuses middle school of about 250 black and Hispanic students, four or five times a week, for at least half the day. I spent another three months in the high school. I observed classes, teacher and staff meetings, and student and staff orientations, and also hung out with students at lunch, after school, and on field trips. I interviewed 58 students and 34 staff members. I think it helps that I look about as old as these middle school students. I obviously wasn’t one of the students but I did try to blend in and do what they did. When they had to walk in straight lines in the hallways, often I would just fall in line. I’d sit next to students in their classes if there was an empty desk. That’s part of the ethnographic tradition, to try to understand a lived experience by putting yourself there.

schooling-in-capitalist-americaEduShyster: One of your conclusions is that the overwhelming emphasis on order in the school, a feature of no-excuses charters, ends up socializing kids to follow rules and obey authority. In fact, you argue that even as they talk the language of social mobility, these schools are instilling in kids some very familiar *working class* skills and behaviors.

Golann: Bowles and Gintis wrote this famous study where they were looking at the history of mass public education in the US. They argue that schooling expanded in large part to quell social unrest. You had these immigrant populations coming into the cities in the mid-nineteenth century, and Bowles and Gintis basically make the argument that factory owners and the professional class wanted a docile workforce. They wanted people who would be obedient and man these factories, and so they used schools as a way to socialize children to follow rules and show deference. Looking at the school I studied, I found the same behaviors but with a very interesting twist. In a new era of accountability, instead of creating workers for the factories, schools are creating *worker-learners* to close the achievement gap. Schools are emphasizing obedience because they need to create order to raise test scores and they see that as the way to social mobility. It’s the same behaviors but for a different purpose.

EduShyster: The leaders of the school you studied seem very aware of this contradiction—that by rewarding compliance and deference to authority, they were undermining the exact skills their students were going to need in college and life.

Golann: I found very clearly that the school leaders wanted the opposite. The principal of the school really wanted to empower students to be able to use their voices and make social change. But she also wouldn’t budge from this highly prescriptive system because of her fear that chaos would ensue. School leaders didn’t see a way out and they weren’t willing to compromise order because they had experienced really chaotic urban schools. They didn’t see an alternative. School leaders would tell me *I haven’t seen anything else that works, and what I have seen work is this no-excuses model so I’m going to adopt it.*

detentionEduShyster: The students you observe complain about being punished indiscriminately, and that the nature of the system means that they have no ability to protest, even if they didn’t do the thing they’re being punished for. You conclude that often times the students were right and the teachers were wrong, and yet in the classrooms you observed this rarely seemed to matter.

Golann: If you constantly give out consequences and don’t have time to stop and reflect and have a discussion with the child about what just happened, you’re going to get into a situation where students start to perceive you as being unfair and the system as being unfair. Ultimately what happens is that you start to lose your authority, and that’s a vicious cycle. You don’t want to have to be dependent on the rules to run your classroom. You want students to willingly comply with you because they respect you, not because you’re going to give them a detention. I think the model really reduces teacher discretion. It’s made to replace or substitute for teacher discretion. In fact, the principal told me that that’s the purpose of the model because discretion doesn’t work. It’s too dependent on a teacher having a good or bad day. But teachers are working with individual students and specific situations—a good teacher is not a robot.

EduShyster: I recently interviewed a student who spent a year attending a school similar to Dream Academy. He’s only 15 but keenly aware of the civic implications of a system where kids can’t question authority. Here’s how he put it: *These students will end up just going along with anything anyone says. The government will make a dumb rule and they’ll be like ‘let’s just follow it.’*

Golann: The schools talk about promoting individual mobility—getting these students to college so that they can get good jobs. But there’s a large critical education tradition that argues that if you really want to make social change in society, you need to teach people to become active citizens, to have the skills, ability and desire to question authority.

If we create an educational marketplace where success is measured by student test scores, perhaps it is not altogether surprising that we end up with a rigid school model that produces these test scores. What we don’t get is a model that teaches students how to speak up or even a model that leaves students feeling like they have had a positive school experience.

EduShyster: While your research focuses on one school, as you point out, the no-excuses model has been extensively replicated. You argue that *the expansion of choice and accountability has led to the copying of a school model that produces test results yet limits the development of students’ higher-level skills.* As a chronicler of the unintended-consequences of education reform, I found your conclusion both affirming and horrifying.

jigsaw-puzzle-pieces-cookie-cuttersGolann: If we create an educational marketplace where success is measured by student test scores, perhaps it is not altogether surprising that we end up with a rigid school model that produces these test scores. What we don’t get is a model that teaches students how to speak up or even a model that leaves students feeling like they have had a positive school experience. While charter schools were originally seen as a way to innovate, a way for communities to develop schools that might better fit their students and families, what’s come to dominate the charter field are charter management organizations and this no-excuses model. For example, in Boston, one study found that 71% of the urban charter schools subscribe to the no-excuses model. Of the high-achieving urban charters, almost all are no-excuses schools. They’ve expanded rapidly because of the support of foundations and the US Department of Education. Some $500 million in private foundation money has gone into replicating these schools.

EduShyster: The last line of your paper is really powerful. In fact, I’d like to take this opportunity to read it aloud so that we can all go forth pondering the essential point you make. *If teachers and administrators committed as much effort to learning about students’ families and neighborhoods as they dedicate to raising test scores or managing behavior, they might discover new ways of instruction and management to get kids to and through college, and perhaps more importantly, prepare them to ‘be the change,’* as one Dream Academy leader described.

If teachers and administrators committed as much effort to learning about students’ families and neighborhoods as they dedicate to raising test scores or managing behavior, they might discover new ways of instruction and management to get kids to and through college, and perhaps more importantly, prepare them to ‘be the change…’

Golann: I believe that. I think these schools work so hard, and the teachers work so hard. They’re smart and passionate, and there are a lot of great individuals working in these schools. It just makes you wonder, what would happen if all of this energy was directed in a different direction? The no-excuses model has become so dominant, but if we really put our energies in a different direction, couldn’t we find a model that doesn’t infringe so heavily on student autonomy? Administrators and teachers have very little time to get to know families and communities because they dedicate so much energy to discipline and raising test scores and trying to maximize instructional time. That makes it hard to find and try other approaches.

Joanne W. Golann is a Ph.D. student in Sociology at Princeton University. She is interested in how social class shapes experiences and skills. She will begin in August 2016 as an Assistant Professor of Public Policy and Education at Vanderbilt University in the Department of Leadership, Policy, and Organizations.

Share on Facebook0Tweet about this on TwitterShare on Google+0Print this pageEmail this to someone

39 Comments

  1. There certainly is a high school model in which students learn to think, talk, respect one another in discussions/debates, research, assert their views (and back them up), develop real relationships with teachers, etc. – the New York Performance Standards Consortium. Most of the schools are in New York City. They demographically mirror the city’s student population (black, Latino, ELL, with a disability, income) but tend to have slightly lower than average test scores. As a result of programs that engage and empower students, they do very well: grad rates exceed city rates for all groups (about double for ELLs and students with disabilities); more grads go to college; and rates of persistence in college exceed national averages. Meanwhile, suspension rates are half those of the city and teacher turnover rates also far lower. This thoroughly proves that ‘no excuses’ is simply not necessary. Indeed, if the goal is thoughtful and engaged humans, as well as college and worklife success, the Consortium succeeds well – and ‘no excuses’ does not. Let’s just say Consortium shows there is no excuse for ‘no excuses’ schools. BTW, unlike lots of charters, Consortium does not have extra funds from hedge funders, foundations and the like. For a detailed report on their success and the graduation performance task process, see
    http://performanceassessment.org/articles/DataReport_NY_PSC.pdf.

    1. The sad thing is that the Performance Standards Consortium has been around for decades with, as you point out, a very strong track record. Given its successes in NYC schools before, during and after the Bloomberg-era school reforms, one has to wonder if “reformers” are willfully ignoring this model. It is, after all, both successful and empowering…

  2. There are numerous examples of public school districts that excel, have engaged faculty and staff, and students who love to come to school and learn. And no they are not all in wealthy districts. It is all about leadership and not the Broad Academy style of authoritarian management. Leaders who are respectful and actively engage all stakeholders in ongoing conversations around questions that matter create these districts. And yes that means giving power away to teachers and supporting innovation, collaboration, and learning. Reformers seem to want robots and how extraordinarily discriminatory is that to impose on minority children and teachers.

  3. Didn’t Woodrow Wilson (yes, the racist president) strongly endorse schools that taught students how to be industrial employees? Stand in line at recess, speak when asked to, do as your told, etc. Sounds like the foundational theory of these charter schools.

    “No excuses” is not a synonym for “the staff is always correct”, is it?

  4. Edushyster,

    You are so smart, but you are really blinkered when it comes to classroom discipline. You listen to Princeton sociology students but not real public school teachers? There is a crisis of disorder in our schools. Believe me, kids are not afraid to speak their minds. (God, if only they would STOP speaking their minds for a second). Let us grant that a disposition to speak one’s mind is useful in college and career. Can we also grant that academic knowledge is also useful in college and career? And can we also grant that disorderly classrooms prevent the acquisition of this academic knowledge that is very useful in college and career? The no-excuses schools are not on some sadistic power-trip. They’re using strong medicine to cure a potent disease…so that they can deliver some other important medicine: knowledge. Edushyster, you will gain a lot more credibility in my eyes if you spend an unchaperoned week wandering the halls of a regular public middle school and THEN declare that no-excuses discipline is a monster that must be slain. I’m not a huge fan of the super-strict no-excuses model, but I’ll take that over chaos any day. Why do you think so many teachers feel embattled in regular public schools? They’re not being nice enough?

    1. I don’t pretend to have answers to every question, but I do have several years experience in both public and private school education. I find that kids tend towards chaos more when they are sure that their voices will not be heard. I usually encourage classroom discussion that often gets chaotic, but once I’ve proved to my classes that I will make sure that every voice gets a chance to be heard, the chaos tends to die down. An overly disciplinarian model serves to quash students’ voices, which leads them to believe that in order to be heard they need to talk louder. The result is chaos.

      When it comes to the “dependent on the teacher’s mood”, I find that honesty is a fabulous policy. I very frequently start off class by telling the class that through no fault of their own, I’m already worn out and on the edge of a foul mood, and that I’m going to do everything I can to not fly off the handle, and would appreciate them doing everything they can to help me to not do the same. Usually, we all get through the class period rather well. The situation is clear from the outset, expectations have been set for all parties involved, with immediate consequences established for both paths of behavior.

      Sure, it goes south from time to time. Sure, it’s not perfect. But I’m pretty sure that my students know that I do my level best to be fair and reasonable, handle each situation individually, and basically model true adult behavior. Which also means that when I fail at being an adult (or even a good human being sometimes), I apologize and ask for their understanding.

      I guess what I’m saying is that offering and expecting adult behavior goes a long way at a critical time when teenagers need every example of maturity they can get.

    2. Ponderosa, real-life (now-former) public school teacher here, who’s taught pre-K-12 in wealthy, middle-class, and poor urban schools.

      I discovered once I hit my stride professionally that I didn’t need “carrots and sticks” or PBIS or stickers or detentions or calls to security to maintain order and keep instruction going, and calls to the office for support were limited to extreme cases (maybe once every month or two?). Making connections with my students was key; not talking down to them was helpful as well. I didn’t have a List of Rules and another of Consequences on my wall – I just TAUGHT. The only times there were regular disruptions was when there were too many kids in a class, or when there simply wasn’t enough support to reach the harder-to-reach students – and 9 times out of 10 (probably more), that happens in the poor under-resourced schools.

      Agree 1000% with scruffy that the noise and chaos was always worse when students didn’t feel “heard;” heck, look at the Presidential campaign now and tell me this doesn’t apply to adults too. :-/ When I had the time, space, and support to allow for connection and for “hearing,” that took care of the vast majority of discipline problems pre-emptively. If you’re in a situation where you’re not getting that kind of time, space, or support, it might be time to start advocating for some rather than putting the blame solely on the students.

      1. I’m glad you didn’t need to use threats of punishment, Crunchymama. The majority of my students don’t need it either. But some do. That’s the thing that Edushyster and others don’t seem to understand. Not all kids are the same. Generalizing from one’s own civil, upper middle class milieu to ALL kids is a grave intellectual error. I frankly admire KIPP founder Dave Levin and his ilk for flouting the anti-discipline orthodoxy. Perhaps they’ve taken it too far, but the regular public schools are being shoved to the opposite, permissive extreme and the results are ugly. Restorative justice sounds good on paper, but teachers need something quick and effective. Otherwise school devolves into a long juvenile court/therapy session, with actual academics as a small side dish. I teach in a very middle middle class suburban district, with a mix of poor and rich tossed in. Even here, by this time of year, most teachers are fairly fried from the daily wear-and-tear of dealing with the usual suspects’ unruliness. I can only imagine what it’s like in a lower-income district. It shouldn’t be this hard.

        On HuffPost today there’s an interesting Andrew Sullivan essay about Trump. Sullivan talks about Plato’s description of late stage democracy where all authority is cast down in an ever-increasing demand for total equality of everyone. In classrooms, “as the teacher … is frightened of the pupils and fawns on them, so the students make light of their teachers.” We’ve reached this state in many classrooms, and the process is being accelerated by PC ideologues and mandates from above that are killing what little power adults still have over kids. Cajolery, or fawning, may be the only tool we’re left with.

        1. Posting this on Crunchymama’s behalf as her computer crunched, leaving her unable to share this response with you herself:

          “Actually, Ponderosa, I’m inclined to think that perhaps you’re reading it wrong if you think that “no discipline at all” is what’s being advocated for here.

          The point isn’t that there are no students who need more structure than others. The point is that when this is the default for ALL students, Every Single Time for Every Single Infraction, then I agree with you that “they’ve taken it too far,” only without your “perhaps” qualifier. Especially for young children, there is no need to start out draconian and progress from there.

          Restorative Justice was never meant to be a quick fix. It’s a process, an atmosphere set up from Day One, and it lays a foundation for the rest of the year and beyond that *saves* time in the long run. Maybe not for the faint of heart, but then again, neither is teaching, KWIM?”

  5. Jennifer,
    I wonder if you’ve ever read Common Ground by J. Anthony Lukas. Great book about the bussing experiment in Boston schools. In some ways you remind me of the WASPy liberal crusaders from the suburbs with their self-righteous black-philia who demonized Boston’s working class whites for their reluctance to integrate. My reading of that book is that the Southie Irish, aware of their own precarious perch on the lower-middle rungs of the socio-economic ladder, did not relish getting lumped together with what they perceived to be an even more precarious group. The sprinkling of inner-city kids who bussed out to the WASPy suburbs were not a threat to the much more secure upper middle class there. Liberal elites’ Afro-philia led to a blinkered, romantic and unjust policy that, conveniently, only had adverse effects on an Other: the working class whites. Your crusade against strict school discipline reminds me of this.

  6. I can’t tell you how spot on Ms golan’s discoveries and comments are to members of the Bread Loaf Teacher Network. . If you listen to public school teachers from the same communities that their students are from (ethnicity is usually not a factor) , you will hear that issues of discipline and intrinsic motivation are a function of dysfunctional academic systems as the one described in this interview and teachers who don’t have the time or space to learn about their students’ cultures. Regarding the test mania and how it cripples students, not only are students unable to navigate the college scene, but when they go to private high schools, the same phenomena occurs. Dozens of successful youth and young adults from public and parochial schools in Lawrence MA are confronting the issues described in this interview and working with teachers and youth workers to show alternatives to the deficit educational pedagogies that dominate the landscape in most of our educational systems. Thank you for EduShyster’s important work.

  7. Thank you, Jennifer and Joanne, for this fascinating conversation. Joanne, I saw so many of the same phenomena when I taught at a no-excuses school in Boston. I think much of the explanation for no-excuses leaders’ actions (and the contradictions between ends and the means) lies in the fact that leaders simply will not engage with what they do not understand— because when you don’t understand something, you can’t control it. (When I taught at the school, leaders did not want to deal with the complexity of students’ experience regarding trauma and racial discrimination, so they simply forbade true, open-ended discussion about emotion and race.) In my opinion, “no-excuses” discipline and pedagogy is oppressive because it caters’ to leaders’ comfort levels, *not* students’.

    Ponderosa, regarding your point, I agree that when there is sheer chaos learning cannot happen. There *is* chaos in a lot of public schools. The solution, though, is *not* no-excuses discipline; it is implementing a school culture which reflects’ students’ backgrounds and which acknowledges and deals with student trauma in a sophisticated, empathetic way. This is a long, hard, expensive process, but it is the only way. Chaos does not reign at *all* urban schools, just at ones at which a) there is insufficient support and resources to address the wide range of students’ academic and socioemotional needs; b) too high a student-teacher ratio; and c) educators who are not trained in progressive discipline methods.

    1. “[The solution] is implementing a school culture which reflects’ students’ backgrounds and which acknowledges and deals with student trauma in a sophisticated, empathetic way.”

      Emily, I’m afraid I don’t believe you. This is PC, education school boilerplate that you seem to be parroting. You’re saying the acceptable thing to say, but do you know this to be true? If so, how? How do you “know” that trauma is the main source of kids’ misbehavior? What if it’s peer dynamics? The confluence of under-preparation and a strong urge to socialize? Allowing an innate sadistic streak that lurks in many of us to have free rein? The failure to consistently enforce discipline? I can think of many other plausible hypotheses. Reducing the sources of classroom disorder to trauma, as many opiners on education do these days, is to me the sign of depressingly shallow thinking on this subject.

      1. Ponderosa, I know this because I’ve taught in schools which do this as well as a no-excuses school which decidedly does not. This isn’t the “acceptable thing to say” in the circles I’ve traveled in at all— it’s my belief based on direct experience in a wide variety of settings.

  8. It breaks my heart to think that only a decade or so ago we (teachers) were being told that our students would have to know how to be creative, and be able to figure out how to adapt and synthesize new ideas if they were going to be able to survive in our “modern day” economy. But look how, instead, we are now shackling and attenuating any kind of student (or teacher) creativity — and then sending these same kids out into a non-factory-line high-tech world.

  9. Chaos does indeed happen in some public schools. This is in large part because teachers (and administrators) have been more and more prevented from inflicting any consequences whatsoever for student misbehavior, no matter how extreme, and teachers are blamed for any and all shortcomings, misbehaviors, and even fights between students.

    Some sort of order is indeed required for learning to take place, and so is mutual respect. When students who are wandering the halls instead of being in class can enter other classes and scream and shout at the students or adults in that room, with no consequences whatsoever, then you have chaos. If a fight occurs between two students, and administrators refuse to take any action whatsoever, you have chaos. When assaults on school personnel or destruction of school property goes unpunished, you have chaos.

    However, this sort of chaos does happen, and administrators are more and more reluctant to enforce any rules for such outrageous behavior, because if they have high suspension rates, then their own jobs are liable to be lost.

    Somehow, the only schools where any sort of discipline can be imposed are certain chains of charter schools. Why are they allowed to impose discipline (even if it’s excessive and overbearing) whereas many public schools – at least the ones I see for the most part here in Washington DC – are forbidden from imposing sanctions of ANY sort on student misbehavior?

    It’s really like something out of Alice In Wonderland in a very twisted and perverse universe.

    1. Well-said, Guy. I would love to see Edushyster crusading for more discipline in regular public schools, instead of crusading for less discipline in charter schools.

      1. Well, Ponderosa, while teachers and administrators in regular public schools should have the right to impose a reasonable sort of discipline, I agree with EduShyster and others that what they have in certain no-excuses charter schools is absolutely insane.

        I know some students who endured KIPP and absolutely hated it.

  10. Joanne Golann: “I found that in trying to prepare students for college, the school failed to teach students the skills and behaviors to help them succeed in college.”

    Edushyster (Dec 2015): “We don’t have data yet on how graduates of Boston’s charters fare in college. (The reports I hear, though, are that completion is a HUGE issue). ”

    It’s a little bit of an apples to oranges comparison (e.g., the BPS figures seem to include highly selective exam schools), but perhaps of interest:

    From: The Boston Opportunity Agenda’s Fourth Annual Report Card (Jan, 2015)
    For Boston Public Schools (BPS) students, “Of the students who entered 9th grade in 2002, 65% completed high school in five years, only 34% enrolled in college, and only 17% obtained a degree within six years of the date they should have graduated from high school.” (page 16)

    Outside of BPS, for Boston charter school students: “Of the students who entered the 9th grade in 2002, 81% completed high school in five years, 60% enrolled in college, and 25% had a postsecondary degree within six years of high school graduation.” (page 17)
    https://www.tbf.org/~/media/TBFOrg/Images/BOA/BOA%202014%20Report%20Card.pdf

    According to the following year’s Report Card, reporting on the 9th grade cohort subsequent to the one above:
    Boston Public Schools (BPS):
    65% graduate within 5 years
    33.8% enrolled in college
    17% graduated within 6 years

    Non-BPS public charter schools
    83% graduated High School in 5 years or less
    69% enrolled in college
    35% graduated within 6 years of HS graduation

    I, myself, commonly end up bemused and confused when looking directly at DESE data, and would be interested in comments from anyone believing the data above is, or is not, a fair reflection of reality.

    Stephen Ronan

    1. Stephen,

      I would not really worry about the inclusion of qualified admission high schools in the BPS data. It is likely that students admitted to those high schools would have graduated from any high school within 5 years, enrolled in a college, and graduated from college within 6 years of graduating high school. I should add that this may not be true for the very most gifted students at the qualified admission high schools because traditional high schools do not offer a deep enough curriculum to keep them interested.

      The more problematic apples vs oranges problem is that all of the students at the charter schools applied to go to the charter schools while the students in BPS either did not apply for a charter school or did not get chosen in the lottery to attend a charter school. If you think that application to a charter school is a signal of family traits like concern for education, comparing all charter school students to all BPS public school students would incorrectly attribute better performance that was actually due to family traits to attending charter schools. That is why the best studies look only at students who have applied to attend charter schools and compare the performance of the lottery winners who attend the charter and the lottery losers who attend traditional public schools.

      Complicating matters, in densely populated areas with rich transportation networks, families choose a school district by their choice of residency. It may be that students who do not win the lottery to attend a charter school (or win a place at the qualified admission high school, for that matter) leave the district to attend school in a different district. While my local school district boundary lies entirely in farmers fields, moving from the 900 block of Beacon to the 1000 block of Beacon allows a student to attend Brookline Public Schools instead of Boston Public Schools. Controlling for this is more difficult.

      1. Thanks, teachingeconomist,

        As you suggest, all the BPS schools that identify and select for admission many of the city’s most highly skilled and motivated students likely skew the BPS results higher, while the parents of other kids, who get into school via lottery and seek charter school admission, are likely relatively highly motivated and well-informed compared to BPS lottery participants who do not seek charter school admission, which bodes well for their children’s success in school.

        How the effects of attrition and backfill play into the picture are perhaps trickier to estimate, especially the latter. Although the Boston charter school attrition rates are substantially lower than those at traditional district schools in the city, it seems plausible, even likely, that charter school attrition more frequently involves the students who are less likely to graduate and attend college than their classmates. But I’m unaware of hard data confirming that. In respect to backfill, I find it tough to estimate the impacts on graduation and college completion rates of the fact that charter schools are more likely to only backfill in respect to younger kids while BPS traditional district schools may also backfill with kids who are already very close to graduating.

      2. Stephen,

        Perhaps I stated it badly. I DO NOT think that including qualified admission high schools in the data set scewed the numbers for BPS by an appreciable amount unless most of those students would have left BPS if the qualified admission high schools did not exist. I think it is obvious that almost all students admitted to qualified admission high schools would be equally likely to graduate from a traditional BPS high school as the qualified admission high school, would have been equally likely to attend a college had they attended a traditional BPS high school, and would be equally likely to graduate from their college in six years had they attended a traditional BPS high school.

        1. “unless most of those students would have left BPS if the qualified admission high schools did not exist”

          But what would be the effect if they were proportionately distributed among non-selective BPS and charter schools? Or disproportionately distributed to charter schools? In either case that would likely lower total BPS high school and college graduation rates and increase charter school graduation rates would it not? And aren’t those scenarios more likely than that those students would just enroll in BPS non-selective schools?

          1. Stephen,

            If they went to non-selective PBS schools, the high school graduation rate, college enrollment rate, and college graduation rate for PBS would be no different than reported now. If they all went to charter or moved out of district, the high school graduation rate, college enrollment rate, and college graduation rate for BPS would be lower than the one reported now. I think that if you closed the selective schools some of the students would enroll in non-selective PBS schools, some would go elsewhere. So if your interested in asking the question “What would BPS high school graduation rates, college enrollment rates, and college graduation rates look like if there were no selective admission high schools in the district”, I think the answer is somewhere between the figure calculated when you include students at the selective schools and the figure calculated if you drop the selective schools. I am not sure why you would be especially interested in asking that question, however.

            As I said before, the far more important selection bias issue is on the charter side. It is very likely that there are unobserved student level variables like family commitment to education that differ between charter students and traditional district school students.

  11. I don’t know about Boston, but the attrition rates in DC leave no doubt whatsoever that DC charter schools have NOT solved the dropout problem.

  12. “I am not sure why you would be especially interested in asking that question, however.”

    I’d only be inclined to ignore that particular factor if I thought it had a statistically insignificant impact. If, say, 15%, 20% or more of Boston BPS HS seniors attend selective schools, I would be loath to assume a statistically insignificant impact.

    I wouldn’t argue with your contention that the “more important selection bias issue is on the charter side.” But I’m less confident than you may be in how to properly weight the various factors that distinguish the apples and oranges.

    Bottom line, do you think the data presented above from the Boston Opportunity Agenda tends to confirm or counter the Edushyster/Golann hypothesis that charter school students are relatively poorly prepared to complete college successfully? Or does that data not helpfully contribute to an analysis of this issue?

    1. Stephen,

      The data is suggests that you might want to look a little deeper, but you can’t come to any conclusion about the impact of different types of schools until you control for the impact of different types of students that attend those schools.

      Let me give you a personal example to illustrate the issue. The elementary school my children attended is one of the poorest in the city with over 64% of the students categorized as economically disadvantaged, yet the standardized test scores of students at the school are as strong as schools with many fewer disadvantaged students (the district average was just below 40% economically disadvantaged) and students there often went on to college. Does that mean that there is something about the way that particular school approaches education? Maybe.

      Let us look a little deeper at the student population. Many of the non-economically disadvantaged students at the school are the children of a research one university faculty member. A significant fraction of the economically disadvantaged students are the children of graduate students at a research one university and would soon not be economically disadvantaged and soon a child of at least one parent with an earned Ph.D.. Does that matter in understanding the test scores of that school? Absolutely it does.

  13. That all makes good sense to me, teachingeconomist.

    Perhaps it would help to look at the above graduation data in conjunction with some matched pair data that strives to compare academic success among applicants to non-BPS Boston charter schools, some of whom do, and some of whom do not, win the luck of the draw. If it were the case that both groups performed similarly in tests of reading and arithmetic, and at much more advanced levels than their non-selective BPS school peers, it would seem especially plausible that any higher odds of Commonwealth charter school 9th graders graduating from college than BPS peers might properly be ascribed to factors other than the charter schools’ impacts, such as levels of family commitment to education.

    As for anecdotal evidence, the notion that students attending schools like MATCH and KIPP are repressed to the point that they lose adequate capacity to “take initiative, be assertive, and negotiate with authority” surely doesn’t correspond well with my experience with the gal upstairs and lad next door whose healthy levels of afterschool teenage ebulliance seem rather more irrepressible than Edushyster and Golann may imagine. One of them a short while ago knocked on my door and successfully negotiated my wifi password despite the fact that the device has been regularly reaching its limit of attached devices. Ah, well…

    1. Stephen,

      Matched pairing of students is certainly one way to get around the selection problem. The CREDO study of charter schools (https://urbancharters.stanford.edu/download/Urban%20Charter%20School%20Study%20Report%20on%2041%20Regions.pdf) uses this approach.

      If you are interested in how economists think about education, there is a nice review paper that has just been released (http://ftp.iza.org/dp9885.pdf). The basic conclusions the author draws from the literature are 1) teachers are the most important factor effecting student outcomes in a school, 2) investment in early years of child development is the most productive investment that society can make, and 3) a structure with strong accountability, school autonomy, and centralized testing at the end of compulsory schooling are important.

      1. Teaching Economist – you might be interested in the on-going debate about CREDO’s methodology w/ respect to its study of urban charters in particular. Andrea Gabor, a professor at Baruch and a friend of mine, hired a respected statistician to plumb the depths (and engaged in a lengthy back-and-forth with CREDO head Mackie Raymond). What she found is fascinating and troubling… https://andreagabor.com/2015/04/28/new-credo-study-new-credibility-problems-from-new-orleans-to-boston/

        1. I’ll look forward to what teachingeconomist may say on this subject. For myself, looking at the Andrea Gabor article in conjunction with the 2013 National Charter School Study Technical Appendix
          https://credo.stanford.edu/documents/NCSS2013_Technical%20Appendix.pdf
          and the CREDO reply to Maul and Gabor http://credo.stanford.edu/pdfs/CREDOResponsetoMaulandGabor3.pdf
          I glossed over much material regarding New Orleans and about some charter schools being selective, none of which seemed relevant to Boston. Found it difficult to find much of anything persuasive in respect to any potential overall significant anti-traditional public school bias.

          Material like this in the Gabor critique falls short of devastating to the CREDO results:
          “Further, while the study matches the test scores of, on average, five public-school students to each charter student, not all the scores are exact matches. Yet, the study never discloses either the percentage of ‘inexact’ matches nor whether these inexact matches are on the high, or the low side. If the majority of ‘inexact’ matches are below those of their charter twins, it would not be surprising that, by the end of the study, the public-school virtual twins would still be testing lower than their charter counterparts. (The converse, of course, is also true. But we don’t know, because the study doesn’t say!)”

          In another spot, Gabor writes: “The study also relies on a controversial methodology that the researchers used in past CREDO studies and that has been critiqued here and here and here.”

          I only checked out the link for the middle “here”. It’s an article by Dr. Caroline Hoxby that argued that CREDO methodological errors had systematically biased _downward_ the effects of charter schools. And it seems to have been effectively refuted by CREDO.

          The CREDO researchers themselves mention in their studies that their methods resulted in some oversampling of the earlier years for TPS students, who were on a downward trajectory, and doing that may have lessened the positive impacts they found for the charter schools:
          “This is likely due in part to a slight downward trend among TPS achievement in our data, (also noted among VCRs in the body of the report). This implies that the exclusion of TPS records from later years, or rather the over sampling of early observations in TPS, biases down the estimated charter effect by biasing up the TPS counterfactual.”

          I can’t say, though, that I found CREDO’s reply here to Dr. Maul at all convincing:
          “We have examined the unmatched students and find that their matches fail not because of their prior academic performance (especially in communities with small fractions of all students enrolled in charter schools), but because they present unusual combinations of the remaining factors. Of these factors, charter school students repeat grades far more often than students in traditional public schools, and this causes many of the match disqualifications that arise.”

          That seems a seriously inadequate response to Maul’s concern that: “the VCR technique found a match for only ‘greater than 80%’ of charter students (Technical Appendix, p. 8), meaning that close to 20% of charter students were excluded from the study. In [an earlier CREDO report] it was indicated that the excluded students had average scores 0.43 standard deviations lower than the average of the included students, introducing the potential for bias due to systematic exclusion of lower-performing students”

          But, as a duffer in the field of statistics, I am inclined to suppose that overall, any biasing pro or agin TPS/charter schools roughly balances out, given my inability to locate any critiques that demonstrate otherwise.

        2. Thanks for the link. I am not sure that an MBA makes a person a respected statistician, but lets see what he says.

          His first concern, that CREDO only compares students at charter schools to students at the public schools that send students to charter schools does not seem to me to be a big problem. First, no one disputes that there are fine public schools that are the match for any charter, but that is not relevant if you can not meet the admission requirements for those fine public schools. It seems like advising someone to pick a Chrysler 200 over a Honda Accord because Chrysler Fiat has a different line of cars, Maserati, that are really nice and stylish. Second, there are likely unobserved variables that impact education that are correlated geographically. By comparing students from the same kinds of low income neighborhoods, the CREDO study minimizes the impact of those unobserved variables on the study.

          The second criticism, that virtual twins to charter school students need not go to a school in the NOLA, is a limitation of the real world that we face. The perfect way to do this would be to send all the students through traditional public school,rewind the clock, and send those same students through a charter school. Empirical social science does not have that ability, so we do what we can. It is not clear to me that this causes the study to be biased in favor the charter school or biased in favor of the public school.

          For the third problem, I know there is the universal belief among opponents of charter schools that charter schools do some screening of students for admission. If there was ever any proof of significant screening, it has yet to be presented. I also think that it is very possible that the parents of students who are doing poorly in the public schools are the ones that are most interested in sending their students to charter schools. If this is the case, the selection bias discussed here works in favor of the public schools by offloading the students who are having the most problems in the traditional public school system to charter schools. The one charter school in my town, for example, was specifically established to take the students who were doing poorly in the traditional public high school out of the traditional public high school and put them in an environment that better suited those students.

          Survivor bias is certainly an issue in any study, but as with the previous issue, it is not clear which way the bias goes. Students who drop out of public high schools (My state, like many others, only requires education until they reach the age of 16) are presumably weaker students who no longer get counted as public school students.

  14. […] Today, largely white philanthropists pour money into charter schools that place a high value on order, efficiency and discipline, serving children who are almost entirely Black and Latino/a. These wealthy elites are increasingly […]

Comments are closed.