A federal report on Washington, D.C.’s school voucher program could be misinterpreted to show the program isn’t working, when in fact, it’s still too early to tell what long-term gains enrolled students will achieve.

The DC Opportunity Scholarship Program lets families who “win” a lottery decide whether to accept scholarships that would give them a choice regarding where their child attends school. The scholarships can be used toward tuition at participating schools. All of the families in the program have an income at or below 185 percent of the federal poverty line, and more than 40 percent come from two of the city’s toughest areas.

The report from the Institute of Education Science, which is part of the U.S. Department of Education, shows that math test scores among students who accepted the scholarship are lower for a second year than students who entered the lottery but were not selected. There was no statistical difference in reading scores between the two groups.

Opponents of school choice programs have pointed to discrepancies in test scores for charter school students as proof that school choice programs don’t work. But choice advocates say the issue is more complicated, and the D.C. program can provide valuable insight.

“The goal is to provide these parents the option that wealthy families have always had: the best environment for their kids,” said Rachel Sotsky, executive director of Serving Our Children, which oversees the DC voucher program.

In the 2017-18 school year, 3,907 students applied for the voucher with 2,551 of those coming in new to the lottery. Scholarships were awarded to 1,653 students who came from families with an average income of $22,163. The students’ racial background shows that 78 percent identify as African American and 17.2 percent Hispanic. In the 2017-18 school years, scholarships up to $8,653 were available for children in kindergarten through 8th grade, and up to $12,981 for grades 9 through 12.

In the 2015-16 school year, 98 percent of the program’s 12th graders graduated from a participating school with 86 percent of graduates accepted into two- or four-year colleges or universities. Parental satisfaction was high – 87 percent were “Very or somewhat happy” with their child’s academic progress, according to information from Serving Our Children.

The voucher program was first created by Congress in 2004 and reauthorized in 2011. Scholarships are awarded annually with a priority put on getting students out of the lowest performing schools. Two of the high schools where many scholarship families live were prominently named in a study from the Washington, DC, Office of the State Superintendent of Education that criticized schools for giving students diplomas despite their not attending school. Some students at Anacostia and Ballou high schools missed as much as three months of classes but somehow managed to graduate.

Educators have criticized federal school reform programs for placing too much emphasis on standardized test scores and graduation rates without determining whether students have attained the knowledge and skills they need to succeed.

Dr. Patrick Wolf, Distinguished Professor of Education Policy and 21st Century Endowed Chair in School Choice in the Department of Education Reform at the University of Arkansas College of Education and Health Professions, co-authored a paper looking at whether standardized test scores matter.

“When you’ve graduated from a school nobody really pays attention to what your standardized test scores are,” Wolf said. “How far you go on the journey is really what’s important.”

Standardized tests “measure a slice of student knowledge with known levels of reliability,” according to the paper. In fact, it’s not uncommon for students who transfer into a more rigorous school through a voucher to experience a decline in scores. There is an adjustment period that can up to two years for students to fully understand what is expected of them in terms of classroom discipline, homework, attendance, and behavior.

The students who participated in the scholarship study are still within the early years of their program. Wolf said researchers and others pay attention to this early data because that’s all that is available, but he cautioned that they are not accurate predictors of student attainment for longer-term goals of graduation and college enrollment.