LITTLE ROCK, Ark. (AP) — On a hot Friday last July, a parolee was mowing a lawn in a small cul-de-sac on the west side of the city when he stopped to ask for a glass of water.

The 70-year-old widow whose yard he was mowing told him to wait on her porch. Instead, she said, he jerked the storm door open, slammed her against the wall, forced her into the bedroom and raped her. The parolee pushed her with such force, she said, that her front teeth were knocked loose.

Trending

Suicide Prevention Week: A Reminder of the Mental Health Crisis Confronting Black Californians

Then he went back to mowing the lawn.

Milton Thomas, 58, said he’s not guilty. His trial is set for March.

Thomas has been in and out of Arkansas prisons since 2008 for nonviolent crimes, including check fraud. After he got out in November 2013, the state predicted he was a low risk to commit another crime, Thomas said, and assigned him the least amount of supervision.

His low-risk prediction would have been calculated based on answers to a lengthy questionnaire, the latest tool among the nation’s court systems to try to predict the likelihood that an offender will commit a crime again.

Across the country, states have turned to a data-driven movement to drive down prison populations, reduce recidivism and save billions of dollars. One emerging practice is the use of risk-and-needs assessment tools, which are questionnaires that explore issues beyond criminal history. They are based on surveys of offenders making their way through the justice system.

In a country with the highest incarceration numbers in the world, these questionnaires are a pillar of a new effort to get people out of prison. Repeat offenders are a major driver of bloated prison populations. But an Associated Press examination found significant problems with the surveys, which are used inconsistently across the United States, sometimes within the same jurisdiction.

Supporters cite some research, such as a 1987 Rand Corp. study that said the surveys can be up to 70 percent accurate in predicting the likelihood of repeat offenses, if they are used correctly. Even the Rand study, one of the seminal pieces of research on the subject, was skeptical of the surveys’ effectiveness.

It’s nearly impossible to measure the surveys’ impact on recidivism because they are only part of broader efforts.

These assessment surveys, used for crimes ranging from petty thievery to serial murders, come with their own set of risks.

Many rely on criminals to tell the truth, though jurisdictions do not always check to make sure the answers are accurate.

The surveys are clouded in secrecy. Some states never release the evaluations, shielding government officials from being held accountable for decisions that affect public safety.

Some have the potential to punish people for being poor or uneducated by attaching a lower risk to those with steady work and high levels of education.

The surveys can include more than 100 questions and explore a defendant’s education, family, income, job status, history of moving, parents’ arrest history, or whether he or she has a phone. A score is affixed to each answer and the result helps shape how the defendant will be supervised in the system.

“How easy would you say it is to acquire drugs in your neighborhood?” a convicted thief could be asked.

“How many prior sex offense arrests (with force) as an adult?” a sex offender could be asked. A bubbled answer sheet lists options ranging from zero to more than three.

The idea is to use data about past offenders to predict what current defendants with similar backgrounds might do when released from prison. A major push is to free up parole and probation officers to focus on those more likely to reoffend, instead of lower risk inmates.

“It is a vast improvement over the decision-making process of 20, 30 years ago when parole boards and the courts didn’t have any statistical information to base their decisions on,” said Adam Gelb, director of the Public Safety Performance Project at the Pew Charitable Trusts, which is working with the U.S. Justice Department on changes to the prison systems nationally.

Cost savings are significant. In Arkansas, a prisoner costs the state $63 a day. A parolee costs $2.

But the sexual assault case in Little Rock also points to shortcomings.

Before Thomas was released, the parole board assessed him as a high risk to commit more crimes. But a second risk assessment, conducted by the state’s community supervision agency, found him to be a low risk, Thomas told the AP. Thomas said he has no recollection of answering questions from the lengthy survey.

Not only were Thomas’s contradictory assessments never explained, but after he was arrested on charges of raping Diana Miller, he was assessed again. The AP doesn’t identify victims of sexual assault, but Miller, now 71, agreed to be identified by her middle and married names because she said it is important for her story to be told.

Stunningly, Thomas’ risk-rating decreased after the rape charge.

The reason for the difference? The state couldn’t figure out how old Thomas was when first arrested, according to Solomon Graves, a member of the state parole board.

In June 2013, Thomas said he wasn’t yet 25 when first arrested, a high-risk factor, Graves said. The following year, Arkansas criminal history reports listed Thomas as having been in his late 30s when first arrested, so Thomas’s risk-rating was lowered. In a letter to the AP, Thomas said he was 19 when he was first arrested.

The calculations for determining probation conditions are often just as bewildering.

In February 2014, an Arkansas consultant said a majority of male parolees and probationers were classified as low risk by the state’s Department of Community Correction but 46 percent were rearrested within 18 months — “creating a potentially dangerous situation for public safety,” consultant JFA Associates LLC concluded.

“Virtually any recidivism reduction plan can be helpful,” said U.S. Rep. Bobby Scott, D-Va., who favors changes in prison policies.

Scott co-sponsored legislation in the last Congress that called for using risk assessment surveys for the federal prison population. “I think you ought to have some assessment and do the best you can and keep updating it based on the research. But you ought not be afraid of a system that’s working on average because of one anecdote.”

___

“Garbage in, garbage out”

Jurisdictions use dozens of different surveys that vary in the kinds of questions asked and how they are used. The Justice Department is helping bankroll this movement by providing millions of dollars to help states develop and roll out changes.

“We really consider them to be a cornerstone or a foundational piece of what we can accomplish,” said Ruby Qazilbash, associate deputy director for policy in the Bureau of Justice Assistance.

The goal is ultimately to save money, since states and counties spend some $92 billion a year on corrections.

But in most cases, the surveys can only work if the rest of the judicial system is working properly.

In September, that gap played out in a juvenile courtroom in Fort Lauderdale, Florida.

There, a teenager faced sentencing for a charge of aggravated battery with bodily harm stemming from an incident one year earlier over a neighbor’s dog. The neighbor let his dog out without a leash, and the barking dog ran toward the teen’s father. The teen’s dad told the dog to go away, upsetting the dog’s owner. Curses and threats followed. The dog owner walked away, then turned back toward the teen’s dad and took his shirt off. When the teen’s father called the police, his son joined the fracas and stabbed the dog owner, police said.

The teen was given a 128-item questionnaire called the Positive Achievement Change Tool, or PACT, standard for juvenile cases in Florida. He was deemed a low risk to commit further crimes. The AP does not generally name juveniles charged with crimes.

The survey asked about his history of school expulsions, views on the value of education, whether his parents had been arrested and whether his friends had committed crimes.

One answer helping earn him the low score: getting credit for having a good group of friends.

But when the prosecutor, Maria Schneider, dug into the defendant’s background, she found that his circle of “friends” had attempted a drive-by shooting of his house because they said he owed them money.

The assessment “is completely flawed,” Schneider said in court. “They were obviously depending just on the information this young man was providing himself,” she said.

She believed the teen was moderate to high-risk, worthy of detaining under a residential program. The judge sentenced him to such a program, where teens are supervised 24 hours a day, but not held behind bars. At low risk, as the survey suggested, he would have received probation.

The case is a perfect example of “garbage in, garbage out,” Schneider said in an interview. “I continue to see where the PACT does not reflect the reality.”

Corrections systems always have been plagued by inaccurate information, said Sean Hosman, founder of Assessments.com, the company that, in collaboration with Florida, fashioned the survey.

Even so, he said, the assessments are an improvement over past practices and help defendants get a fair shot.

“Too often,” Hosman said, “we would do a gut check or you’d use intuition … or assess them based on the color of their skin, or the fact that we knew their brother and they went through here before.”

In California, a public defender in San Joaquin County, Christine Kroger, said she is not seeing much improvement. She has had trouble even getting access to the surveys conducted on her juvenile clients.

“You feed all this information into a computer and it spits out what should happen to this child,” Kroger said.

In Kentucky, the surveys have helped decide which defendants should be released before trial, said Ed Monahan, Kentucky’s public advocate.

In his state, Monahan has seen a 3 percent increase in the number of defendants released before trial with “no adverse consequence to appearance or criminal behavior.”

These tools are also used by the parole board, but Monahan said he has been less impressed with the outcomes.

Of inmates who were eligible for parole and assessed to be a low risk of committing future crimes, only 60 percent are being released, Monahan said. “Millions of dollars that could be saved there — that’s not being done,” he said. The state’s parole board did not respond to questions about the discrepancy.

___

Questions of Potential Bias

The surveys have been largely confined to parole and probation decisions, but they increasingly are being used for sentencing.

One 2013 study said at least one court system in 20 states is using these questionnaires at some stage of sentencing. In some states, such as Michigan, the tools are used statewide.

Offenders in Michigan have been assessed since 2007 when they go to prison. Now the risk assessment is incorporated into a report given the judge before sentencing. Among the 135 questions:

—In the last 12 months before this incarceration, how often did you move?

—Was one of your parents (or parent figure who raised you) ever sent to jail or prison?

—In the last couple of years before this incarceration, how many of your friends/acquaintances had ever been arrested?

Sonja B. Starr, a University of Michigan law professor who wrote the 2013 study, said the surveys could punish people for being poor.

“They are about the defendant’s family, the defendant’s demographics, about socio-economic factors the defendant presumably would change if he could: Employment, stability, poverty,” Starr said. “It’s basically an explicit embrace of the state saying we should sentence people differently based on poverty.”

In a sense, the justice system has taken a page from the insurance industry, using data to assess risk.

“This is a way to put a blood pressure cuff on the guy and say, ‘This is what the data show. Now what are we going to do for the patient?'” said Thomas A. Powell, a forensic psychologist in Vermont, which uses an assessment tool called VASOR for sex offenders.

The Justice Department’s position on the tools has been inconsistent. On one hand, it’s funding them, and on the other, the department is putting on the brakes.

“Criminal sentences must be based on the facts, the law, the actual crimes committed, the circumstances surrounding each individual case, and the defendant’s history of criminal conduct,” U.S. Attorney General Eric Holder told the National Association of Criminal Defense Lawyers in August. “They should not be based on unchangeable factors that a person cannot control, or on the possibility of a future crime that has not taken place.”

The Justice Department cautioned the U.S. Sentencing Commission, which sets national policies, about relying too much on the new surveys. But jurisdictions are sorely tempted to test new systems that could save billions.

For instance, North Carolina could save $560 million by 2017, a Justice Department report concluded. Between 2011 and 2014, the North Carolina prison population decreased by more than 3,000 people, according to the state. Overhauls, including the use of risk assessments, have saved the state nearly $84 million, and it plans to use $32 million of those savings for community treatment programs.

___

A low-risk serial killer

Texas has been widely praised for overhauling its prison system — including use of surveys — in ways that helped drive down prison populations by more than 9 percent between 2007 and 2012, according to the Texas state figures.

The case of Darren Vann, however, exposed deadly flaws.

In 2004, Vann, then 33, grabbed an ex-girlfriend in a chokehold, doused her with gasoline, threatened to light her on fire and dragged her through an alley, according to court documents. He spent about one month in jail. His charge was downgraded to a misdemeanor, and he was sentenced to a year in jail, which he served on probation. The original presiding judge in his case was kicked off the bench about six months after Vann’s sentence because of a backlog of cases, the state’s supreme court said.

Vann violated his probation in 2005 and was to spend 12 days in jail. But it was unclear whether he did. Court records show a hearing scheduled for November of that year was canceled.

Four years later, Vann landed in jail again, this time for raping a woman in Austin, Texas.

When he completed his sentence in 2013, Texas performed an assessment on Vann, as it does on all sex offenders. Vann refused to take the three surveys that Texas typically uses. An employee filled out a 10-question survey, called Static-99R, to score Vann’s likelihood that he would commit another sex crime. Vann scored a 1 — the lowest risk — on a scale of 1 to 6. His assessment predicted only a 3.8 percent chance he would commit a future sex crime in the next five years.

But that was only if the answers to the questions were actually true.

One of the questions asked whether Vann had a previous sex offense. Because Texas had no record of his 2004 crime, a criminal history check noted in the survey indicated he had a clean record.

Had Vann scored higher on the Static-99R, Texas would have sent postcards to the community where he was living — if he was living in Texas. But Vann moved to Indiana after his release.

About one year later, 19-year-old Afrika Hardy was found dead in a Motel 6 bathtub 20 miles southeast of Chicago.

Vann had found her through an online escort ad, calling himself “Big Boy Appetite.”

According to police, Vann said he had sex with Hardy on Oct. 17 then things got rough. Wearing white gloves, he strangled Hardy with his hands and an extension cord, he told investigators. Vann was arrested after police traced him through cellphone records.

In custody, Vann confessed to the killing, and also told police he murdered six other women, directing them to abandoned homes in Gary, Indiana, where the bodies lay. Under a pile of tires and teddy bears at one home was the body of 35-year-old Anith Jones, who had gone missing earlier that month.

It was unclear how many of the women had died after Vann had taken the Static-99R.

Experts say that the risk-and-needs assessment tools should be evaluated every few years. Texas, which started using the instrument that evaluated Vann in 2000, is just now doing this, state spokesman Jason Clark said.

“States and localities and all the jurisdictions that are working on risk assessment right now, they’re in different places with respect to their ability to implement a good risk assessment,” Gelb said. “But it’s absolutely critical that they do.”

___

Crowded prisons

With more than 2.2 million people in jails and prisons across the country, the U.S. leads the world in incarceration, ahead of both China and Russia, according to the International Centre for Prison Studies. Successful changes in some states means lawmakers are considering changes for the federal system, too.

“We know that our prisons are overcrowded, and pretty much everyone agrees that recidivism, the percentage in which people repeat crimes, is way too high,” said U.S. Sen. John Cornyn, R-Texas. He cited a national recidivism rate of about 68 percent within three years, a figure from a 2014 Justice Department estimate of prisoners released in 30 states.

Cornyn and U.S. Sen. Sheldon Whitehouse, a Rhode Island Democrat, reintroduced legislation this month that would require the use of a risk assessment tool in the federal prison system.

“We’ can’t simply incarcerate our way to public safety,” Whitehouse said. “We have to be smart.”

There is a separate bill in the House that would also require risk assessments for the federal prison population.

Which risk assessment survey would be used and how it would be applied is a subject of debate.

Despite problems with the tools in Arkansas, the Justice Department in 2013 heralded the changes the state made to its corrections system.

Arkansas state Sen. David Sanders, however, said the changes have made the community less safe because it pushes responsibility to the parole and community supervision systems.

“Our parole system is absolutely dysfunctional,” said Sanders, an outspoken opponent of some of the changes in corrections policies made in Arkansas. He is pushing for more transparency about the use of the survey tools.

Even those who advocate for surveys acknowledged that they are imperfect.

Graves, the Arkansas parole board member, said over time the tools will become more dependable.

“We’re never going to have a 100 percent predictive tool,” he said. “We’ll never be there.”

___

Associated Press writers Alex Sanz in Atlanta and Tom Coyne in Crown Point, Indiana, contributed to this report.

___

Read the risk assessment for Indiana’s alleged serial killer here:

https://www.documentcloud.org/documents/1672461-vann-static-99.html

___

Read a letter from Arkansas parolee Milton Thomas to The AP here:

https://www.documentcloud.org/documents/1675187-thomas-12-15-letter-1.html

___

Follow Eileen Sullivan on Twitter at www.twitter.com/esullivanap