Harvey Weinstein has lost another defense lawyer, just three months before he is set to go on trial for rape and sexual assault.

Attorney Jose Baez told the judge overseeing the case that Weinstein had taken actions that made it difficult to represent him, according to a letter obtained by the New York Post. Baez also complained that Weinstein was ignoring their fee agreement.

Weinstein is set to go on trial on Sept. 9. Baez’s co-counsel, Ronald Sullivan, withdrew from the case in May, citing a conflict with his teaching schedule at Harvard. He had faced blowback for his decision to represent Weinstein, and was forced to step down as a faculty dean.

“When one of the team members left the case due to the personal pressures that were placed on him, it removed a critical component to the team and legal strategy, which was not yet being filled,” a Weinstein spokesperson said in a statement. “Mr. Weinstein will shortly be welcoming a team that will fulfill each of the essential components needed to properly serve his due process and his rigorous defense.”

Weinstein’s trial in Manhattan was supposed to begin on June 3. But in April, Baez won a postponement, saying he needed more time to prepare. It is not yet clear if Weinstein’s new team will be ready to go in September, though it would not be surprising if the change led to another delay.

Baez is a celebrity attorney who was best known for his successful defense of Casey Anthony, the Florida mother accused of killing her child. He also represented NFL star Aaron Hernandez and Rose McGowan, who is one of Weinstein’s many accusers.

Baez took over for Ben Brafman, the New York defense attorney who handled the Weinstein’s case for a little more than a year. Brafman left the case in January following disagreements over strategy.

Weinstein faces five charges of rape and sexual assault, and could face life in prison if convicted.