House Speaker Paul Ryan told members of the Republican majority that he would no longer defend Donald Trump and will focus instead on retaining the party’s congressional majority, multiple news outlets reported on Monday.

Ryan did not withdraw his endorsement, but his decision to essentially abandon Trump’s campaign is the latest fallout following the release of a 2005 “Access Hollywood” video on Friday in which Trump makes lewd and vulgar comments about women.

According to the New York Times, some House members reacted angrily to Ryan’s comments, seeing it as a sign that he was giving up on the Trump campaign.

The NBC News/Wall Street Journal poll, taken after the release of the tape but before the debate, showed that Hillary Clinton garnered 46% support among likely voters in a four-way matchup. Trump had 35%.

Trump’s senior communications adviser, Jason Miller, said on Twitter, “Nothing’s changed. Mr. Trump’s campaign has always been powered by a grassroots movement, not Washington.”

Ryan was to campaign with Trump on Saturday in an appearance in Wisconsin. But in the wake of the leak of the 2005 video, Ryan issued a statement saying that Trump would no longer attend — essentially disinviting him. The Trump campaign then said that running mate Mike Pence would attend, but that too was canceled.

Trump himself sent out  tweet bashing the House speaker.

“Paul Ryan should spend more time on balancing the budget, jobs and illegal immigration and not waste his time on fighting the Republican nominee,” Trump said on Twitter.