As soon as the New York Times story about Bill O’Reilly’s sexual harassment settlements hit on April 1, Color of Change began to mobilize.

The nonprofit African-American civil rights group sent an email blast to its 1.2 million members, calling on them to help ramp up a campaign to pressure advertisers to pull money out of Fox News’ “The O’Reilly Factor.”

The campaign was remarkably effective, hastening the withdrawals of more than 60 major advertisers from the top-rated hour in cable news. The success that Color of Change enjoyed was rooted in a previous effort to target O’Reilly over his racially charged statements, and the two-year battle COC waged against former Fox News personality Glenn Beck.

This time around, the factor that spurred sponsors to bail on “The O’Reilly Factor” was a number: $13 million, the value of O’Reilly’s settlements with five women since 2004, per the Times report.

“The settlements were particularly challenging for any corporate advertiser to defend,” Rashad Robinson, executive director of Color of Change, told Variety. “We asked them ‘Would they allow their own employees to get away with that?’ We said that the money they were paying for ‘O’Reilly Factor’ ads was actually paying for these settlements. It was hard for corporations to defend.”

Color of Change knew who to target and how best to reach “O’Reilly Factor” advertisers like Mercedes-Benz because of its 2015 campaign against the show. Color of Change members were outraged by what the org termed O’Reilly’s “fabricated” on-air comments about being attacked by black people while he was covering the Los Angeles riots in 1992.

Color of Change built a network of contacts and relationships during that effort that proved invaluable after the sexual harassment story surfaced. The focus is not just on CEOs but “everywhere from the PR department to the diversity department to employee resource groups,” Robinson said. Those efforts include recruiting employees of a given advertiser to lodge complaints about supporting O’Reilly or Beck, or other targets. Color of Change takes credit for convincing top brands to pull millions of dollars in sponsorship coin from the Republican National Convention in Cleveland last July.

The COC campaign against Beck ran nearly two years and contributed to his parting with Fox News in 2011. O’Reilly was a harder target, given the success of his show and the strong backing he had from the highest levels of Fox News and 21st Century Fox patriarch Rupert Murdoch.

“We built our infrastructure and had an understanding of the landscape of (‘O’Reilly’) advertisers, but we didn’t the muscle. We got some traction with advertisers in the past, but not the type with the legs to do the damage you know you need to do,” Robinson said. COC members in various cities were organized into viewing teams to document all of the local and national advertisers in the show. Robinson cited the work of liberal media watchdog group Media Matters for America as a contributor to COC’s efforts.

In the past, “O’Reilly Factor” advertisers often would quietly pull spots for a short period — often motivated by Color of Change’s online targeting campaigns, including videos taking aim at a company’s advertising decisions. Members of the group are adept at flooding the social media feeds of advertisers with protest messages.

But most of the time sponsors would return after controversy fades. With the harassment allegations, “O’Reilly Factor” advertisers “recognized that they couldn’t go back,” Robinson said. Mercedes-Benz was the first big name to pull out, starting a domino effect of advertisers that took the additional step of publicly acknowledging their decision.

Color of Change was preparing online videos calling out specific “O’Reilly Factor” advertisers but “corporations were dropping out so quickly that as we were developing our ad we would hear that they’d already dropped.”

“We had (companies) calling us to let us know they were leaving,” he said. “It wasn’t ‘Let’s do a conference call next week’ anymore.”

Color of Change worked with women’s advocacy group UltraViolet and the National Organization for Women in organizing anti-O’Reilly rallies outside Fox News headquarters in Midtown Manhattan. The same coalition had teamed with the org behind the Women’s March on Washington for what they hoped would have been a larger rally planned for April 24, the day O’Reilly had been scheduled to return to his show after a two-week vacation. That event, which has now been scrapped, was scheduled to feature an appearance by one of three African-American Fox News employees who filed a racial discrimination suit against the network earlier this year.

COC’s biggest asset is its feet on the street — an active group of more than 250,000 people who can be counted on to make phone calls, send emails, be vocal on social media, and show up at rallies. “Our email (list) allows us to move people up the ladder of activism,” he said. “It doesn’t just last one day.”

Based in New York, COC is funded by small donations from members, larger donations from individuals active in civil rights and social justice, and by grants from foundations including the Ford Foundation and Kellogg Foundation. The org does not accept money from corporations or from government entities, Robinson said.

The ouster of O’Reilly amounts to a big win for COC and its allies, but it does not mean the end of the groups’ focus on Fox News and its parent company, 21st Century Fox. COC was among the entities that filed a petition critical of Fox with U.K. regulators now reviewing Fox’s $14 billion bid to buy out the rest of satcaster Sky — a pressure point for the company that is far beyond the halls of Fox News.

“Fox has to clean up the racism and sexism more broadly,” Robinson said. “We are working for a less harmful and less hostile world. From our perspective we hope that Fox recognizes that it has to be better. We are not holding our breath that this represents some big sea change, although it is incredibly important.”