Esther Newberg has been a titan among book agents for nearly 40 years. But the co-head of publishing at ICM Partners still enjoyed a career first last year when she played matchmaker between two agency clients: Katie Holmes and author Annie Weatherwax.

Holmes optioned Weatherwax’s debut novel, “All We Had,” as the vehicle for her film directorial debut after a meeting with book department agents set up by ICM’s New York talent head Adam Schweitzer. The movie, penned by “The Fault in Our Stars” director Josh Boone, begins production in August.

“In 39 years, that never happened before,” Newberg says of being part of making a book-to-movie deal happen so quickly. It’s an example of the new spirit at the agency in the three years since ICM underwent a management buyout by 29 senior agents and execs.

By all accounts, the agency is a more harmonious place, but it is also in the precarious position of trying to rebuild at a time when its largest competitors have grown bigger, and command much greater market share. ICM’s fate in the next few years may well be a referendum on whether full-service mid-sized agencies can survive in the current environment or whether they will succumb to merge-or-die pressure at a time when industry insiders predict more consolidation.

ICM’s transition to its current partner-owned structure was messy — and public. The agency’s leaders clashed with the financial dictates of its then-majority shareholder, private equity firm Rizvi Traverse. There was infighting among camps of staffers, and a clash of agendas between ICM president Chris Silbermann and chairman-CEO Jeff Berg. The turmoil led to the departures of significant clients and agents.

ICM’s L.A.-based leaders include Chris Silbermann, Greg Lipstone, Kevin Crotty, Ted Chervin, Chris von Goetz, Eddy Yablans, Dan Donahue, Steve Levine and Rick Levy

Silbermann and others realized the relationship with Rizvi Traverse was untenable about three years into the 2006 union of ICM and their Broder Webb Chervin Silbermann agency. They spent the next three years navigating a buyout that would push Berg and Rizvi out by mid-2012.

“To have any successful business, you need an aligned leadership team that is rowing in the same direction with the same set of values,” says partner Ted Chervin. “We didn’t have that.”

There’s no question that ICM’s stature among the major agencies has fallen at a time when CAA and WME/IMG have bulked up considerably. And UTA has been on a growth spurt, elbowing past ICM as the No. 3 agency. Yet, ICM’s stronghold is its television department, and its New York-based book division still outshines its Hollywood rivals.

In hindsight, the biggest surprise is that ICM didn’t splinter. The tipping point came in late 2011, when contracts for key players — including Silbermann, Chervin and Eddy Yablans — came up for renewal. If Rizvi didn’t work out an exit strategy, those three and others would walk, leaving their investor with the shell of an agency.

ICM wound up swapping the lion’s share of its receivables and some older debt from the agency’s first 40 years in business to Rizvi in exchange for the latitude to shift to a partner-owned structure with no outside investors. The management buyout was financed without taking on any third-party debt, beyond some interest-free obligations to be paid to Rizvi over time.

“It was a grueling process, but it was a defining moment for us that made us stronger,” Silbermann says. “Every year (since the buyout) has been better than the last. That’s not to say that we haven’t had our growing pains, but the throughline is resoundingly positive.”

Now ICM faces the challenge of growing while living under more modest means, without the cushion of the same level of cash flow from past hits like “Two and a Half Men” and “The Big Bang Theory.” However, the success of NBC’s “The Blacklist,” which nabbed a $2 million an episode off-network sale to Netflix, marks a big win for the new-model ICM, which packaged the show.

The highest hurdle is rebuilding its diminished film business following a spate of losses capped by the departure of Berg.

ICM has sought to make up for lost time by adding more than 30 agents since the buyout, including indie film vets Bart Walker and Peter Van Steemburg. That kind of recruiting was impossible under the previous regime, where salaries were squeezed, and there was no ability to offer partner-level positions. To date, the number of partners has grown to 40, and the agency has about 475 employees, 175 of them agents.

New York-based toppers include Esther Newberg, Jennifer Joel, Adam Schweitzer and Sloan Harris

ICM has expanded so quickly that many new staffers from the New York and Los Angeles offices met for the first time in April, when the agency held its first company-wide retreat in years, at the Terranea resort in Rancho Palos Verdes, Calif.

“This is a highly entrepreneurial marketplace for talent right now,” says Eddy Yablans, a partner and ICM vet. “This agency is a great engine for generating opportunities because every department is working together. We’re not silo-ed.” Adds Sloan Harris, who co-heads the publishing department with Newberg: “I have lived through several different versions of this company, and this is the happiest and most collegial and most productive by a country mile.”

To help foster a spirit of collaboration, ICM leaders instituted everything from a weekly information-sharing meeting to summertime barbecues. That’s a far cry from the past.

“This was a place where everyone was in it for themselves, and you could behave very badly as long as you were making money,” says one longtime ICM rep.

Today, the ownership structure ensures that the agency will rise or fall on the strength of what the partners bring to the table every day.

“We took some short-term losses in losing people we didn’t think would fit in with the culture we are building here,” says Schweitzer, a 15-year ICM vet. “Now a win for a book agent is a win for a talent agent, and a win for a music agent is a win for a talent agent. We are all in this together.”

(Pictured at top: Ted Chervin and Chris Silbermann)