Speaking with the Times on Monday, Baldwin addressed the allegations against Toback, and said, "In all the time I've known Jimmy, I never had one conversation about his sex life."
"I’d always heard Jimmy was peculiar. I remember years ago, with [the movie] The Pick-up Artist, Jimmy had this reputation 20, 25 years ago or more of hitting on [women], going up and saying provocative things to them," Baldwin claimed. "I'd heard of that. But in my time with Jimmy, it was absolutely, positively business… I never had one conversation with Jimmy about his sex life. Never."
The actor received an honor from the The Paley Center for Media on Thursday, which included a discussion with Tonight Show announcer Steve Higgins, according to The Hollywood Reporter.
Baldwin addressed the spate of sexual misconduct scandals that have been rocking Hollywood in recent weeks, and admitted to having treated women badly at various points in his career.
"I certainly have treated women in a very sexist way. I’ve bullied women. I’ve overlooked women. I’ve underestimated women," Baldwin said. "Not as a rule, [but] from time to time, I’ve done what a lot of men do, which is … when you don’t treat women the same way you treat men."
"I'm from a generation where you really don't, and I'd like that to change," he continued. "I really would like that to change."
The 59-year-old star went on to say that sexism, sexual harassment and gender bias in the workplace -- especially in Hollywood, but also across the board -- has far-reaching ramifications on the industry's viability.
“I think it’s important for us to try to make the workplace and beyond not only comfortable and right and fair and appropriate, but as productive as well," he explained. "A lot of what we’re dealing with within this issue is hurting our business. It’s making it less productive."
The 30 Rock alum also addressed whether or not he was aware past speculation and rumors regarding some of the high-profile men who have since been publicly accused of sexual misconduct -- such as Harvey Weinstein, Brett Ratner, Kevin Spacey and numerous others.
"I knew of certain things, that there were rumors of things happening to people, but I didn’t necessarily know the scope, when you hear the hundreds and hundreds of women who are complaining about this… It’s been a very eye-opening experience for me personally," he shared.
"We've got to be vigilant in a new way to make sure that everybody is comfortable and that we get the job done together that we're there to do," he added.
One of the numerous Hollywood figures to face allegations of sexual harassment and misconduct is director James Toback, who collaborated with Baldwin on his last two films -- The Private Life of a Modern Woman, in which the actor had a small role opposite Sienna Miller, and the documentary Seduced and Abandoned, which Baldwin and Toback essentially created together.
In October, the Los Angeles Timespublished a report featuring allegations from over 30 women, which described an alleged pattern of Toback's intimidation and inappropriate behavior, including lewd comments and touching. The Times later reported that the number of accusers had grown to over 320.
Toback, 72, denied the allegations to the Times, and said he either had never met or or barely recalled meeting the women making the claims.