CULVER CITY, CA - JUNE 07: Comedian Don Rickles speaks the 40th AFI Life Achievement Award honoring Shirley MacLaine held at Sony Pictures Studios on June 7, 2012 in Culver City, California. The AFI Life Achievement Award tribute to Shirley MacLaine will premiere on TV Land on Saturday, June 24 at 9PM ET/PST. (Photo by Kevin Winter/Getty Images for AFI)

Don Rickles isn’t backing down from a joke comparing President Barack Obama to a janitor.

The octogenarian comedian drew controversy for a joke he told during a tribute to actress Shirley MacLaine last week, in which he said, “I shouldn’t make fun of the blacks. … President Obama is a personal friend of mine. He was over to the house yesterday, but the mop broke.”

According to Politico, an unnamed Rickles’ spokesman pushed back on the criticism, saying “it was a joke, as were the other comments Don made that night,” and adding, “anyone who knows him knows he’s not a racist.”