WASHINGTON (AP) — A prostitution scandal involving the Secret Service has grown in scope, with the disclosure that U.S. agents and military personnel had been with at least 20 women in hotel rooms before President Barack Obama arrived in Colombia for a summit with Latin American leaders.

Secret Service Director Mark Sullivan, facing questions on Capitol Hill about whether the escapades could have jeopardized the president’s security, said he had referred the matter to an independent government investigator.

WATCH THE LATEST UPDATE IN THE SECRET SERVICE SCANDAL

Sullivan said the 11 Secret Service agents and 10 military personnel under investigation were telling different stories about who the women were. Sullivan has dispatched more investigators to Colombia to interview the women, said Republican lawmaker Peter King, chairman of the House Homeland Security Committee.

“Some are admitting (the women) were prostitutes, others are saying they’re not, they’re just women they met at the hotel bar,” King said in a telephone interview. Sullivan said none of the women, who had to surrender their IDs at the hotel, were minors. “But prostitutes or not, to be bringing a foreign national back into a secure zone is a problem.”

Sen. Susan Collins of Maine, the ranking Republican on the Homeland Security Committee, said Tuesday that “20 or 21 women foreign nationals” were brought to the hotel. Eleven of the Americans involved were Secret Service, she said, and “allegedly Marines were involved with the rest.”

King said it appeared the agency actually had “really lucked out.” If the women were working for a terrorist organization or some other anti-American group, King said, they could have had access to information about the president’s whereabouts or security protocols while in the agents’ rooms.

“This could have been disastrous,” King said.

The growing scandal has become an election-year embarrassment for Obama, who has said he would be angry if the allegations proved to be true. The White House said Obama had confidence in the Secret Service chief.

“Director Sullivan acted quickly in response to this incident and is overseeing an investigation as we speak into the matter,” White House press secretary Jay Carney said.

Last Thursday, 11 Secret Service agents were recalled to the U.S. from Colombia and placed on administrative leave after a night of partying that allegedly ended with at least some bringing prostitutes back to their hotel. On Monday, the agency announced that it also had revoked the agents’ security clearance.

At least 10 U.S. military personnel staying at the same hotel were also being investigated for their role in the alleged misconduct.

Two U.S. military officials said they include five Army Green Berets. Both officials spoke on condition of anonymity about an investigation that is still under way.

One of the officials said the group also includes two Navy Explosive Ordinance Disposal technicians, two Marine dog handlers and an Air Force airman. The Special Forces Green Berets were working with Colombia’s counterterrorist teams, the official said.

The agents and servicemen were in Colombia setting up security ahead of Obama’s three-day trip to the port city of Cartagena for a summit attended by about 30 other world leaders.

People briefed on the incident said the agents brought women back to Cartagena’s Hotel Caribe, where other members of the U.S. delegation and the White House corps also were staying. Anyone visiting the hotel overnight was required to leave identification at the front desk and leave the hotel by 7 a.m. When a woman failed to do so, by this account, it raised questions among hotel staff and police, who investigated. They found the woman with the agent in a hotel room and a dispute arose over whether the agent should have paid her.

While the identities of those being investigated have not been revealed, Maryland Republican Senate candidate Daniel Bongino told The Associated Press on Tuesday that his brother, an agent who was on duty in Colombia, is “cooperating” with the investigation. Bongino, a former agent himself, insisted his brother was not a target of the investigation.

The Secret Service has insisted Obama’s security was not undermined by the incident, which happened before he arrived in Colombia.

In at least one of his briefings with lawmakers, Sullivan said he was calling on an inspector general to hold an independent review. Republican Sen. Chuck Grassley welcomed that news, saying an independent review “should help the agency regain some respect from the American taxpayers and from people around the world.”

The Secret Service did not immediately respond to a request for comment about Grassley’s account.

Copyright 2012 The Associated Press.