In a rare confrontation, police and anti-Wall Street protesters clashed Sunday over a wood structure erected overnight on McPherson Square, blocks from the White House, and more than 20 were arrested as of 9 p.m.

On the previous night, members of the Occupy D.C. movement started to put up the two-story structure on the square near a tent city where the protesters have been camping out since October, and nearly finished the job by 4 a.m. (0900 GMT) Sunday. But before they could put a roof on it, police came in at 7 a.m. (1200 GMT), and ordered the protesters to tear the structure down, citing the lack of appropriate building permit.

The police came three hours later, according to Kyle Szlosek, a protester from Maine, who witnessed the day's developments on the square. Police cars blocked the area while officers were lining up K Street, where the square is located.

U.S. Park Police also sent in officers on horseback in case of a clash. A SWAT van and other emergency vehicles were also spotted nearby.

In the afternoon, police began installing metal barriers around the structure, and a building inspector deemed the structure as "dangerous" and ordered it to be torn down. But some protesters holed up inside the structure.

At the time, other protesters tried to enter the structure, leading to a scuffle with police. Szlosek said one protester got his head "thrown onto the ground" by police, and another one was elbowed straight on the throat by an officer.

Police issued warning for three times before forcibly removing protesters from the structure and arresting them. They moved in on the structure by dusk, and pulled at least eight people out of the base of the structure, and proceeded to remove another six who were on the top of it.

"A woman went through the structure, got pulled by the police. One guy went over to hug her, the police forcibly removed him, threw him on the ground, four officers jumped on top of him, and then they got on his head and pushed it to the pavement," said Szlosek. Witnesses put the total number of the arrested at above 20.

The clash slowly turned into a standoff as protesters loudly chanted slogans and read from the U.S. Constitution as the police used a cherry picker, a type of aerial work platform with a bucket at the end of a hydraulic lifting system, to remove those on top of the structure. Flood lights lit the area, and officers on top of the structure filmed the protesters below and the process of removing those on the structure.

The last man was removed by the police before 9 p.m.(0200 GMT Monday) , and he tried to cling himself to the structure, as protesters shouted supporting slogans and demanded speaking with a commanding officer.

Dick Gregory, an influential comedian and activist, came to the park as the police tried to remove the last man from the structure, and denounced the police's actions.

"When you talk about making the system work right," said Gregory, "that scares them."

Protesters said they built the structure to help those camping on the square to get through the winter, and they were planning to install an eco-friendly heating system inside. Szlosek argued they were allowed to build temporary structures, and the two-story building was in fact a non-permanent one, and it didn't break any laws.

Different from anti-Wall Street protests in other parts of the United States, protesters in Washington D.C. have maintained a relatively cordial relationship with authorities, and the confrontation Sunday marked the first large-scale scuffle between them and the police.

Szlosek said the confrontation injected new vigor into the movement, as "there are tons of people who never walked into this plaza" who were there witnessing the event.

He also acknowledged the presence of international media on the square. "People outside of our country really want to know what the people, the citizens of America are actually trying to do with our government, which has been so tyrannical around the world," said Szlosek. "We want finally shut it down and work on peace... we are going to let the people stand for once."

Despite the approaching harsh winter, Szlosek said they are in for the long haul. As the police were busy removing protesters, some in the tent city on the plaza began decorating a Christmas tree.