The Republican's about-face comes after nine black church members were gunned down, allegedly by a young white man who embraced the flag as a symbol of white supremacy.
"150 years after the end of the Civil War, the time has come," Haley said after rousing applause, surrounded by Democrats and Republican lawmakers. "That flag, while an integral part of the past, does not represent the future of our great state"
The flag has flown in front of the state capitol for 15 years after being moved from atop the Statehouse dome.
U.S. Sens. Lindsay Graham and Tim Scott, an African-American appointed by Haley, were standing with Haley during her announcement. When she finished, she hugged Scott and South Carolina's only other black congressman, Democratic Rep. Jim Clyburn.
The announcement came after state lawmakers met urgently with each other and the governor. The head of the Republican National Committee has also called for its removal.
State House Speaker Jay Lucas said in a statement that last week's "terrorizing act of violence shook the very core of every South Carolinian."
"Moving South Carolina forward from this terrible tragedy requires a swift resolution of this issue," he said earlier in the day.
A growing number of religious and political leaders said they would push for the flag's removal Tuesday during a rally in the capitol. The White House said President Barack Obama respects the state of South Carolina's authority to decide the issue, but believes the flag belongs in a museum.
"The flag got appropriated by hate groups. We can't put it in a public place where it can give any oxygen to hate-filled people," said Charleston Mayor Joseph P. Riley Jr., a Democrat.
But the conservative politicians who have led South Carolina for a quarter-century have rebuffed many previous calls to remove the flag.
The last governor to take this political risk, Republican David Beasley, was hounded out of office in 1998 by the Sons of Confederate Veterans, and they made sure his political career was over thereafter. Their influence doomed Beasley's front-runner campaign for U.S. Senate, a seat later won by Republican Jim DeMint.
The group announced Monday that it will vigorously fight any effort to remove the flag now.
Sen. Darrell Jackson, D-Columbia, helped broker a compromise in 2000 that moved the Confederate flag from the Statehouse dome to a Confederate monument outside. That deal made it very difficult to make any other changes: A super-majority of two-thirds of both houses is required.
"It is my opinion — as someone who's fallen on that sword before and shed blood and had the scars to prove it and the threats to prove it — I think the House should do it first. That's the largest body. If the House is serious, send us a two-thirds 'sine die' amendment, and I think the Senate will do the right thing," Jackson said.
The compromise has kept the Confederate flag flying high outside the Statehouse since the shooting, even after state and U.S. flags were lowered to half-staff to honor the victims.
It also means that when the state aims Wednesday to honor Emanuel's slain senior pastor, state Sen. Clementa Pinckney, by welcoming thousands of people to file past his coffin inside the Statehouse, the mourners will likely see the Confederate flag on their way in or out.
This symbolism has angered many, particularly now that photos surfaced showing Dylann Storm Roof burning one American flag and stepping on another, while waving and posing provocatively with Confederate banners.
"Do not associate the cowardly actions of a racist to our Confederate Banner," South Carolina Commander of the Sons of Confederate Veterans Leland Summers said in a statement. "There is absolutely no link between The Charleston Massacre and The Confederate Memorial Banner. Don't try to create one."
At least one lawmaker acknowledged that he wasn't courageous enough to take a stand before, but the death of his friend Pinckney, who had served 19 years in the state House and Senate, changed that.
"I just didn't have the balls for five years to do it," said state Rep. Doug Brannon, who was elected in 2010.
"When my friend was assassinated for being nothing more than a black man, I decided it was time for that thing to be off the Statehouse grounds," Brannon said. "It's not just a symbol of hate, it's actually a symbol of pride in one's hatred."
The debate over the flag has been revived as thousands of people converged on Charleston to show their solidarity with the victims and join a mix of rallies, marches and funerals. Bells tolled across Charleston Sunday as the Emanuel African Methodist Episcopal Church reopened.
"It sends a message to every demon in hell and on earth," said the Rev. Norvel Goff, who led the first Sunday service since the killings at the historic church known as "Mother Emanuel."
Thousands marched later Sunday on the city's iconic Arthur Ravenel Jr. Bridge, a 2-mile span with towering cable supports that is named after a former state lawmaker and vocal Confederate flag supporter. It is one of many symbols of white power that remain in South Carolina and can't easily be changed because of the flag deal.
Less than 2 miles from Emanuel, someone vandalized a Confederate monument, spray-painting "Black Lives Matter" and "This is the problem. # RACIST" in bright red paint. City workers covered the statue with a tarp.