ATLANTA -- Experts are voicing concerns about how and why former Gov. Zell Miller departed the Governor's Mansion with over $60,000 in taxpayer dollars.

Every living Georgia governor, including Jimmy Carter, says they would not have kept the money or considered that it belongs to them.

Miller is busy promoting his new book, "A Deficit in Decency."

But his critics say the former U.S. Senator needs to write another chapter about the money some say did not belong to him. To millions of Americans, Miller is a national treasure, the senator who stands for what's right.

"He's somebody who's done the right thing at the right time for all of us," said one Miller fan.

Records uncovered by Channel 2 Action News are raising some experts' concern.

"If there's even an inkling of a conflict or an inkling of an integrity issue then that should have been addressed as he left office," said Dr. Hasan Crockett, a political science professor who teaches at the Brisbane Institute of Morehouse College.

The issue goes back to 1999 when Miller retired.

He packed his track and in his words was, "Headed for the mountains."

He also left with the mansion allowance, an account provided by Georgia taxpayers, which had ballooned to an estimated $61,000.

Said Crockett: "To drain the account and leave the next governor without that account and take that account for whatever reason ... that's very unethical, and I think an issue that should be looked at further."

According to Georgia law, the mansion allowance is an amount specified in the legislature's appropriations act provided for the operation of the Governor's Mansion.

We asked the man who wrote those checks to tell us what that means to him.

"It was not a salary supplement," said Winford Poitevant, the retired division director for the state's Office of Planning and Budget. "It was a mansion allowance, and that's what (the law) says."

When asked what Poitevant would say if told that Miller apparently left office with tens of thousands of dollars from the mansion allowance, he replied, "To be on the safe side, I wouldn't have done that."

Poitevant said Miller never called to ask about the funds.

The state did not list the money as income on his W-2 tax form.

Miller declined requests for an interview, but acknowledges that he kept the money.

"When I retired from state government, I received only what I was advised was legal, ethical and traditional," he said in a statement sent to Channel 2 Action News. "I would never, ever knowingly have done anything else.

Upon taking office, we were told that the mansion fund was our's to use as we chose: Use it up or live cheaply and pay personal taxes on what was left. We chose the latter."

In addition, Miller's former chief of staff and legal advisor said, "There is nothing in the law that prevents the governor from treating the allowance as additional compensation."

He cites the attorney general's opion from 1969 which reads, "Amounts paid to state officials for expenses are part of such officials' gross income and, in my opinion, are taxable to the extent that they are not used for such official business."

That doesn't wash with the group Common Cause.

"I don't think where the legislature appropriates money for an explicit purpose, for the operation of the mansion, that's what it says in the statute," said Emmit Bondurant, an attorney for the group. "That the governor or anyone else is allowed to use it for any other purpose whether or not they pay taxes on it."

What do other Georgia governors have to say?

Each one, Carter, Frank Harris, Roy Barnes and Sonny Perdue, told us they did not consider it their personal income and left any balance to their successor or the state.

A spokesman for former Gov. George Busbee, who is deceased, said Busbee did not consider it to be personal income and left any balance to his successor or the state.

The spokesman said Busbee did not keep the balance at the end of every year, saying it was minimal, just a few thousand dollars.

Miller's total, over $60,000 taxpayer dollars, is a worth a year and a half in mansion appropriations.

"In my view that is a clear statutory violation," Bondurant said. "That should not have occurred, and whoever did it should repay the state."

Crockett agrees.

"Oh of course," he said. "Of course."

When asked, both Carter, a Democrat, and Perdue, a Republican, said they did not and do not view the mansion money as personal compensation.

"No one has notified us to this day that what we did with the mansion fund was improper," Miller said.

When asked if he planned to repay the money, his attorney said, "If the state tells us there was a mistake, Sen. Miller is going to do whatever it takes to correct the mistake."