Federal agents have decided to rework their case to deport Rene Lima-Marin back to Cuba, three weeks after a judge said he would be released for good.

"I think he's optimistic, also reserved," Aaron Elinoff, Lima-Marin's attorney said. "I mean again, the rug has been pulled out from him on more than one occasion in the past, but I think he has got some confidence in what we're doing and where we're going."

Lima-Marin is a convicted felon who was mistakenly released in 2008. Years later, after Lima-Marin started a family and a new life, the courts realized their mistake and remanded him back to prison. He was then released again, only to be detained by ICE (Immigration and Customs Enforcement) agents.

"He doesn't pose a danger to the community just based on the six years he was out and committed no new offense," Elinoff said.

A pardon from Governor Hickenlooper this year cleared Lima-Marin's aggravated felony he committed as a teenager, but the feds are reworking their case for deportation into a firearms violation related to the original convictions. Lima-Marin was found guilty of robbing video stores in Denver in 1998 with another man. Immigration officials are using the firearms element of that now-pardoned conviction as the grounds for deportation.

RELATED: How did no one know a felon who was ordered to go free was flagged for deportation?

Elinoff said Lima-Marin's story, and the attention it has received, does not sway immigration judges.

"The discretionary part for the judge's authority are pretty limited in scope, really a judge is limited based on the immigration and nationality act," Elinoff said. "So it all comes down mainly to legal argument."

ICE confirmed its plans to appeal the order to release Lima-Marin.

"The equities of the case don't really matter most of the time, except coming up- if we get to have a bond hearing, then his past history is going to play a significant factor," Elinoff said.

If either side appeals the next judgment, the case will go to the 10th Circuit Court of Appeals.

Kyle Clark's full interview with Lima-Marin's attorney: