If there is a bench warrant out for his arrest on criminal charges, NY will honor it if he is stopped by a traffic cop, for example, and the warrant turns up. However, NY will not go to the expense of returning him to GA. GA (or the relevent county) would have to retrieve him at its expense to face charges in GA.
Lots of times the state or county that issued the warrant does not really want to "pony up" when the time comes. In that case, the state that arrested him would have to let him go.
No way from here to tell what will happen if he should happen to be arrested in NY. If the pending charges are misdemeanors, there is a very good chance that GA will not claim him. The stakes rise if there are multiple charges and/or the charges are serious felonies.