I read this after watching The Late Late Night Show with Craig Ferguson and couldn't stop hearing Craig's voice speaking the words with Geoff Peterson throwing comments in between
So anyway, I noticed this a long time ago, but never bothered to look into it further,
until now 
. The button works half way. If the new general introduced to the army stack is higher ranking than the current army commander, the button will switch the Army HQ unit to the higher ranking general. If it's the other way around, even if you have just switched generals and want to switch them back, it won't work.
Work Around:
- Put the new, lower ranking general into the army stack
- Click on the army commander
- Click on the Dismiss Army Commander button to split the Army HQ unit from the current army commander
- Now remove the previous army commander from the army stack
- Click the Create Army button
et voila you have put your new army commander in charge of your army

.
On a side note all these steps are transient, that is to say, anything you do in between the turn's starting setup and the orders saved before executing the turn (when you click the bugle directly, your orders will be saved automatically) have no meaning.
This allows you to disband an army and re-create it in the same planning phase so that it's name will be changed to that of the historical army of the grand region. So if you've created the Army of the James and sent it down to New Orleans, if you disband and then recreate the army, it will now be named the Army of the South.
One Caveat, when you do this, all corps attached to the army will be detached and revert to non-corp status regardless of how far way they are. So if they are out of range of the army they lose their corp status and cannot regain it until they are within range of the army. If they are however within range of the army, you just re-attach them to your newly named army and the boni they got from the old army remains unchanged.