A LAG doesn't only require configuration on the virtual side, but also on the physical switch(es).
If you want to reconfigure the LAG, I'd suggest you either unplug the appropriate network cable first (the port that you want to remove), or shutdown the physical switch port. Don't forget to remove the physical port from the channel group on the switch before you take it online again.
André
