Question: When is it appropriate for a man at a dinner party to remove his jacket?
Answer: When he qoes to bed.
It's slightly facetious... except it's not really because the answer is broadly correct. A man's jacket is part of his main outfit not his outerwear and, as such, he ought not to remove it outside the privacy of his own bedroom any more readily than he would remove his trousers.
The hat, overcoat and gloves are outerwear. The jacket is not.
The reality is, of course, that times have changed. Removing your jacket and revealing your shirt and braces is no longer socially unacceptable and, if the evening involves dancing in to the small hours, few would blame you for doing so. The key is to remember that a man in trousers and a shirt alone rarely looks well-dreased. Abandoning your jacket means also giving up on your carefuly assembled outfit, so it needs to be timed right. If your first action upon sitting down at dinner is to remove your jacket then you are not only being rude, you are also shortchanging yourself.
As I mentioned before though, this is an excellent argument for wearing a waistcoat so that, when you do remove your jacket, you'll still look smart.