Most times you hear “That’s immoral!”, the goal of the speaker is to evoke negative emotions in their audience.
The goal is not to encourage the audience to recognize the act in question as falling into any actual perceptible category of immorality within some universally accepted moral scheme.
In this way, calling something “immoral” is parallel to calling something “magical”.
To say “That’s magical!” is normally to attempt to evoke positive emotions in the audience, the same emotions the speaker feels.
There is no actual realm of magic necessary to accomplish the goal of evoking these positive emotions.
Proclaiming some action “immoral” successfully accomplishes the goal of evoking the negative emotions that might serve as social pressure to stop the behavior, all without needing to justify the actual existence of a moral realm that would legitimate such utterances as actually referring to an immorality as real as a real magical moment in which laws of nature are violated.
This is a common equivocation employed by those hoping to impose their emotionally-derived values on others.
Perhaps it is merely a slight dishonesty, often committed subconsciously.
But I find it most often indicative of a lack of actual intellectual integrity, and an impediment to any productive discussion.