This is a common quip used to counter people trying to make a point of a seemingly insignificant moment or interaction. It’s a valid argument in some cases and trying to make gravity out of levity can definitely be off-putting or ruin “the vibes.” That said I think it’s also important to pay attention to when and how it’s being used. It ain’t that deep. It’s just a joke. Why so serious? All these phrases are defensive and are often used (often not intentionally) to invalidate the feelings of individuals with valid concerns and to deny the impact that their (or someone else’s) words had another person, regardless of the intent. In many cases phrases like these can be used to dismiss microaggressions which perpetuate cultures and attitudes that further suppress already marginalized populations.
At the same time I think its important to recognize the reverse perspective. People need to give each other the benefit of the doubt in conversation. If I hear something that I think is inappropriate or that is “triggering” especially from someone I know well, I would like to think that their intentions were not bad, and rather than lecture or attack them, I can ask for clarification or expresses my concerns in a non-judgmental way.
Though from there it is also on the “offender” to not feel offended themselves by a perceived attack on their character. And this can go back and forth forever. Human’s are fragile creatures, but I think the more we seek to understand each other as opposed to seeking to impose our worldview on each other, the closer we can get to making genuine progress on issues that really matter.