This was nothing like your interaction though. Asking why doesn't mean they don't acknowledge the other person doing it for fun, it's just that they are curious if there's a reason.
-Haha, why?
+Found it cool and felt like it.
-That's cool, I used to play X when I was younger but....
It can also just be a conversation starter, nothing wrong with asking why.