I think people convince themselves. The models just predict what token will give them a reward. the more positive feedback from users the more it goes down that algorithm
AI can mimic emotional responses. But it will never have feelings. AI is just a thing, not human being.