Shit, did you read my mind? I've already started a backlog on a similar story, but the dude just wants to live a normal life after crawling his way out of rock bottom.
And he's sent to one of those worlds but with a lot more ntr, which is the main reason he rejects everyone's advances. Of course, he still finds himself someone, but it's on his terms and not the aphrodisiac-fueled world's.
Eventually the world and forces that brought him there start pushing sex onto him in more and more ridiculous ways, like having the bank he tries to open an account in have the employees be victim of sexual blackmail that he can solved by doing it with their hot manager. But instead, and out of spite for the system forcing it (and since he wants nothing to do with a blackmailer), he ends up trying to unionize them so they can stand up to it themselves. Of course, they misinterpret him and think he's trying to organize an orgy and show up for that instead. He still gets them to unionize, but leaves as soon as the "team building exercises" start right afterwards out of frustration.
Title: NTR is a shit fetish