The way I did nobility in my story has to do with the cultural views of kingship in my demon society. They see their kings (used as a gender neutral term) as the person able to protect an entire kingdom (mega cities for them) by being able to solve the big problems no one else can. So it often falls to the most physically fit, because their environment is actively hostile, but occasionally it does fall to someone more on the charismatic/clever side if they can recruit physically capable people who believe in their leadership ability.
For the Ascended Demon King, it's the person able to pass through a battle royale known as the Royal Trials that are open to all, and then a test of will/fortitude known as the Thousand Caves of Torment. The historical basis was the first Ascended King asking the gods for the power to unite the entire Demon Realm, but they didn't want to just hand it to him for lore reasons, so set up a system to earn it. Anyone is able to compete in the Trials, but only the singular winner can enter the Caves. The Trials were actually a second request by the first King, because he thought the loss of life of people fighting to enter the caves was a waste when the skills of those lost still could have benefitted the realm.
So my advice would be to consider the way kingship/rulership is viewed by your particular society and then develop the methods of inheritance based in that view and the associated cultural values. I consider my work a "translation" a la Tolkien, so terms like "king" and "lord" are used, but not the exact terminology of the actual society. So you can still use English nobility terms, but you need to redefine what they mean and that meaning still needs to be something in the ballpark of the original to make sense as a "translation."
Present day demon nobles (some 50k years post the first Ascended King uniting the realm) are descendants of the advisors and inner circles of past kings that won their seats in the warring era, and the Petty Kings of the united city-states are all primarily inherited titles - but Petty Kings tend to select their successors by capability (which is subject to bias) amongst their children, not on the basis of age or gender. The Ascended King, however, has only had 4 in the same generation at the longest due to the open nature of the Trials. The current King nominates a successor, but it's only a nomination. The rest of the realm decides how strong the competition they want to send is based on how much they approve or disapprove of the nomination. (It's usually fairly polarizing, actually.)
Demons do also find things like poison or (medieval) firearms completely fair game, because they have a survivalist culture, so there is some balance for those less physically powerful in the Trials. It would also be considered fair for a stronger competitor to enter with a weaker one and fight on their behalf, then concede the seat itself. The only thing they strong disapprove of is those intending to compete taking each other out before the Trials begin, since it doesn't show how well a competitor can handle themselves on the fly - since emergencies are the main thing kings are expected to handle.
So yeah - define cultural values, then consider what the position of ruler is seen as, and expand from there on how it is attained. That's how you'll get rules that make sense, because they really just need to make sense for the culture they apply to. Demons in my story see kingship as a civil office - so those who want it are expected to prove their capabilities, and typically don't serve for life. The proving ground involving physical combat has to do with the historical kinds of challenges the King is expected to face in their society. And then there's the addition of historical context.
Hope that helps!
One additional note - Petty Kings can still be challenged for their position through combat, though it rarely happens, for a few reasons (not necessarily in this order). One, people not wanting the job. Two, people not feeling like they could do a better job than the current ruler, even if they have complaints. Three, people in positions of power have more time to hone their combat skills, are able to elect champions that only hone those skills, and have access to better equipment, so... yeah. Just because they can be challenged, doesn't mean they can be defeated, and all that.
(Sorry for the longer post, but didn't want to make it seem like I was trying to force people to read my work to give some advice based on my own craft methods lol. And yeah, as someone else mentioned, researching irl non-patriarchal societies can help a lot as well.)
