Scandal - Amanda Quick

Ok.  Simon, Earl Hotpants is, well, clearly, an Earl.  He's been plotting revenge for almost as long as the heroine has been alive.  And now it's been made very clear that socially he outranks her family by a laughable amount.

 

So why is his revenge something that takes 20+ years to plan?  I mean, it sounds like he has the political weight just of his position to do horrible things to the family.