An idea from djc in discord:
In fact it would be nice if we can relax it even more, like allow the bot to run if we previously merged a PR from the same author
This will probably be a little bit trickier than #32 - I'm not sure if GitHub exposes this state through the API. Some research will be required.