I'm not finding the herbalist, nor am I able to knock on the hunter's door. E doesn't work for knocking, but has worked with everything else. Is this part of the game or a bug?

The herbalist is west of town. As for the hunter, I don't remember if you knock or what; he is out hunting though. Try using Witcher sense and see if there are any tracks nearby that you can examine.