hedwards: Considering that they have yet to get Singleplayer monster AI right, I have a very hard time believing that it was less challenging. Especially since amateurs were making some pretty damn fine bots as early as the late '90s.
No, they are making singleplayer monster AI just the way it's supposed to be. Targets with simple comprehensible patterns and limitations for the player to exploit - and when singleplayer AI sucks it's usually not because of bad tech, it's because of bad design choices. Single player games are about one unstoppable hero who is able to defeat hundreds or thousands of enemies *without dying*. And death in a single player game is something frustrating, you don't want the player to need more than a few attempts before he moves on to the next challenge and gets to see the next part of the story or world. In multiplayer a player is already successful if the amount of kills is *barely* above the number deaths, that's why multiplayer games *need* better AI that is on-par with the player. Plus the goal of a single player AI is to do maximized harm to a player - with the occasional gimmick thrown in just to make the game more immersive - a multiplayer bot's goal is to win the match. The latter is a far more complex thing to achieve.
And as for the bots: yes, modders did create some amazing bots back in the day but that doesn't mean that those weren't pretty darn complex. You always had to feed them with information about the environment via manually placed nodes per map or had to use clever algorithms that would analyze a map and create all the AI paths itself which is quite a bit more complex. And on top of that a genuinely *good* bot would have to evaluate that data during the match to make good tactical choices. Additionally bots had to be able to maneuver in combat to be a hard target without dying from the environment - you can cheat around these things in singleplayer by removing the monster's ability to fall down ledges and stuff, you (normally) don't have these options with multiplayer bots, though, which have exactly the same amount of actions and features at their disposal as the player, else they will simply be perceived as cheaters - not your goal when making a bot.
Additionally singleplayer enemies generally have to navigate maps that are designed to be believable, so they are generally easier to navigate. Multiplayer maps are mainly designed to provide interesting options for the players and thus navigation has to be more complex there - and that means that the requirements are higher to the AI on multiplayer maps as well.
Anyway, if anything you shouldn't be complaining about developers focusing on multiplayer or making bad AI for singleplayer games, the problem is that there are few game concepts where the basic gameplay would actually benefit from "good" AI. The quality of AI in games isn't really measured by its complexity and capabilities, it's measured by how much it contributes to the desired game experience - and rarely will a totally complex AI with complicated patterns do so. What you want isn't more effort put into AI, you want game designers to come up with game concepts which will benefit from a believable and varied AI.