Tough call but have to go with act 3 since it seems you have to walk for ages to find your way through. It only gets frustrating if you're looking for a waypoint before you can save and exit. At least the desert maps are square .
Narrow passages. Very hard to traverse, especially for summoners. That would be act 2, mostly (maggot lair, sanctuary). Maybe now that i am soon to max Corpse Explosion, i won't "hate" such places so much...
As a point of interest, any char that you think may have difficulty in the Maggot Lair, just throw them 3-pieces of Trang Oul's set. The +Firewall will pretty much fill a corridor if angled correctly, and nothing in Maggot Lair is naturally Fire Immune (well, Flying Scimitars, but do those really count?).
The first half of Act 3 and the sewers in Kurast are the only areas of the entire game that I tend to find unpleasant to play through, but that is because it often requires more backtracking than any other section of the game. At least Arcane Santuary has a central waypoint you can use to start a new branch if you chose poorly.
Of course, there are ways to deal with any problem in any area of the game so no matter what we each choose it will largely depend on our play style, and there is no truly definitive answer.