Except for the USA & Russian "mandatory" factories, building factories seems to me to ONLY make sense if started pretty much turn one or turn two (I'll make some allowance for a German or Brit turn 3 start, but have serious doubts) - the cost is just too horrendous and payback period too long otherwise, IMHO.
However, I do think there is a strong argument for building a second German factory (started the second or possibly third turn). The seeming rationale for not doing so is that the PP are much more critically needed for preparing for Barbarossa. However, Barbarossa does not effectively get underway until late 1941 or early 1942 - in most games, the Russian player is busily beating a hasty retreat into the interior of Mother Russia the first two or three turns of Barbarossa, largely refusing significant combat (or at least that seems to be the common occurrence - standing and fighting early usually seems a losing gambit). Of course, fewer German units may encourage a tougher initial Russian stance, but then again, maybe not.
As such, assuming a second factory can be started even as late as the third turn of the game, it breaks even in Winter 41-42 and which is when combat seems to start in earnest on the Eastern Front and when the ground units are actually needed (and not really before). In fact, given all the givens, pushing the break-even to Spring '42 seems tolerable (i.e., starting the second factory turn 4), given the big win that five extra production points provides forever and ever.
The trade-off is whether or not the Germans can do without the extra ground units during 1940 and early 1941. From what I've seen, I think that the Germans usually have a breathing spell and can in fact usually do without the extra six-to-eight ground units a factory costs during Fall-of-France to Barbarossa. By the time they get Barbarossa really going (Spring '42), they will have built all the "lost" units and in fact then start to get ahead of the game.
In fact, it has been my experience that with two German factories, the extra ten PP goes largely to being able to build subs or other things to work on keeping the Brits in check and which I think is utterly critical to the Germans – something has to be done to keep the Brits in check else they always seem to torpedo the Germans by sometime in 1943 if not earlier.
Of course, building two early factories does indeed call for incredible fortitude - dare I say guts - on the part of the German. But, I think the German HAS to play for the long haul, looking towards 1942 and 1943 almost right from the first turn.
All I can say is that I have never ever regretted a single German Factory that I've built. I even used to build four German factories when they cost only 5 PP per turn - but the increase to six PP per turn has discouraged even me from buidling more than just two.
dd