DIN 1.6582 (34CrNiMo6) is a slightly more highly alloyed version of the U.S. steel SAE 4340. It is normally forged, normalized, annealed, rough machined, hardened (austenitized and quenched) and tempered. This is followed by finish machining and (optionally) a case-hardening process. There is also a high Si (1.5%) version of this steel. A grain size of 6-8 is typical.
The procedure you describe is indeed referred to as soft annealing and is designed to achieve a maximum hardness of 248 HB. An example of the type of microstructure you should achieve is shown in Figure 1 (hardness approximately 244 HB). You report a hardness of 320 HB, indicating (to me) that either the part wasn't completely uniform in temperature before the onset of cooling or that the cooling rate was too severe.
As you know, annealing causes softening but also coarsening of the resultant ferrite-pearlite microstructure. You can either achieve a predominately pearlitic structure (normally not desirable) or a predominately spheroidized structure (preferred for both machining and heat treating). This can be obtained by either: heating to 750°C (1380°F), rapid cooling to 650°C (1200°F) followed by a 12-hour soak at 650°C (1200°F) followed by an air cool OR heat to 750°C (1380°F), cool rapidly to 705°C (1390°F), then furnace cool slowly (3°C/h maximum) to 565°C (1050°F) followed by an air cool.
Alternately, a full anneal can be achieved by heating to 810°C (1490°F), holding for 1 hour after reaching uniform temperature throughout the section, then furnace cooling from 810°C (1490°F) to 355°C (670°F) at 11°C/h (approximate total time 41 hours) followed by air cooling to ambient temperature.