Liquid crystalline cholesteric blue phases (BPs) are of high interest for tunable electro-optic applications owing to their fast response times and quasi-polarization-independent phase modulation capabilities. Various approaches have recently been proposed to control the crystal orientation of BPs on substrates, but their basic orientation properties on standard, unidirectionally orienting substrates had not been investigated in detail. Here, detailed studies have been made on the Kossel diagrams of BPs on unidirectionally orienting substrates to understand the three-dimensional crystal orientation of BPs. We find that BPs show strong thermal hysteresis and that the structure of the preceding phase determines the orientation of BPs. Specifically, the BP II-I transition is accompanied by a rotation of the crystal such that the crystal direction defined by certain low-value Miller indices transform into different directions, and within the allowed rotations, different azimuthal configurations are obtained in the same cell depending on the thermal process. Our findings demonstrate that, for the alignment control of BPs, the thermal process is as important as the properties of the alignment layer.