Range-based scanners built upon multiple cameras and projectors offer affordable, entire-shape and high-speed setups for 3D scanning. The point cloud streams produced by these devices require large amounts of storage space. Compressing these datasets is challenging since the capturing process may result in noise and surface irregularities, and consecutive frames can differ substantially in the overall point distribution. Exploiting spatial and temporal coherency is difficult on such conditions, but nonetheless crucial for achieving decent compression rates. This paper introduces a novel data structure, the temporal sparse voxel octree, capable of grouping spatio-temporal coherency of multiple point cloud streams into a single voxel hierarchy. In the data structure, a bit mask is attached to each node, existing nodes can then be reused at different frames by manipulating their bit masks, providing substantial memory savings. Although the technique yields some losses, the amount of loss can be controlled.