This paper proposes a framework for human interface device for Video Games. Generally, a certain input device is designed to transmit its input data to its designated output devices. However, in some cases, it would be necessary to transmit the input data to other types of devices besides such designated output devices. For such situations, the proposed framework provides common protocols to exchange data among input and output devices based on Server-Client mechanism. Therefore, even when different developers individually make programs for each input and output devices, with the use of the framework, data communication among them will be possible without any modifications of the programs. To clarify the usefulness of the framework, this paper shows a serious game example, i.e., a rehabilitation game, developed using the framework.