To realize context-aware services in a ubiquitous computing environment, it is essential to extract the spatial structure of non-linguistic context information that involves a person at a certain location in the real world. With the techniques of multimodal behavior observation and knowledge science, we propose a method of location-based activity analysis to extract the explicit or tacit roles of location in human activities and to express the spatial contexts of the world. We confirmed that location-based estimation of real-world spatial contexts enables a close approximation of the contexts of real-world people. We also showed that spatial contexts can be multidirectionally understood by accumulating, comparing, and integrating the results of spatial interpretations made by people with different attributes and different viewpoints. The estimated spatial contexts can be used for onsite and offsite support services that consider the inherent characteristics of each location and potential human needs arising at each location.