This paper presents an algorithm for discovering surprising exception rules from data sets. An exception rule, which is defined as a deviational pattern to a common sense, exhibits unexpectedness and is sometimes extremely useful. A domain-independent approach, PEDRE, exists for the simultaneous discovery of exception rules and their common sense rules. However, PEDRE, being too conservative, have difficulty in discovering surprising rules. Historic exception discoveries show that surprise is often linked with interestingness. In order to formalize this notion we propose a novel approach by improving PEDRE. First, we reformalize the problem and settle a looser constraints on the reliability of an exception rule. Then, in order to screen out uninteresting rules, we introduce, for an exception rule, an evaluation criterion of surprise by modifying intensity of implication, which is based on significance. Our approach has been validated using data sets from the UCI repository.