This paper examines the effect of horizontal coordination mechanisms on coordination activity in organizations and the moderating effect of organizational differentiation. In classical research, such as the contingency theory of organizations and information processing view, horizontal coordination mechanisms are effective coordination devices for organizations. In particular, to integrate differentiated functions, horizontal coordination mechanisms are treated as crucially important. However, recent studies indicate the cost of these coordination mechanisms and emphasize the importance of organizational consensus. Based on these theoretical gaps, this paper investigates the relationships among horizontal coordination mechanisms, organizational differentiation, and organizational performance in the SBUs of Japanese companies. The result shows that (1) horizontal coordination mechanisms have a negative impact on organizational coordination activity, (2) increasing functional differentiation strengthens this relationship, and (3) increasing hierarchical differentiation weakens this relationship. These findings indicate that the functions of horizontal coordination mechanisms are different in SBUs and functional organizations.