This paper proposes a system for managing users and books in a room using various sensors. The proposed system consists of an RFID (Radio Frequency Identification) sensor, a non-contact IC card sensor and camera sensors. When a user enters a room, he/she is identified using IC card, and a tracking process by camera sensors starts at the same time of the identification. Users in the room are tracked by multiple camera sensors. Each book has an RFID tag, and an RFID sensor is attached to a bookshelf. An existence of the books is observed by the RFID sensor. In addition, a user who takes out or returns the book can be recognized by the camera sensors. A prototype was implemented, and the system has demonstrated that the users and books were observed without cumbersome procedures.