The original Portraits of the Five Patriarchs of the Shingon Buddhism that Kukai brought back from Tang have been handed down in Toji Temple. With the portraits of Ryumo and Ryuchi newly added in Japan in the 12th year of Konin (821), the portraits of the seven patriarchs were completed. It is generally believed that the inscriptions on the seven portraits were handwritten by Kukai, but there were some doubts about it in the old days. In this paper, based on the study of the historical research works, we support the opinion that the biographies on the four patriarchs (Ryumo, Ryuchi, Zenmui, and Ichigyo) out of the seven were not written by Kukai. On the other hand, in the collection of Kukai's letters, 'the Koya Collection of Letters', is the letter concerning the restoration of the Portraits of the Five Patriarchs. The letter addressed to the Left Chief of the Guards (Fujiwara no Fuyutsugu) is the one. It has been believed that the letter was written around the 6th year of Konin (815), but based on our study, it has been made clear that it was written around the 12th year of Konin. In this letter, Kukai requested Emperor Saga to restore the Portraits of the Five Patriarchs, and at the same time, he requested the emperor to write the biographies on the Portraits of the Five Patriarchs following the historical fact about the Emperor of Tang Dynasty. It is highly possible that the Emperor wrote them to reply to Kukai's request. So it can be considered that the inscriptions on the fourpatriarchs, in a different handwriting from Kukai's, should have been written by Emperor Saga.