The spread of the plague prompted the bishop of Auxerre, lord of the place, to endow the castellations of Varzy with a leprosarium in the 13th century. Annexe of the Hotel-Dieu, it was located 3 km from the town, in Vaumorin. At the end of the twelfth century, next to the malady was built a chapel dedicated to Saint Lazarus, patron of lepers.
Leprosy disappeared at the end of the fifteenth century, and the maintenance of the chapel is assured as best as possible by the Hôtel-Dieu of Varzy. From 1940, the process of disintegration amplified. The roof is partly open, the walls are gutted and the portal crumbling. Choir and nave are devastated by the waste collectors. Fortunately, the Camosine undertook its restoration from 1977 to 1982.