Lyrium should have been an issue if you sided with the Templars and had to help supply them or wean them off or whatever, yeah. Could have Ser Barris be the one who wrestles with it on a personal level.