Removing backrest. Need some help

Been following the solo seat thread and I have to admit the bikes look great. So today I removed the back seat, ugh, dirty, dirty fender. Now I get ready to remove the backrest with my allen wrench and realize it just wont fit in the groove between the chrome plate and the hard bags. So, I know I am going to have to loosen the hard bags, so my question is, do I just untighten the bolts on the inside or what.

I knew not to proceed any further because I will screw up a unscrewed screw. Have to admit it is going to look nice and should not take much time to put back, once I have the secret answer from someone here on the hardbags. Thanks