It does seem counterintuitive but it works. I wear running tights (light ones) under my riding gear and it helps prevent rash and scratches from the gear moving repetitively and chaffing and allows for sweat to wick away and cool the body.
Not sure what sort of pants you wear but the knee armour in my suit protects my knees pretty well although I understand where you are coming from. I can feel the sun burning the tops of my legs on days such as we have experienced recently.
It may seem counterintuitive but I wear silk leggings and top under my suit. I read an article a while back about a racing team out in Spain wearing the same gear under their leathers when the track side temperatures were in the 40's. The company I bought mine from are in the UK but I guess it can be found online.
They certainly make getting into and out of both my Rallye Pro suit and my leathers easy when the body gets sweaty and sticky.