I thisnk it would be good to split the warning "-Wsign-conversion" in two. One for asignment of signed to unsigned and another one for the opposite sense. The reason is that the former is, I think, more common, and has a perfectly defined behavour, viz. to reduce the number modulo the corresponsing power of two, while the latter is not well defined if the unsigned variable has a value greater than the maximum value the signed variable being assigned to can represent.
I had to turn off that warning, even I think it is really usefull for the unsigned-to-signed sense, because of many annoying warnings. You can see this entry among gcc Non-bugs page, "Certain Changes We Don't Want to Make":
Warning about assigning a signed value to an unsigned variable.
Such assignments must be very common; warning about them would cause more annoyance than good.
Regards