If you are not into backpacks and feel it would look inappropriate with your dress or for the occasion go in for smaller compact purses. Firstly, since they are smaller in size you will have to cut down on the things you put in it that would automatically make it light weight.
"Clutches" are a perfect example for compact small ergonomic purses. They are stylish and chic and would fit in your palm and cause no strain what-so-ever to your back. Keep a number of them to befit the occasion and keep only the essentials you need for that time in them.
The ergonomic clutch in the picture is from: Teemzone Genuine Top