I don't know how to carry off this J.Crew skirt

J.Crew skirt (HERE)

J.Crew cashmere jumper (HERE)

Chloé shoes (few years old but similar HERE, HERE, and HERE)

The skirt is so bright and ladylike that I feel it has to be casualised.  That's not a word but you know what I mean.  So I am thinking a big jumper (maybe a camel color instead?) and a thick heel or a sneaker.

I don't know, anyone have any advice?  I was seduced by the colors, but maybe I should have slept on it...