blog changes

I want to make some design changes to my blog to make it look more modern looking and maybe help it to load a little faster with a better theme. Will changing the layout of my blog hurt it's search engine rankings?