October 2019 update: Restore Privacy's guide to Firefox may be the best yet. Arch Linux's wiki entry on Firefox privacy and Mozilla's own page on privacy settings are also good starting points.
February 2018 update: There is a handy site called Firefox Profilemaker that poses a few questions and, based on your answers, generates a custom privacy-optimized Firefox profile.
January 2018 update: The Privacy Tools page has a section how to change Firefox's about:config settings for security.
May 2016 update: Because other pages on Firefox privacy have more detail and are updated more often, I am deprecating this post.
Out of the box Firefox has a lot of "features" that leak information about your browsing and provide targets for malicious attacks. Let's talk about fixing this in Firefox 41 (and maybe older and newer versions as well).
Some of Firefox's unwanted features can be turned off in the options GUI. Here the desirable settings are pretty clear (e.g., it's obvious that you would want to check the box next to "tell sites not to track me" or whatever.) But if you want to see all Firefox's options, you need to use the about:config screen (which you get to by typing about:config in the address bar). Some of the about:config options mentioned below might already have been toggled to a safe default by your choices in the options GUI. Nonetheless, my advice is to make sure the following settings are such:
beacon.enabled = false
browser.cache.disk.enable = false
browser.cache.diskcachessl = false
browser.cache.offline.enable = false
browser.pocket.enabled = false
dom.event.clipboardevents.enabled = false
dom.storage.enabled = false
geo.enabled = false
geo.wifi.uri = 127.0.0.1
media.peerconnection.enabled = false
network.cookie.lifetimePolicy = 2
network.dns.disablePrefetch = true
network.http.sendRefererHeader = 0 (this will break a few sites)
network.http.sendSecureXSiteReferrer = false
network.http.speculative-parallel-limit = 0
network.prefetch-next = false
pdfjs.disabled = true
webgl.disabled = true
Note: the about:config screen shows in bold any settings that have been changed from the default.
I recommend a few extensions. Some are for privacy and security, others are just to make the browsing experience a bit better.
Other neat things
Here are some other interesting changes you can make.
In about:config you can tell Firefox to open searches in a new tab with browser.search.openintab = true
Apparently the gnome project has an official theme for Firefox. I rather like it.
I drew on a few sites for this post.