“Although there is no difficulty in theory in estimating fitnesses, in practice, the difficulties are virtually insuperable” (Lewontin, 1974).Fitness is a key concept in evolutionary biology. In many models, this is a quantity that can be defined without (too much) ambiguity. When it comes to test these models of evolution and measure fitness empirically, however, many difficulties arise, in addition to these definitional problems. In this presentation, I will try to summarize and provide an overview of these issues. I will present different approaches to measure fitness in the lab or in the field, distinguishing 'forward' and 'backward' methods. To illustrate each case, I will use examples from my work that include the most precise estimates that have been obtained to date. I will finally relate the problems of fitness measures to the neutralist-selectionist debate.