Memory or paper?

Here is a question to ponder: is it better to write notes down on paper so you can refer back to them, or is it better to rely on your memory? Is memory a muscle that improves with practice (i.e. no notes) or something else?