Keeping a lab notebook

Reproducible software pipelines

• Clearer experiments
• Gives you a sharper aim
• You can reconstruct your past thoughts

George V. Neville-Neil. 2023. Dear Diary: On keeping a laboratory notebook. Queue 21, 5, Pages 50 (September/October 2023)

A possible workflow

• Create a file title with the current date and title of the experiment
• State your hypothesis
• Record the configuration of the experiment
  • Environment (software, dependencies, machine, etc)
  • Parameters
• Record the outcomes
• Analyze the results

Tips

• You can git commit hashes to refer to the current state of the code

• If you encode a full experiment in a version-controlled script, you can just refer to its name

• You can use markdown files, obsidian/logseq, Emacs org mode, quarto notebooks…

• If you use quarto/jupyter notebooks, make sure to cache the computations and freeze old entries

  • Otherwise it is too slow
  • You risk disrupting old entries with new data

• Never delete things: strike them out

Leverage git

• You can use git identifiers to refer to the version of the code of the experiment you are running
• You can use git diff to record any change from the last commited version

An example