1# darktable-curve-tool 2 3 4## About 5 6 7The _darktable-curve-tool_ program will help you approximate more accurately the transfer 8curves used by your in-camera JPEG engine. 9 10This tool does so by analyzing both the RAW data and the resulting 11JPEG data from your camera. 12 13 14## Limitations 15 16 17The computed curves are by no mean a way to have the exact same rendering as 18your in camera JPEG engine. Many more algorithms are used by your camera to 19generate the JPEG. The curves are only one of them. 20 21The tool has some known limitations: 22 23 - it computes RGB basecurve on the sole G channel, though all three channels are 24 analysed 25 - the tool supposes the JPEG files are sRGB ones. It doesn't know or understands 26 about ICC profiles (not even AdobeRGB). 27 - the tool is happily confused by JPEG files that are portrait rotated, and their 28 raw is not. The helper script tries to auto correct that during the conversion step. 29 - the current procedure relies on dcraw which may not have a correct color matrix 30 for your camera. Check if `darktable/src/external/adobe_coeff.c` and 31 `dcraw.c::adobe_coeff::table` do match. If not, copy darktable's one. Beware 32 both tables do not use the same camera names. 33 34## Requirements 35 36 37Utilities mentioned below require following dependencies to be installed: `dcraw`, `libexiv2-dev`, `sqlite3` and `gnuplot`. 38 39## Building 40 41 42You can build the tool using the following commands: 43 44 $ cd "$DARKATBLE_SRC_ROOT/tools/basecurve" 45 $ mkdir build 46 $ cd build/ 47 $ cmake -DCMAKE_INSTALL_PREFIX="@CMAKE_INSTALL_PREFIX@" -DCMAKE_INSTALL_LIBEXECDIR="@CMAKE_INSTALL_LIBEXECDIR@" -DCMAKE_BUILD_TYPE=Release .. 48 $ cmake --build . -- install 49 50You are invited to print the help message to get to know the tool's options: 51 52 $ "@CMAKE_INSTALL_FULL_LIBEXECDIR@/darktable/tools/darktable-curve-tool" -h 53 54It may help you better understand the following paragraphs. 55 56 57## Determining a basecurve/tonecurve using _darktable-curve-tool-helper_ script 58 59 60An additional helper script called _darktable-curve-tool-helper_ is provided. This 61script should automate many steps of the curve determination process. 62 63It is assumed that `$YOUR_INSTALL_PATH` is in your `$PATH`. If not you can run in bash/zsh: 64 65export PATH="$PATH:@CMAKE_INSTALL_FULL_LIBEXECDIR@/darktable/tools/" 66 67 68### Gathering the statistics 69 70 71 $ for raw in my raw file list ; do 72 darktable-curve-tool-helper "$raw" 73 done 74 75_darktable-curve-tool-helper_ will look for corresponding JPEG files by itself; if no 76corresponding JPEG file is found, the embedded JPEG file from the raw is extracted. 77 78 79### Computing the curves 80 81 82 $ darktable-curve-tool -z -e <one of the RAW files> | tee mycameracurves.sh 83 84At this point, you should have some console output explaining how to apply these 85curves, or submit them for final inclusion by the darktable developers 86 87 88### Applying the curves 89 90 91The following command will inject the computed curves in your database. 92It is highly recommended to back it up first! 93 94 $ cp $HOME/.config/darktable/data.db $HOME/.config/darktable/data.db.bcp 95 96Now you can safely run the inject script: 97 98 $ sh ./mycameracurves.sh 99 100 101## Determining a basecurve/tonecurve with _darktable-curve-tool_ alone 102 103 104Using _darktable-curve-tool-helper_ may not be sufficient and you need to either 105have more control or understand what is done behind the hood by the script. 106The following chapters will explain in depth all the steps required for 107determining a curve with _darktable-curve-tool_ alone 108 109 110### Creating the PPM versions of your JPEG and raw files 111 112 113Let's say you have FILE.RAW (eg: .NEF/.CR2) and FILE.JPG 114 115 $ dcraw -6 -W -g 1 1 -w FILE.RAW 116 $ mv FILE.ppm FILE-raw.ppm 117 118This creates a PPM file, named FILE.ppm, that we rename to FILE-raw.ppm. This 119file contains the data from your sensor in a convenient format for darktable-curve-tool 120to read. This data represents the data used as input by your in camera JPEG 121engine. 122 123Let's now convert the JPEG file to the same convenient format: 124 125 $ convert FILE.JPG FILE-jpeg.ppm 126 127This creates another PPM file. But this new PPM file contains the data that your 128in camera JPEG engine has output. This step may also involve a rotation of your 129image so that the PPM from the raw and the JPEG share the same orientation. 130 131 132### Gathering a round of statistics 133 134 135It is now time to let _darktable-curve-tool_ analyse these two files so that it can gather 136some statistical data for a later computation of the curves 137 138It is assumed _darktable-curve-tool_ is in your `$PATH`. 139 140 $ darktable-curve-tool FILE-raw.ppm FILE-jpeg.ppm 141 142This command loads and analyses the corresponding pixels found in both images. It 143writes, to a state file, the correspondence found for each pixel value. 144 145Given the histogram of each photography, you may need to repeat this operation 146multiple times to cover the whole range of values that your camera is able to 147capture. There is no exact number of files to be analysed, this all depends on 148your camera tonal range, and the scenes being photographed. 149 150The only thing you have to take care, is to point _darktable-curve-tool_ to the same save 151state file with the option _-s_ (which stands for **s**tate file). Let's say you specify 152the _-s_ option even on first run like this 153 154 $ darktable-curve-tool -s "$HOME/tmp/mycamera.dat" FILE-raw.ppm FILE-jpeg.ppm 155 156You are then able to accumulate more data for this camera doing something like this 157 158 $ darktable-curve-tool -s "$HOME/tmp/mycamera.dat" FILE-raw2.ppm FILE-jpeg2.ppm 159 $ darktable-curve-tool -s "$HOME/tmp/mycamera.dat" FILE-raw3.ppm FILE-jpeg3.ppm 160 ... 161 $ darktable-curve-tool -s "$HOME/tmp/mycamera.dat" FILE-rawN.ppm FILE-jpegN.ppm 162 163Beware that _darktable-curve-tool_ uses 32bit counters internally to keep track of the number 164of times a RGB/Lab sample has been encountered. As cameras these days do have many pixels 165a photo, do not be zealous; do not run the tool on your complete catalog. In the 166case too many pixels have been sampled already, an error is printed on the 167console and _darktable-curve-tool_ refuses to process any further image. 168 169It may be smart to pick from 20 to 50 pics covering the whole tonal range of your 170camera; there is no need for thousands of pictures, firstly, it'd be real slow, and 171secondly the resulting accuracy would not be improved significantly. 172 173It is now time to analyse the data and output the curves. 174 175 176### Analysing and outputting the curves 177 178 179So you gathered data in `$HOME/tmp/mycamera.dat`, that's perfect. Let's compute the 180curves now. 181 182 $ darktable-curve-tool -z -e <one of the RAW files> -s ~/tmp/mycamera.dat | tee mycameracurves.sh 183 [this will print you a script on screen and in the mycameracurves.sh file] 184 185Little explanation before trying out the computed curves. 186 187The _-z_ option tells the _darktable-curve-tool_ program to read the save state and compute 188the curves. The _-e_ option is just a convenient option for pointing _darktable-curve-tool_ 189to a file containing EXIF data that can provide your camera Model name. 190 191You can generate curves with more or less points to approximate the values 192gathered during step 1. See option _-n_. The tool does not accept more than 19320 points maximum. Something between 10 to 16 should be enough. 194 195 196### Applying the curves 197 198 199Feeling adventurous ? Ready to try your curves ? 200 201First backup your darktable database: 202 203 $ cp "$HOME/.config/darktable/data.db" "$HOME/.config/darktable/data.db.bck" 204 205Then go on, import the curves: 206 207 $ sh mycameracurves.sh 208 209Spawn _darktable_, and check you got a new curve in the tonecurve module presets 210and the basecurve module presets. If you provided the _-e_ option to the final 211_darktable-curve-tool_ command run, the preset should be named as your camera Model name. 212Otherwise, they will be named 'measured basecurve/tonecurve' 213 214### Applying the curves automatically 215 216Do not hesitate to setup the preset so that it is automatically applied when you 217import/edit photos from this camera model. 218 219Use the usual darktable GUI options for that. Either global options preset editor 220or the module preset little tiny button once you selected that preset->Edit preset. 221 222### Plotting the data for checking validity of data and fit 223 224On the final tool invocation (with _-z_ option) you may be interested in looking at 225what _darktable-curve-tool_ munged and analysed for you. 226 227Two [GNUPlot](http://gnuplot.info/) scripts are provided in the same source directory to do so. 228They require files `basecurve.dat` and `basecurve.fit.dat` resp. `tonecurve.dat` 229and `tonecurve.fit.dat` to be present in the `$DARKATBLE_SRC_ROOT/tools/basecurve` directory. 230 231 $ gnuplot -c "@CMAKE_INSTALL_FULL_DATAROOTDIR@/darktable/tools/basecurve/gnuplot.tonecurve" 232 $ gnuplot -c "@CMAKE_INSTALL_FULL_DATAROOTDIR@/darktable/tools/basecurve/gnuplot.basecurve" 233 234This generates a `basecurve.pdf` resp. `tonecurve.pdf` file with a graph of 235the gathered data and the fitted curves. This can help you measuring how much of 236the tonal range your sampling photos have covered. 237