1.. include:: ../../global.inc 2.. include:: manual_chapter_numbers.inc 3 4.. _new_manual.inputs.code: 5 6############################################################################################################################################################################################################ 7|new_manual.inputs.chapter_num|: Python Code for Manipulating task inputs via string substitution using :ref:`inputs() <decorators.inputs>` and :ref:`add_inputs() <decorators.add_inputs>` 8############################################################################################################################################################################################################ 9 10.. seealso:: 11 12 * :ref:`Manual Table of Contents <new_manual.table_of_contents>` 13 * :ref:`inputs() <decorators.inputs>` syntax 14 * :ref:`add_inputs() <decorators.add_inputs>` syntax 15 * Back to |new_manual.inputs.chapter_num|: :ref:`Manipulating task inputs via string substitution <new_manual.inputs>` 16 17****************************************************************************************************************************************************** 18Example code for adding additional *input* prerequisites per job with :ref:`add_inputs() <decorators.add_inputs>` 19****************************************************************************************************************************************************** 20 21.. _new_manual.inputs.example1: 22 23=================================================================== 241. Example: compiling c++ code 25=================================================================== 26 27 .. code-block:: python 28 29 # source files exist before our pipeline 30 source_files = ["hasty.cpp", "tasty.cpp", "messy.cpp"] 31 for source_file in source_files: 32 open(source_file, "w") 33 34 from ruffus import * 35 36 @transform(source_files, suffix(".cpp"), ".o") 37 def compile(input_filename, output_file): 38 open(output_file, "w") 39 40 pipeline_run() 41 42 43 Giving: 44 45 .. code-block:: pycon 46 47 >>> pipeline_run() 48 Job = [hasty.cpp -> hasty.o] completed 49 Job = [messy.cpp -> messy.o] completed 50 Job = [tasty.cpp -> tasty.o] completed 51 Completed Task = compile 52 53.. _new_manual.inputs.example2: 54 55====================================================================================================================================== 562. Example: Adding a common header file with :ref:`add_inputs() <decorators.add_inputs>` 57====================================================================================================================================== 58 59 60 .. code-block:: python 61 :emphasize-lines: 12 62 63 # source files exist before our pipeline 64 source_files = ["hasty.cpp", "tasty.cpp", "messy.cpp"] 65 for source_file in source_files: 66 open(source_file, "w") 67 68 # common (universal) header exists before our pipeline 69 open("universal.h", "w") 70 71 from ruffus import * 72 73 @transform( source_files, suffix(".cpp"), 74 # add header to the input of every job 75 add_inputs("universal.h"), 76 ".o") 77 def compile(input_filename, output_file): 78 open(output_file, "w") 79 80 pipeline_run() 81 82 Giving: 83 84 .. code-block:: pycon 85 86 >>> pipeline_run() 87 Job = [[hasty.cpp, universal.h] -> hasty.o] completed 88 Job = [[messy.cpp, universal.h] -> messy.o] completed 89 Job = [[tasty.cpp, universal.h] -> tasty.o] completed 90 Completed Task = compile 91 92.. _new_manual.inputs.example3: 93 94===================================================================== 953. Example: Additional *Input* can be tasks 96===================================================================== 97 98 .. code-block:: python 99 :emphasize-lines: 11,17,19 100 101 # source files exist before our pipeline 102 source_files = ["hasty.cpp", "tasty.cpp", "messy.cpp"] 103 for source_file in source_files: 104 open(source_file, "w") 105 106 # common (universal) header exists before our pipeline 107 open("universal.h", "w") 108 109 from ruffus import * 110 111 # make header files 112 @transform(source_files, suffix(".cpp"), ".h") 113 def create_matching_headers(input_file, output_file): 114 open(output_file, "w") 115 116 @transform(source_files, suffix(".cpp"), 117 # add header to the input of every job 118 add_inputs("universal.h", 119 # add result of task create_matching_headers to the input of every job 120 create_matching_headers), 121 ".o") 122 def compile(input_filename, output_file): 123 open(output_file, "w") 124 125 pipeline_run() 126 127 Giving: 128 129 .. code-block:: pycon 130 131 132 >>> pipeline_run() 133 Job = [hasty.cpp -> hasty.h] completed 134 Job = [messy.cpp -> messy.h] completed 135 Job = [tasty.cpp -> tasty.h] completed 136 Completed Task = create_matching_headers 137 Job = [[hasty.cpp, universal.h, hasty.h, messy.h, tasty.h] -> hasty.o] completed 138 Job = [[messy.cpp, universal.h, hasty.h, messy.h, tasty.h] -> messy.o] completed 139 Job = [[tasty.cpp, universal.h, hasty.h, messy.h, tasty.h] -> tasty.o] completed 140 Completed Task = compile 141 142.. _new_manual.inputs.example4: 143 144================================================================================================================================================================================================================================================ 1454. Example: Add corresponding files using :ref:`add_inputs() <decorators.add_inputs>` with :ref:`formatter <decorators.formatter>` or :ref:`regex <decorators.regex>` 146================================================================================================================================================================================================================================================ 147 148 .. code-block:: python 149 :emphasize-lines: 11,17,19 150 151 # source files exist before our pipeline 152 source_files = ["hasty.cpp", "tasty.cpp", "messy.cpp"] 153 header_files = ["hasty.h", "tasty.h", "messy.h"] 154 for source_file in source_files + header_files: 155 open(source_file, "w") 156 157 # common (universal) header exists before our pipeline 158 open("universal.h", "w") 159 160 from ruffus import * 161 162 @transform( source_files, 163 formatter(".cpp$"), 164 # corresponding header for each source file 165 add_inputs("{basename[0]}.h", 166 # add header to the input of every job 167 "universal.h"), 168 "{basename[0]}.o") 169 def compile(input_filename, output_file): 170 open(output_file, "w") 171 172 pipeline_run() 173 174 Giving: 175 176 .. code-block:: pycon 177 178 >>> pipeline_run() 179 Job = [[hasty.cpp, hasty.h, universal.h] -> hasty.o] completed 180 Job = [[messy.cpp, messy.h, universal.h] -> messy.o] completed 181 Job = [[tasty.cpp, tasty.h, universal.h] -> tasty.o] completed 182 Completed Task = compile 183 184********************************************************************************************* 185Example code for replacing all input parameters with :ref:`inputs() <decorators.inputs>` 186********************************************************************************************* 187 188.. _new_manual.inputs.example5: 189 190================================================================================================================================================================================================================================================ 1915. Example: Running matching python scripts using :ref:`inputs() <decorators.inputs>` 192================================================================================================================================================================================================================================================ 193 194 .. code-block:: python 195 :emphasize-lines: 11,17,19 196 197 # source files exist before our pipeline 198 source_files = ["hasty.cpp", "tasty.cpp", "messy.cpp"] 199 python_files = ["hasty.py", "tasty.py", "messy.py"] 200 for source_file in source_files + python_files: 201 open(source_file, "w") 202 203 # common (universal) header exists before our pipeline 204 open("universal.h", "w") 205 206 from ruffus import * 207 208 @transform( source_files, 209 formatter(".cpp$"), 210 # corresponding python file for each source file 211 inputs("{basename[0]}.py"), 212 213 "{basename[0]}.results") 214 def run_corresponding_python(input_filenames, output_file): 215 open(output_file, "w") 216 217 218 pipeline_run() 219 220 Giving: 221 222 .. code-block:: pycon 223 224 >>> pipeline_run() 225 Job = [hasty.py -> hasty.results] completed 226 Job = [messy.py -> messy.results] completed 227 Job = [tasty.py -> tasty.results] completed 228 Completed Task = run_corresponding_python 229 230