1.. include:: ../global.inc 2.. _decorators.files_on_the_fly: 3.. index:: 4 pair: @files (on-the-fly parameter generation); Syntax 5 6.. seealso:: 7 8 * :ref:`@files <new_manual.on_the_fly>` in the **Ruffus** Manual 9 * :ref:`Decorators <decorators>` for more decorators 10 11.. |custom_function| replace:: `custom_function` 12.. _custom_function: `decorators.files.custom_function`_ 13 14 15################################################ 16Parameters on the fly with @files 17################################################ 18 19******************************************************************************************* 20*@files* (|custom_function|_) 21******************************************************************************************* 22 **Purpose:** 23 24 Uses a custom function to generate sets of parameters to separate jobs which can run in parallel. 25 26 The first two parameters in each set represent the input and output which are 27 used to see if the job is out of date and needs to be (re-)run. 28 29 By default, out of date checking uses input/output file timestamps. 30 (On some file systems, timestamps have a resolution in seconds.) 31 See :ref:`@check_if_uptodate() <decorators.check_if_uptodate>` for alternatives. 32 33 **Example**: 34 :: 35 36 from ruffus import * 37 def generate_parameters_on_the_fly(): 38 parameters = [ 39 ['input_file1', 'output_file1', 1, 2], # 1st job 40 ['input_file2', 'output_file2', 3, 4], # 2nd job 41 ['input_file3', 'output_file3', 5, 6], # 3rd job 42 ] 43 for job_parameters in parameters: 44 yield job_parameters 45 46 @files(generate_parameters_on_the_fly) 47 def parallel_io_task(input_file, output_file, param1, param2): 48 pass 49 50 pipeline_run([parallel_task]) 51 52 is the equivalent of calling: 53 :: 54 55 parallel_io_task('input_file1', 'output_file1', 1, 2) 56 parallel_io_task('input_file2', 'output_file2', 3, 4) 57 parallel_io_task('input_file3', 'output_file3', 5, 6) 58 59 60 **Parameters:** 61 62 63.. _decorators.files.custom_function: 64 65 * *custom_function*: 66 Generator function which yields each time a complete set of parameters for one job 67 68 **Checking if jobs are up to date:** 69 Strings in ``input`` and ``output`` (including in nested sequences) are interpreted as file names and 70 used to check if jobs are up-to-date. 71 72 See :ref:`above <decorators.files.check_up_to_date>` for more details 73 74 75 76 77 78