1.. include:: ../global.inc
2.. _decorators.files_on_the_fly:
3.. index::
4    pair: @files (on-the-fly parameter generation); Syntax
5
6.. seealso::
7
8    * :ref:`@files <new_manual.on_the_fly>` in the **Ruffus** Manual
9    * :ref:`Decorators <decorators>` for more decorators
10
11.. |custom_function| replace:: `custom_function`
12.. _custom_function: `decorators.files.custom_function`_
13
14
15################################################
16Parameters on the fly with @files
17################################################
18
19*******************************************************************************************
20*@files* (|custom_function|_)
21*******************************************************************************************
22    **Purpose:**
23
24        Uses a custom function to generate sets of parameters to separate jobs which can run in parallel.
25
26        The first two parameters in each set represent the input and output which are
27        used to see if the job is out of date and needs to be (re-)run.
28
29        By default, out of date checking uses input/output file timestamps.
30        (On some file systems, timestamps have a resolution in seconds.)
31        See :ref:`@check_if_uptodate() <decorators.check_if_uptodate>` for alternatives.
32
33    **Example**:
34        ::
35
36            from ruffus import *
37            def generate_parameters_on_the_fly():
38                parameters = [
39                                    ['input_file1', 'output_file1', 1, 2], # 1st job
40                                    ['input_file2', 'output_file2', 3, 4], # 2nd job
41                                    ['input_file3', 'output_file3', 5, 6], # 3rd job
42                             ]
43                for job_parameters in parameters:
44                    yield job_parameters
45
46            @files(generate_parameters_on_the_fly)
47            def parallel_io_task(input_file, output_file, param1, param2):
48                pass
49
50            pipeline_run([parallel_task])
51
52    is the equivalent of calling:
53        ::
54
55            parallel_io_task('input_file1', 'output_file1', 1, 2)
56            parallel_io_task('input_file2', 'output_file2', 3, 4)
57            parallel_io_task('input_file3', 'output_file3', 5, 6)
58
59
60    **Parameters:**
61
62
63.. _decorators.files.custom_function:
64
65        * *custom_function*:
66            Generator function which yields each time a complete set of parameters for one job
67
68    **Checking if jobs are up to date:**
69        Strings in ``input`` and ``output`` (including in nested sequences) are interpreted as file names and
70        used to check if jobs are up-to-date.
71
72        See :ref:`above <decorators.files.check_up_to_date>` for more details
73
74
75
76
77
78