1function resultCell = startmulticoremaster(functionHandleCell, parameterCell, ... 2 multicoreDir, maxMasterEvaluations) 3%STARTMULTICOREMASTER Start multi-core processing master process. 4% RESULTCELL = STARTMULTICOREMASTER(FHANDLE, PARAMETERCELL, DIRNAME) 5% starts a multi-core processing master process. The function specified 6% by the given function handle is evaluated with the parameters saved in 7% each cell of PARAMETERCELL. Each cell may include parameters in any 8% form or another cell array which is expanded to an argument list using 9% the {:} notation to pass multiple input arguments. The outputs of the 10% function are returned in cell array RESULTCELL of the same size as 11% PARAMETERCELL. Only the first output argument of the function is 12% returned. If you need to get multiple outputs, write a small adapter 13% function which puts the outputs of your function into a cell array. 14% 15% To make use of multiple cores/machines, function STARTMULTICOREMASTER 16% saves files with the function handle and the parameters to the given 17% directory DIRNAME. These files are loaded by function 18% STARTMULTICORESLAVE running on other Matlab processes which have access 19% to the same directory. The slave processes evaluate the given function 20% with the saved parameters and save the result in another file into the 21% same directory. The results are later collected by the master process. 22% 23% RESULTCELL = STARTMULTICOREMASTER(FHANDLE, PARAMETERCELL, DIRNAME, 24% MAXMASTEREVALUATIONS) restricts the number of function evaluations done 25% by the master process to MAXMASTEREVALUATIONS. After the given number 26% of evaluations is reached, the master process waits for other processes 27% to complete instead of working down the list of parameter sets itself. 28% Use this value if the overall number of function evaluations is quite 29% small or the duration of a single evaluation is very long. However, 30% note that if the slave process is interrupted, the master process will 31% never terminate the computations! 32% 33% Note that you can make use of multiple cores on a single machine or on 34% different machines with a commonly accessible directory/network share 35% or a combination of both. 36% 37% RESULTCELL = STARTMULTICOREMASTER(FHANDLE, PARAMETERCELL) uses the 38% directory <TEMPDIR2>/multicorefiles, where <TEMPDIR2> is the directory 39% returned by function TEMPDIR2. Use this form if you are running on 40% multiple cores on a single machine. 41% 42% RESULTCELL = STARTMULTICOREMASTER(FHANDLECELL, PARAMETERCELL, ...), 43% with a cell array FHANDLECELL including function handles, allows to 44% evaluate different functions. 45% 46% Example: If you have your parameters saved in parameter cell 47% PARAMETERCELL, the for-loop 48% 49% for k=1:numel(PARAMETERCELL) 50% RESULTCELL{k} = FHANDLE(PARAMETERCELL{k}); 51% end 52% 53% which you would run in a single process can be run in parallel on 54% different cores/machines using STARTMULTICOREMASTER and 55% STARTMULTICORESLAVE. Run 56% 57% RESULTCELL = STARTMULTICOREMASTER(FHANDLE, PARAMETERCELL, DIRNAME) 58% 59% in one Matlab process and 60% 61% STARTMULTICORESLAVE(DIRNAME) 62% 63% in other Matlab processes. 64% 65% Markus Buehren 66% Last modified 13.11.2007 67% 68% See also STARTMULTICORESLAVE, FUNCTION_HANDLE. 69 70startWaitTime = 0.1; 71maxWaitTime = 2; 72 73% check input arguments 74if isa(functionHandleCell, 'function_handle') 75 functionHandleCell = repmat({functionHandleCell}, size(parameterCell)); 76else 77 if ~iscell(functionHandleCell) 78 error('First input argument must be a function handle or a cell array of function handles.'); 79 elseif any(size(functionHandleCell) ~= size(parameterCell)) 80 error('Input cell arrays functionHandleCell and parameterCell must be of the same size.'); 81 end 82end 83nOfFiles = numel(functionHandleCell); 84if ~iscell(parameterCell) 85 error('Second input argument must be a cell array.'); 86end 87if ~exist('multicoreDir', 'var') 88 multicoreDir = ''; 89elseif ~ischar(multicoreDir) 90 error('Third input argument must be a string.'); 91end 92if exist('maxMasterEvaluations', 'var') 93 if isempty(maxMasterEvaluations) || maxMasterEvaluations > nOfFiles 94 maxMasterEvaluations = nOfFiles; 95 elseif maxMasterEvaluations < 0 96 error('Maximum number of master evaluations must not be negative.'); 97 end 98else 99 maxMasterEvaluations = nOfFiles; 100end 101 102% create slave file directory if not existing 103if isempty(multicoreDir) 104 multicoreDir = fullfile(tempdir2, 'multicorefiles'); 105end 106if ~exist(multicoreDir, 'dir') 107 try 108 mkdir(multicoreDir); 109 catch 110 error('Unable to create slave file directory %s.', multicoreDir); 111 end 112end 113 114% remove all existing parameter files 115existingSlaveFiles = [findfiles(multicoreDir, 'parameters_*.mat'), findfiles(multicoreDir, 'result_*.mat')]; 116for k=1:length(existingSlaveFiles) 117 deletewithsemaphores(existingSlaveFiles{k}); 118end 119 120% build parameter file name (including the date is important because slave 121% processes might still be working with old parameters) 122dateStr = sprintf('%04d%02d%02d%02d%02d%02d', round(clock)); 123parameterFileNameTemplate = fullfile(multicoreDir, sprintf('parameters_%s_XX.mat', dateStr)); 124 125% generate parameter files 126if maxMasterEvaluations == nOfFiles 127 % save parameter files with all parameter sets except the first one 128 % save files with highest numbers first 129 fileIndex = nOfFiles:-1:2; 130else 131 % only save parameter files for slave processes 132 fileIndex = nOfFiles:-1:maxMasterEvaluations+1; 133end 134for fileNr = fileIndex 135 parameterFileName = strrep(parameterFileNameTemplate, 'XX', sprintf('%04d', fileNr)); 136 functionHandle = functionHandleCell{fileNr}; %#ok 137 parameters = parameterCell {fileNr}; %#ok 138 sem = setfilesemaphore(parameterFileName); 139 save(parameterFileName, 'functionHandle', 'parameters'); 140 removefilesemaphore(sem); 141end 142 143% evaluate function 144resultCell = cell(size(functionHandleCell)); 145for fileNr = 1:maxMasterEvaluations 146 parameterFileName = strrep(parameterFileNameTemplate, 'XX', sprintf('%04d', fileNr)); 147 resultFileName = strrep(parameterFileName, 'parameters', 'result'); 148 149 % remove parameter file if existing, so that a slave process does not load it 150 sem = setfilesemaphore(parameterFileName); 151 if existfile(parameterFileName) 152 delete(parameterFileName); 153 end 154 removefilesemaphore(sem); 155 156 % check if the current parameter set was evaluated before by a slave process 157 resultLoaded = false; 158 sem = setfilesemaphore(resultFileName); 159 if existfile(resultFileName) 160 load(resultFileName, 'result'); 161 delete(resultFileName); 162 resultCell{fileNr} = result; 163 resultLoaded = true; 164 end 165 removefilesemaphore(sem); 166 167 % evaluate function if the result could not be loaded 168 if ~resultLoaded 169 functionHandle = functionHandleCell{fileNr}; %#ok 170 parameters = parameterCell {fileNr}; %#ok 171 if iscell(parameters) 172 resultCell{fileNr} = feval(functionHandle, parameters{:}); 173 else 174 resultCell{fileNr} = feval(functionHandle, parameters); 175 end 176 end 177end 178 179% if number of master evaluations is restricted, wait for results of slaves 180if maxMasterEvaluations < nOfFiles 181 for fileNr = nOfFiles:-1:maxMasterEvaluations+1 182 parameterFileName = strrep(parameterFileNameTemplate, 'XX', sprintf('%04d', fileNr)); 183 resultFileName = strrep(parameterFileName, 'parameters', 'result'); 184 185 resultLoaded = false; 186 curWaitTime = startWaitTime; 187 while 1 188 sem = setfilesemaphore(resultFileName); 189 if existfile(resultFileName) 190 load(resultFileName, 'result'); 191 delete(resultFileName); 192 resultCell{fileNr} = result; 193 resultLoaded = true; 194 end 195 removefilesemaphore(sem); 196 if resultLoaded 197 break 198 end 199 200 % wait a moment before checking again 201 pause(curWaitTime); 202 curWaitTime = min(maxWaitTime, curWaitTime + startWaitTime); 203 end 204 end 205end 206 207