1# MCollective Puppet Agent 2 3This agent manages the *puppet agent*, unlike the older *puppetd* plugin 4this one supports Puppet 3 and recent changes made to its locking and status files. 5 6In addition to basic support for Puppet 3 and 4 this adds a number of new features, most 7them usable under Puppet 2.7 and later. 8 9 * Supports noop runs or no-noop runs 10 * Supports limiting runs to certain tags 11 * Support splay, no splay, splaylimits 12 * Supports specifying a custom environment 13 * Supports specifying a custom master host and port (needs to be explicitly allowed) 14 * Support Puppet 3 features like lock messages when disabling 15 * Use the new summary plugins to provide convenient summaries where appropriate 16 * Use the new validation plugins to provider richer input validation and better errors 17 * Data sources for the current puppet agent status and the status of the most recent run 18 19To use this agent you need: 20 21 * MCollective 2.2.0 at least 22 * Puppet 2.7 or later 23 24# TODO 25 26 * Add a new puppet commander 27 28## Agent Installation 29 30Follow the basic [plugin install guide](http://docs.puppetlabs.com/mcollective/deploy/plugins.html) 31 32## Configuring the agent 33 34By default it just works but there are a few settings you can tweak in *server.cfg*: 35 36 plugin.puppet.command = puppet agent 37 plugin.puppet.splay = true 38 plugin.puppet.splaylimit = 30 39 # plugin.puppet.config leaves it to Puppet's default location 40 plugin.puppet.windows_service = puppet 41 plugin.puppet.signal_daemon = true 42 43These are the defaults, adjust to taste. 44 45If `plugin.puppet.command` is not set, it will try to find `puppet` via the PATH 46environment variable. On non-Windows systems, `/opt/puppetlabs/bin` will be appended 47to PATH if the `command` doesn't include a file path. 48 49> **Warning**: If Puppet is not on the PATH and you are not using the `puppet-agent` 50package provided by Puppet, this can result in running a binary placed by any user that 51has write access to `/opt`. If that is a concern, ensure `plugin.puppet.command` is configured. 52 53The agent allows managing of any resource via the Puppet RAL. By default it refuses to 54manage a resource also managed by Puppet which could create conflicting state. If you 55do wish to allow any resources to be managed set this to true: 56 57 plugin.puppet.resource_allow_managed_resources = true 58 59The resource action can manage any resource type Puppet can, by default we blacklist 60the all types due to the potential damage this feature can do to your system if not 61correctly setup. You can specify either a whitelist or a blacklist of types this 62agent will be able to manage - you cannot specify both a blacklist and a whitelist. 63 64 plugin.puppet.resource_type_whitelist = host,alias 65 plugin.puppet.resource_type_blacklist = exec 66 67If you supply the value *none* to *type_whitelist* it will have the effect of denying 68all resource management - this is the default. 69 70On Windows, the name of the Puppet service is needed to determine if the 71service is running. The service name varies between Puppet open source and 72Puppet Enterprise (puppet vs. pe-puppet); the default is puppet, but it can be 73explicitly specified: 74 75 plugin.puppet.windows_service = puppet 76 77The agent will by default invoke `plugin.puppet.command` to initiate a 78run, passing through any applicable flags to adjust behavior. On 79POSIX-compliant platforms where Puppet is already running in 80daemonized mode we support sending the daemonized service a USR1 81signal to trigger the daemonized process to perform an immediate 82check-in. This will inhibit customizations to the run (such as noop 83or environment), but it is the default. It's reccomended that you 84disable this like so: 85 86 plugin.puppet.signal_daemon = false 87 88The agent will not by default accept the server option. If passed then 89the agent returns an error. Passing the option can be allowed in the 90configuration file like so: 91 92 plugin.puppet.allow_server_override = true 93 94 95## Usage 96### Running Puppet 97 98Most basic case is just a run: 99 100 $ mco puppet runonce 101 102...against a specific server and port (needs to be explicitly allowed): 103 104 $ mco puppet runonce --server puppet.example.net:1234 105 106...just some tags 107 108 $ mco puppet runonce --tag one --tag two --tag three 109 $ mco puppet runonce --tag one,two,three 110 $ mco puppet runonce --tags one,two,three 111 112...a noop run 113 114 $ mco puppet runonce --noop 115 116...a actual run when noop is set in the config file 117 118 $ mco puppet runonce --no-noop 119 120...in a specific environment 121 122 $ mco puppet runonce --environment development 123 124...a splay run 125 126 $ mco puppet runonce --splay --splaylimit 120 127 128...or if you have splay on by default and do not want to splay 129 130 $ mco puppet runonce --no-splay 131 132...or if you want to ignore schedules for a single run 133 134 $ mco puppet runonce --ignoreschedules 135 136These can all be combined to your liking 137 138### Requesting agent status 139 140The status of the agent can be obtained: 141 142 $ mco puppet status 143 144 * [ ============================================================> ] 2 / 2 145 146 dev1.example.net: Currently stopped; last completed run 9 minutes 11 seconds ago 147 dev2.example.net: Currently stopped; last completed run 9 minutes 33 seconds ago 148 149 Summary of Applying: 150 151 false = 2 152 153 Summary of Daemon Running: 154 155 stopped = 2 156 157 Summary of Enabled: 158 159 enabled = 2 160 161 Summary of Idling: 162 163 false = 2 164 165 Summary of Status: 166 167 stopped = 2 168 169 170 Finished processing 2 / 2 hosts in 45.01 ms 171 172### Requesting last run status 173 174We can show a graph view of various metrics of the last Puppet run using the 175*mco puppet summary* command. 176 177 $ mco puppet summary 178 179 Summary statistics for 15 nodes: 180 181 Total resources: ▇▁▁▁▁▁▁▁▁▂▁▂▁▁▂▂▁▁▁▂ min: 112.0 avg: 288.9 max: 735.0 182 Out Of Sync resources: ▇▂▁▄▂▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁ min: 0.0 avg: 2.5 max: 7.0 183 Failed resources: ▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ min: 0.0 avg: 0.0 max: 0.0 184 Changed resources: ▇▂▁▄▂▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁ min: 0.0 avg: 2.5 max: 7.0 185 Corrected resources: ▇▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁ min: 0.0 avg: 0.7 max: 2.0 186 Config Retrieval time (seconds): ▇▂▁▁▃▁▃▂▁▁▁▁▁▁▁▁▁▁▁▁ min: 2.4 avg: 6.6 max: 15.0 187 Total run-time (seconds): ▇▃▂▁▃▂▁▁▂▁▁▁▂▁▁▂▁▁▁▁ min: 6.1 avg: 22.9 max: 73.4 188 Time since last run (seconds): ▇▁▂▁▁▂▁▁▁▂▁▁▁▁▁▂▂▂▂▃ min: 86.0 avg: 758.9 max: 1.7k 189 190Here each bar indicates the number of nodes that fall within the region, for 191example we can see there are a group of nodes on the right that took longer 192to run than the others. 193 194You can find which of those nodes took longer than 50 seconds: 195 196 $ mco find -S "resource().total_time>50" 197 198#### Problems with Displaying the Bars 199 200Not all popular SSH clients display the bars correctly. Please ensure your client has UTF-8 201enabled, and uses a suitable font such as [dejavu](http://dejavu-fonts.org/wiki/Main_Page). The 202following clients have been confirmed to work: 203* [PuTTY](http://www.chiark.greenend.org.uk/~sgtatham/putty/) on Windows 204* [mintty](http://code.google.com/p/mintty/) on [Cygwin](www.cygwin.com) on Windows 205 206### Enabling and disabling 207 208Puppet 3 supports a message when enabling and disabling 209 210 $ mco rpc puppet disable message="doing some hand hacking" 211 $ mco rpc puppet enable 212 213The message will be displayed when requesting agent status if it is disabled, 214when no message is supplied a default will be used that will include your 215mcollective caller identity and the time 216 217### Running all enabled Puppet nodes 218 219Often after committing a change you want the change to be rolled out to your 220infrastructure as soon as possible within the performance constraints of your 221infrastructure. 222 223The performance of a Puppet Master generally comes down to the maximum concurrent 224Puppet nodes that are applying a catalog it can sustain. 225 226Using the MCollective infrastructure we can determine how many machines are 227currently enabled and applying catalogs. 228 229Thus to do a Puppet run of your entire infrastructure keeping the concurrent 230Puppet runs as close as possible to 10 nodes at a time you would do: 231 232 $ mco puppet runall 10 233 234Below is the output from a run using a concurrency of 1 to highlight the output 235you might expect: 236 237 $ mco puppet runall 1 238 2013-01-16 16:14:26: Running all nodes with a concurrency of 1 239 2013-01-16 16:14:26: Discovering enabled Puppet nodes to manage 240 2013-01-16 16:14:29: Found 2 enabled nodes 241 2013-01-16 16:14:32: Currently 1 node applying the catalog; waiting for less than 1 242 2013-01-16 16:14:37: dev1.example.net schedule status: Started a background Puppet run using the 'puppet agent --onetime --daemonize --color=false' command 243 2013-01-16 16:14:38: 1 out of 2 hosts left to run in this iteration 244 2013-01-16 16:14:40: Currently 1 node applying the catalog; waiting for less than 1 245 2013-01-16 16:14:44: Currently 1 node applying the catalog; waiting for less than 1 246 2013-01-16 16:14:48: Currently 1 node applying the catalog; waiting for less than 1 247 2013-01-16 16:14:52: Currently 1 node applying the catalog; waiting for less than 1 248 2013-01-16 16:14:56: Currently 1 node applying the catalog; waiting for less than 1 249 2013-01-16 16:15:00: Currently 1 node applying the catalog; waiting for less than 1 250 2013-01-16 16:15:04: Currently 1 node applying the catalog; waiting for less than 1 251 2013-01-16 16:15:08: Currently 1 node applying the catalog; waiting for less than 1 252 2013-01-16 16:15:13: dev2.example.net schedule status: Started a background Puppet run using the 'puppet agent --onetime --daemonize --color=false' command 253 254Here we can see it first finds all machine that are enabled and then periodically 255checks if the amount of instances currently applying a catalog is less than the 256concurrency and then start as many machines as it can till it once again reaches 257the concurrency limit. 258 259Note that you can pass flags like --noop and --no-noop but the splay settings will not work 260as the runall command does forced runs which negates splay. 261 262If you wish to repeat this in a loop forever you can pass the --rerun argument giving it 263the minimum amount of time a loop over all the nodes must take: 264 265 $ mco puppet runall 1 --rerun 3600 266 267This performs the same run logic as before but when it comes to the end of the run it 268will sleep for the difference between 3600 seconds and how long the run took. If the 269run took longer than 3600 seconds it will immediately start a new one. 270 271### Discovering based on agent status 272 273Two data plugins are provided, to see what data is available about the running 274agent do: 275 276 $ mco rpc rpcutil get_data source=puppet 277 Discovering hosts using the mc method for 2 second(s) .... 1 278 279 * [ ============================================================> ] 1 / 1 280 281 282 dev1.example.net 283 applying: false 284 daemon_present: false 285 disable_message: 286 enabled: true 287 lastrun: 1348745262 288 since_lastrun: 7776 289 status: stopped 290 291 Finished processing 1 / 1 hosts in 76.34 ms 292 293You can then use any of this data in discovery, to restart apache on machines 294with Puppet disable can now be done easily: 295 296 $ mco rpc service restart service=httpd -S "puppet().enabled=false" 297 298You can restart apache on all machine that has File[/srv/www] managed by Puppet: 299 300 $ mco rpc service restart service=httpd -S "resource('file[/srv/wwww]').managed=true" 301 302...or machines that had many changes in the most recent run: 303 304 $ mco rpc service restart service=httpd -S "resource().changed_resources>10" 305 306...or that had failures 307 308 $ mco rpc service restart service=httpd -S "resource().failed_resources>0" 309 310Other available data include config_retrieval_time, config_version, lastrun, 311out_of_sync_resources, since_lastrun, total_resources and total_time 312 313### Integration with the Action Policy Authorization plugin 314 315The Action Policy plugin supports querying the above data plugins to express 316Authorization rules. 317 318You can therefore use the enabled state of the Puppet Agent to limit access 319to other actions. 320 321The use case would be that you want: 322 323 * Only allow services to be stopped during maintenance periods when Puppet is disabled 324 * Only allow the site manager to disable Puppet 325 326You can control the service agent with the following policy using the *service.policy* 327file: 328 329 allow cert=joe stop puppet().enabled=false 330 331And you can then allow the manager user to disable and enable nodes using the 332*puppet.policy* file: 333 334 allow cert=manager disable * * 335 allow cert=manager enable * * 336 337Together this allows you to ensure that you both have a maintenance window and a 338period where Puppet will not start services again without your knowledge 339 340Note: The runall action is implemented in terms of the runonce action. When setting up Actionpolicy rules, be sure to include a runonce action permission. 341 342### Managing individual resources using the RAL 343 344Puppet is built on resource types and providers for those types, an instance of 345a resource type looks like: 346 347 host{"db": 348 ip => "192.168.1.10" 349 } 350 351These are known as the Resource Abstraction Layer or the RAL. 352 353You can use MCollective to manage individual parts of your servers using the 354RAL. 355 356To add a host entry to your webservers matching the above resource you can 357do the following: 358 359 $ mco puppet resource host db ip=192.168.1.11 -W role::webserver 360 361 * [ ============================================================> ] 11 / 11 362 363 364 node4.example.net 365 Result: ip changed '192.168.1.10' to '192.168.1.11' 366 . 367 . 368 Summary of Changed: 369 370 Changed = 1 371 372 Finished processing 11 / 11 hosts in 118.97 ms 373 374Here we used the RAL to change the hosts entry for the hostname *db* to 192.168.1.11 375and the output shows you it changed from a previous value to this entry. 376 377Any hosts where the operation failed will fail in the normal manner 378 379This is a very dangerous feature as people can make real changes to your machines 380and potentially cause all kinds of problems. 381 382We support a few restrictions: 383 384 * You can whitelist or blacklist which types can be executed, you want to avoid 385 exec types for example 386 * You can whitelist or blacklist which resource name can be executed, you want to avoid 387 ssh package name for example 388 * You can allow or deny the ability to change resources that Puppet is also managing 389 as you'd want to avoid creating conflicting state 390 391By default if not specifically configured this feature is not usable as it defaults 392to the following configuration: 393 394 395 plugin.puppet.resource_allow_managed_resources = true 396 plugin.puppet.resource_type_whitelist = none 397 398You can allow all types except the exec, service and package types using the 399following config line: 400 401 plugin.puppet.resource_type_blacklist = exec,service,package 402 403You can say which resource names are allowed or denied. You define whitelist or blacklist 404for resource type by adding resource type after plugin.puppet.resource_name_whitelist or 405plugin.puppet.resource_name_blacklist, for example: 406 407 plugin.puppet.resource_name_blacklist.package = ssh 408 409If you not defined list for resource type, all names are allowed. 410 411You cannot mix and match white and black lists. 412 413So to repeat by default this feature is effectively turned off as there is an empty 414whitelist by default - no types are allowed to be managed. You should think carefully 415before enabling this feature and combine it with the Authorization system when you do 416 417## Maintenance 418 419Maintainers: Alessandro Parisi <alessandro@puppet.com>, Michael Smith 420<michael.smith@puppet.com>, Michal Ruzicka <michal.ruzicka@puppet.com>. 421 422Tickets: File bug tickets at https://tickets.puppet.com/browse/MCOP. 423