1<?xml version="1.0" encoding="UTF-8"?> 2<!DOCTYPE egbokdoc SYSTEM "http://www.egbok.com/dtd/egbokdoc.dtd"> 3<!-- $Id: PORCMOLSULB.xml,v 1.9 2003/04/21 23:39:03 hbo Exp $ --> 4<egbokdoc> 5 6<title> 7The Problem of PORCMOLSULB 8</title> 9<subtitle> 10Can Root be Controlled in Engineering Environments? 11</subtitle> 12 13<maintainer> 14 <name>Howard Owen</name> 15 <affiliation>EGBOK Consultants</affiliation> 16 <email>hbo@egbok.com</email> 17</maintainer> 18 19<copyright> 20 Copyright © 2002 EGBOK Consultants. Some Rights Reserved.<br/> 21 This work is licensed under the Creative Commons Attribution License. <br/> 22 To view a copy of this license, visit <link href="http://creativecommons.org/licenses/by/1.0/" text="http://creativecommons.org/licenses/by/1.0/"/><br/> 23or send a letter to Creative Commons, 559 Nathan Abbott Way, Stanford, California 94305, USA.<br/> 24 Originally published in the August 2002 issue of ;login, the magazine of USENIX and SAGE. 25</copyright> 26 27<content> 28 29<section title="Introduction"> 30<para> 31 32 The other day I got a call from a user who was having trouble 33 checking in her latest changes to CVS. Since she was using a Linux 34 box that was not supported by our group, I could have refused to 35 look at the problem. But I'm pathologically interested in making 36 sure our users get the most out of the computing 37 environment. Besides, it could have been an issue with the CVS 38 server, whose health as a system I am responsible for. So I strolled 39 over to her cube and had a look. Our CVS check in process has hooks 40 that cause email to be sent. I soon determined that the problem had 41 to do with the fact that the sendmail binary on her system was owned 42 by her. 43 44<footnote ref="1"> 45 On most Unix systems, the sendmail binary has the "setuid" 46 bit set. This means that when it is run, it takes on the system 47 identity of the user that owns the binary. Usually this is the root 48 user, or some other system account that has permission to write to 49 the mail spool, where temporary mail files are kept. Since the user 50 didn't have this permission, and since the sendmail binary was owned 51 by her, sendmail couldn't write to the spool and mail was broken. 52</footnote> 53 54 Her boss, a senior engineer, had installed Linux for her when she 55 started. He didn't ask the systems group to do the work because we 56 would have set the root password to something he didn't know, and 57 given the user sudo instead. (More about sudo shortly.) I walked 58 over to his cube and asked him if he knew what was up with sendmail 59 being owned by the user. "Sure," he said, "I did a 60 'chown -R <user> /usr" so she wouldn't have permission 61 problems." I'm slightly ashamed to say I laughed out loud 62 before telling him "Well, you are going to have to reinstall 63 Linux." He got annoyed and asked "Why can't you just do 64 'chown -R root /usr'?" I told him why 65 66<footnote ref="2"> 67 In any Unix-like system, the files stored in /usr will be owned by a 68 variety of users. There is usually a reason why a particular file 69 would be owned by a particular user. The example of sendmail being 70 owned by root given above is one case-in-point. Once the ownership 71 is changed through a global chown as in this case, it's very hard to 72 set things back the way they were. It's easier just to reinstall 73 the system. 74</footnote> 75 76 and handed him the Linux CDs. They were back on my desk within 77 twenty minutes, so I knew he had decided to implement his 78 "solution" rather than reinstalling the system. This would 79 solve her email problem, but other problems would surely be 80 created. I told the user that I wouldn't touch her system until 81 Linux was reinstalled. 82</para> 83 84<para> 85 I knew that the senior engineer in this case was no dummy, despite 86 the incredibly bone-headed mistake he had made. He was in fact a 87 very bright guy, engaging and witty in conversation and trusted with 88 a critical role in designing the software our small startup 89 90<footnote ref="3"> 91 I work for a small startup company in Silicon Valley. The company 92 wants to keep their name out of this article. Since I am telling 93 potentially embarrassing stories about our engineers, I wholly agree 94 with this position. The arrangement also allows me to be more frank 95 about certain things, as long as I remain circumspect about others, 96 such as names. 97</footnote> 98 99 was betting everything on. What could account for the extreme wrong 100 headedness he displayed? Why did he resist the reasonable 101 restrictions we asked our users to accept in order to receive 102 support on their personal workstations? How could I reconcile my 103 certain knowledge that this was an extremely sharp and competent 104 senior engineer with the apparently abysmal lack of wit his actions 105 showed? 106 107</para> 108<subsection title="PORCMOLSULB"> 109<para> 110 The above problem is an example of PORCMOLSULB: Proliferation Of 111 Randomly Configured, More-Or-Less Screwed-Up Linux Boxes. It's been 112 showing up more and more in environments I work in. This is partly 113 due to the increasing popularity of Linux, but the main cause is the 114 desire on the part of software engineers to control root on their 115 personal workstations. This desire conflicts with the systems 116 administrator's imperative to maintain systems in a supportable 117 condition and to prevent anonymous damage to other systems on the 118 same network from inexperienced root enabled users. The conflict is 119 based not only on differing goals, but on real differences in the 120 competencies and enthusiasms of the two groups. PORCMOLSULB adds an 121 interesting new twist to the struggle that tends to shift the 122 balance of power toward the users in the ongoing battle. In this 123 paper I will describe the battle in a little more detail, then ask 124 and answer the question "can root access on engineering 125 workstations be controlled in the face of PORCMOLSULB?" 126</para> 127</subsection> 128</section> 129<section title="The Conflict Over Root Access"> 130<para> 131 I've worked as a system administrator for eighteen years, in 132 academia, for government contractors and in private industry. In 133 each of those environments I have found a peculiar local version of 134 low-intensity warfare between the computer users and sysadmins. I 135 hasten to add that this conflict was rarely the only characteristic 136 of relations between the two groups, or even the defining one. 137 Nonetheless, the conflict was always present in some form. The most 138 common form I have seen this conflict take is the struggle over root 139 access. There are compelling business, psychological and technical 140 reasons why this should be so. 141</para> 142<subsection title="The Role of Business Imperatives"> 143<para> 144 From the perspective of the business employing them, systems 145 administrators and technical computer users such as software 146 engineers come to work for the same reasons. That is, to make widgets, 147 grommets, yo-yos or whatever else the enterprise is 148 producing. However looking a little deeper reveals differences in 149 the business roles played by the two types of employees. Generally 150 speaking, businesses hire systems staff to ensure that their 151 computing environments are maintained in a state fit for maximizing 152 the productivity of the enterprise. Software engineers generally are 153 employed to design and write products for sale. It is their 154 productivity that the systems staff must maximize. 155 156<footnote ref="4"> 157 This is a sweeping generalization. There are plenty of systems 158 engineers directly contributing to product, just as there are many 159 software engineers working on the productivity of others. However 160 I'll stand by the generalization for the purposes of this 161 discussion, since my experience tells me it's true more often than 162 not. 163</footnote> 164 165 This difference in business imperatives affects a lot of the 166 interaction between the two groups. Specifically, it shows up when a 167 software engineer demands root access to get her job done. Granting 168 the access may in fact help the engineer to be more productive, at 169 least until she shoots herself in the foot with her rootly 170 power. The sysadmin is bound to see a threat to the stability and 171 security of the systems under his care, and to discount the 172 possibility that any benefit might accrue from granting the access 173 that couldn't also be accomplished with a less sweeping grant of 174 privilege. Before I examine that in more depth, I'll tackle the most 175 difficult to characterize cause of conflict between system 176 administrators and their technical users: the personalties of the 177 people themselves. 178</para> 179</subsection> 180<subsection title="The Role of Personalities"> 181<para> 182 I've always thought that the conflict over root access was 183 particularly strange in the context of Unix software startups in 184 Silicon Valley. It seems to me that Unix systems administrators and 185 Unix software engineers have a lot in common. However the difference 186 in business roles described above, plus differing enthusiasms and 187 capacities, tends to lead bright people with an interest in computer 188 technology down different paths. 189</para> 190<subsubsection title="Apologia"> 191<para> 192 Since I'm a system administrator, I can't avoid telling this part of 193 the story from that perspective. I've tried hard to understand my 194 users, and I've gotten pretty good at it over the years, but the 195 coloration my own place in the scheme of things will lend to my 196 discussion of the personalities involved in this conflict is 197 unavoidable. With that warning issued, I hope you will forgive the 198 personal nature of the discussion that follows. 199</para> 200</subsubsection> 201<subsubsection title="My Choices"> 202<para> 203 Why am I drawn to system administration? Why not be a software 204 engineer, for instance? I do a fair amount of programming in my 205 work. I can code some Perl for several hours, enjoying all the 206 things you must do to program effectively, such as holding several 207 dozen details in your mind at once. Best of all, I love integrating 208 all those details into a finished solution that actually 209 <em>does</em> something. However, I don't like to wait too long to 210 get to that point. I'm impatient. I also get burned out quickly 211 doing that sort of thing. Finally, I get bored really, really 212 easily. Fortunately, as a sysadmin I am compelled to do a lot of 213 other things. I have to deal with other human beings, frequently 214 under difficult circumstances. I work with computer hardware a lot, 215 racking up systems or diving under desks to replace bad 216 components. And best of all, I get to work with computer 217 systems. Unix, Linux, Windows, Palm, it doesn't matter. I love 218 systems. I can make them stand on their heads or dance the two-step. 219 I love the feeling of control and accomplishment that going to the 220 exact center of a difficult problem in complex systems gives me. 221</para> 222</subsubsection> 223<subsubsection title="My Users' Choices" break="page"> 224<para> 225 How is all that different from what a typical software engineer 226 does? This is hard question for me to answer, because I have to try 227 to put myself in the place of an engineer, and I tend to just assume 228 that she thinks exactly the way I do. However there are some clues in 229 the experiences I've had with such engineers that have helped me 230 make the leap of imagination. First, I've noticed that these folks 231 seem to have powers of concentration that are rather absurd, by my 232 standards. Whereas I need to take a break after a couple of hours of 233 coding, these guys stay glued to their screens and keyboards 234 throughout their fourteen hour days. Second, I've noticed that 235 their technical knowledge tends to be less broad than mine, but 236 deeper. Both of these observations start to add up to a (perhaps) 237 obvious conclusion: <em>software engineers are specialists.</em> 238 Another conclusion I've drawn has taken a lot longer to arrive at, 239 because it cuts so directly against my own stance toward 240 technology. <em>Software engineers are generally not enthusiastic 241 about computer systems.</em> Instead, they are enthusiastic about 242 <em>software!</em> They view systems as a vehicle for software, a 243 means to an end. I view systems as ends in themselves. Once this 244 idea struck me, I marveled at how long it took me to see it. It 245 seems that both systems administrators and software engineers are 246 constitutionally suited for the differing roles they are asked to 247 play in the enterprises that employ them. 248</para> 249<para> 250 Now that we've introduced the players, let's set the scene: Unix in 251 all its common permutations, including Linux. 252</para> 253</subsubsection> 254</subsection> 255 256<subsection title="The Role of Technology"> 257<para> 258 Unix operating systems generally provide a rather primitive model 259 for distributing privilege to system users. The power to control all 260 system processes and resources is granted to the single all-powerful 261 user, root. Other users may be granted varying levels of access, 262 depending mainly on which Unix group they belong to and on how group 263 access permissions are set on various objects in the system. However 264 root (or any user with UID 0) is the only user that can arbitrarily 265 change access permissions. As a result, when non-root users 266 encounter a restriction in access permissions, they must call upon 267 the power of the root user to rearrange things so that they may 268 continue their work. 269</para> 270<subsubsection title="C2"> 271<para> 272 There are exceptions to this monolithic permissions model among 273 various proprietary and free Unix implementations. Many OS vendors, 274 including most Unix vendors, have applied for and received DOD 275 Orange Book C2 certification for one or more of their products. (For 276 an exhaustive list see <link 277 href="http://www.radium.ncsc.mil/tpep/library/fers/tcsec_fers.html" 278 text="http://www.radium.ncsc.mil/tpep/library/fers/tcsec_fers.html"/>.) 279 However these vendors generally do not ship their systems with C2 280 security enabled. Even Microsoft, whose Windows NT code base 281 implements many of the facilities that C2 requires, such as Access 282 Control Lists, doesn't do that. Since Microsoft, at least, has had 283 to submit a version of NT with network access disabled in order to 284 get certification, that's not entirely surprising. And though I'm not 285 an expert on the topic, I suspect that the reason even those vendors 286 who may be able to run C2 while on a network don't ship with it 287 enabled is because C2 access controls are fairly burdensome to users 288 and administrators alike. Regardless of the real reason, the fact 289 remains that the Unix systems found in most commercial environments, 290 from vendors like Sun, HP, IBM, as well as Linux and xBSDs, come 291 configured by default with an antiquated permissions model. 292</para> 293</subsubsection> 294<subsubsection title="Working Within the Model"> 295<para> 296 Even given the monolithic Unix permissions model, it is possible to 297 give users most of what they want without unleashing the full power 298 of root. Issues that concern shared access to files can be dealt 299 with by judiciously adjusting group membership and permissions. If 300 a user needs to open low-numbered TCP/IP ports, for example, it's 301 possible to setuid root just a particular binary, though that 302 carries with it all sorts of other security implications. There 303 are many strategies that help users to "work within the 304 system." Each of these has in common one fatal flaw, 305 however. If the model needs to be adjusted because of an unforeseen 306 condition, root (a.k.a. the sysadmin) needs to get involved to make 307 the adjustment. In a rapidly changing environment like a software 308 startup, this has several impacts on the user. First, it slows her 309 down. A overworked and harassed sysadmin has to be located by an at 310 least equally overworked and harassed engineer to make the 311 change. According to Murphy, this will always happen at 3:00 AM 312 on the morning before a critical demo. I <em>really, really hate</em> to have my 313 cell phone ring at that hour! Even if that apocalyptic scenario doesn't 314 get played out, resentment may be fostered on both poles of the 315 struggle. The user may start to see the sysadmin as a power-mad 316 tightwad, jealously guarding root access for his own nefarious 317 purpose. The sysadmin may feel put out by the fact that the user 318 isn't willing to learn enough about Unix to get around her 319 problem. He may also be blind to any benefit that might accrue to 320 the user and the enterprise from allowing the access. There's a bit 321 of irony here. Systems administrators often complain (with 322 justification) that what they do is never visible until something 323 breaks. This is a consequence of the natural outcome of great 324 sysadmin: quietly working systems. In a similar way, the sysadmin is 325 unlikely to see any benefit from giving a user root, because those 326 benefits short-circuit trouble calls to the sysadmin! 327</para> 328<para> 329 Before moving on, I'd like to note that neither of the 330 characterizations presented above is fair, and they rarely play out 331 in such an extreme form in the real world. But their flavor is 332 correct, at least to judge by the places I've been. 333</para> 334</subsubsection> 335<subsubsection title="Tools That Try to Help"> 336<para> 337 Because working within the system is so troublesome, the 338 ever-inventive Unix community has produced many tools 339 340<footnote ref="5"> 341A comprehensive list of such tools is maintained 342<link href="http://www.courtesan.com/sudo/other.html" text="here"/> 343</footnote> 344 345 that try to add finer-grained control to the monolithic Unix 346 permissions model. One of the most popular is <link 347 href="http://www.courtesan.com/sudo" text="sudo"/>. This tool is 348 familiar to many sysadmins. It allows users to invoke specific 349 commands with root privilege. It uses the user's own password to 350 authenticate access, thus protecting the root password. It also logs 351 each command invocation with the name of the user, thus providing an 352 audit trail of root access. Typically, tools like sudo are deployed 353 to meet a specific user need for root access, such as to enable them 354 to change permissions on directories. However, security issues can 355 arise from the use of such commands that allow the user to evade 356 the restrictions sudo provides, either innocently or deliberately. 357 For example, the chmod command that can make an inaccessible directory 358 available could also allow the user to set the setuid bit on a binary. 359 If that binary is /bin/bash, or any other shell, and it's owned by root, 360 as is usual, running the shell after a 'chmod 4755' would result in an 361 unaudited root shell. One way around this would be to wrap a script 362 around the chmod command that disallowed setuid mapping. But then 363 you have to worry about the security of shell scripts running as 364 root. In fact, in a relatively open environment like a software 365 startup, there is no sure way to protect yourself from malicious 366 misuse of privilege in all cases, even without sudo. You end up having 367 to fall back on trust, treating abuse of that trust as a personnel problem. 368</para> 369<para> 370 The problem gets even worse as more and more commands are added to 371 the suite of those offered to sudoers. Each new command brings its 372 own particular set of security holes. The problem, once again, is 373 that Unix assumes a monolithic permissions model that tools like 374 sudo can only work around, not cure. And if you allow shells to be 375 run with sudo, you might as well just give the user the root 376 password. This is because sudo only logs the initial invocation 377 of any command. If that command is a shell, such as /bin/bash, 378 subsequent commands issued from that shell are not logged. 379</para> 380<para> 381 The weakness in the Unix permissions model shows up again as 382 problems with programs like sudo that have nothing to do with the 383 quality of the code, and everything to do with the fact that hacks 384 like sudo are necessary in the first place. 385</para> 386 387<para> 388 For example, sudo has difficulty with IO redirection: 389 390<pre> 391 392 hbo@egbok > ls -l /tmp/foo 393 -r--r--r-- 1 root other 1464 Mar 25 13:10 /tmp/foo 394 hbo@egbok > sudo ls >>/tmp/foo 395 bash: /tmp/foo: Permission denied 396 hbo@egbok > sudo ls | sudo cat >>/tmp/foo 397 bash: /tmp/foo: Permission denied 398</pre> 399 This problem occurs because I/O redirection is implemented by the 400 shell before the command (sudo) is executed. The monolithic Unix 401 permissions model leads the shell to assume that the identity that 402 does the I/O redirection is the same as the one that will result 403 from the execution of the command. This is false in the case of 404 sudo, which violates that permissions model. The following trick 405 gets around the problem: 406<pre> 407 hbo@egbok > sudo ls | sudo tee -a /tmp/foo >/dev/null 408</pre> 409 But it's not very intuitive. This also works: 410<pre> 411 hbo@egbok > sudo sh -c "ls >>/tmp/foo" 412</pre> 413But as previously noted, if you allow shell access with sudo, you might as 414well give out the root password. 415</para> 416<para> 417Globbing is broken too: 418<pre> 419 hbo@egbok > mkdir fff 420 hbo@egbok > chmod 700 fff 421 hbo@egbok > touch fff/foo 422 hbo@egbok > sudo chown root fff 423 Password: 424 hbo@egbok > cd fff 425 bash: cd: fff: Permission denied 426 hbo@egbok > sudo cd fff 427 sudo: cd: command not found <strong># cd is a bash builtin!</strong> 428 hbo@egbok > sudo rm fff/* 429 rm: cannot remove `fff/*': No such file or directory 430</pre> 431 The 'globbing' expansion requested by the use of the asterisk fails 432 because, once again, the shell tries to do it before executing the 433 sudo command. We also see in this example the problem of trying 434 to 'cd' into a protected directory. Since 'cd' is a bash builtin, 435 sudo doesn't know what to do with it and you are out of luck. 436</para> 437<para> 438 Of course, you could put code to solve either problem in a 439 script. But if you let your users run, for instance, Perl with 440 sudo, what's to stop them from writing something like this? 441<pre> 442 #!/usr/bin/perl 443 exec "/bin/bash"; 444</pre> 445 Once again, there goes your audit trail! In fact, if your users have 446 successfully agitated for sudo access to more than a handful of 447 commands you will almost certainly face an impossible number of 448 holes in your security policy. 449</para> 450 451</subsubsection> 452</subsection> 453<subsection title="PORCMOLSULB, Again"> 454<para> 455 So far, we've seen two groups of professionals, apparently similar 456 on the surface, engaged in a struggle for control of root access on 457 personal workstations. Each group is trying to carry out the goals 458 that their respective business imperatives demand. The software 459 engineer wants root so that she can get around restrictions in the 460 Unix system in order to get her work done. The system administrator 461 is trying to ensure that the user's system stays functioning. What 462 are some possible outcomes of this struggle? Complete victory by 463 either side is unlikely. To borrow a concept from chemistry, a more 464 plausible outcome is that some sort of "dynamic 465 equilibrium" will be reached as managers in support and 466 engineering struggle to balance competing business interests. When 467 the struggle concerns root access on servers, the business 468 imperatives lean more toward the sysadmin's view of things, because 469 the technical problems of sharing root on a server are less 470 tractable, and the consequences of a root compromise more 471 obvious. On engineer's personal workstations, however, the business 472 case for allowing unfettered root is more compelling, because the 473 workstation is a primary tool enabling the engineer's 474 productivity. If the balance of power shifts toward the sysadmin, we 475 start to see the phenomenon of PORCMOLSULB showing up. This occurs 476 when support departments can't keep up with the demands of their 477 user base for development "play-pens," or when they put 478 restrictions on those play-pens beyond what the users are willing to 479 accept. It turns out that engineers are increasingly able to 480 convince their managers that a completely uncontrolled Linux box 481 would be a boost to their efforts in the rush to meet insane 482 deadline pressure. 483 484<footnote ref="6"> 485 Managers generally feel bad about asking their people to work 12+ 486 hour days to meet unreasonable deadlines, no matter how they put a 487 brave face on the matter. Giving their engineers the tools they need 488 is therefore not only good sense from an organizational standpoint, 489 it lets the managers hand out a perk or two. 490</footnote> 491 492 The sysadmin crew is probably feeling the pressure too, so they are 493 in worse shape than normal to resist this trend. Indeed, they may 494 not even become aware the box exists until it shows up in the 495 critical path for some important milestone. If support departments 496 know the box is being deployed, but lack the power to prevent it, 497 they frequently disavow support, citing the extreme difficulty of 498 maintaining a box with a random configuration. But in a software 499 startup, they can still find themselves stuck with fixing the box 500 under killer time pressure, with the business on the line and with 501 no advance idea of how it was configured by its amateur 502 sysadmin. 503</para> 504</subsection> 505</section> 506<section title="What is to be Done?"> 507<para> 508 That nightmare scenario didn't actually happen to me in the case I 509 opened this paper with. But we <em>were</em> facing a killer 510 deadline, and the mere possibility of it happening made me 511 nervous. I had faced similar situations before, so I knew that 512 arguing for the "right" way of doing things wouldn't lead 513 me anywhere useful. I'd also recently had my epiphany regarding the 514 surface similarity and deep difference between sysadmins and 515 software engineers. Here's how this particular comedy <em>did</em> 516 play out. 517</para> 518<subsection title="Do you sudo?"> 519<para> 520 About 10 days after the senior developer got his engineer working 521 again by doing a 'chown -R root /usr', she showed up in my cube 522 asking for the Linux disks. I was mildly surprised that it had taken 523 that long for a side effect of that solution to convince her that 524 she needed to reinstall. But I tried not to act smug, and handed her 525 the disks without asking why she wanted a reload of Linux. But I did 526 ask her if she wouldn't rather that I do the install. I'd set her up 527 so that her home directory on her workstation would automount 528 underneath her when she went out to the network. I'd also arrange 529 for it to be backed up regularly, and support it so that she could 530 come to me if she had problems. She allowed as how that might be a 531 good thing. So I delivered the punchline: "All you have to do 532 is give up root and use sudo. It takes a little getting used to, but 533 I'll help out." Well, she readily agreed to that too, and I was 534 in a self-congratulatory mood when she came back in ten minutes 535 saying her boss had nixed the idea. He said she had to have root 536 instead of sudo. 537</para> 538<para> 539 I actually took a short time out before going over to his cube. My 540 question to him was rather sharp, but nothing like it could have 541 been. "Do you really think backups, the automounter and 542 support are worth having the root password?" "Yes," 543 he said. "<em>Why</em> fer ----ssake??" I politely 544 asked. "Because you won't let us run shells with sudo!" I 545 proceeded to tell the story of 27 eight-and-a-half by ten colored 546 glossy audit trails.. he said "Stop right there! What good 547 would an audit trail do you if someone did 'chown -R <user> 548 /usr'?" 549</para> 550<para> 551 Well, he had a point. But I had an answer: "because the audit 552 trail would tell me right away that I had to reload Linux, rather 553 than some less drastic solution. And besides most problems aren't 554 caused by thoroughly bone-headed moves like that one!" He 555 laughed and said "OK. Give her sudo." 556</para> 557<para> 558 I felt pretty good after that. It could have turned out differently, 559 but it didn't. Despite the sharpness of the exchange, I felt like 560 I'd made a critical connection with this guy. 561 562<footnote ref="7"> 563 I probably neglected to mention that I'm new on the job. 564</footnote> 565 566 In addition, I had a toehold in his group with a supported Linux box 567 that would <em>not</em> be randomly configured, and would be less, 568 not more screwed up. And his new engineer would be using sudo! I 569 would work hard to make sure that she had as good an experience with 570 it as I could manage. 571</para> 572<para> 573 In fact, over the next couple of days they came to me several times 574 with things they couldn't do with sudo, and could I please just run 575 the command with root? Each time it had nothing to do with sudo, and 576 each time I cheerfully fixed it for them, or pointed them in the 577 right direction. Soon, I had a couple of converts. 578</para> 579</subsection> 580<subsection title="Beyond Sudo"> 581<para> 582 Now it turns out that the senior developer, and all his colleagues, 583 were resistant to using sudo because we restricted shells. This is 584 an area where a sysadmin can argue unto blue-facedness about the 585 lack of a need for a shell when you have more-or-less unrestricted 586 sudo access. Indeed, since we had such unrestricted access, escaping 587 from sudo and its audit trail was a trivial exercise. Given those 588 facts, I decided to just accept that despite the technical 589 arguments, sudo alone was not a workable solution for these senior 590 engineers on their workstations. In half a day, I whipped up a pair 591 of Perl scripts that used script(1) and a fifo to provide an audited 592 root shell using sudo. 593 594<footnote ref="8"> 595 The result of considerably more than half a day's effort is available 596 <link href="http://www.egbok.com/sudoscript" text="here"/>. 597</footnote> 598 599 This gave them practically nothing they didn't have already with our 600 open sudo policy, and preserved our audit trail. All the senior 601 engineers accepted a support regime which included these scripts. 602</para> 603</subsection> 604</section> 605<section title="Caveats and Conclusions" break="page"> 606<subsection title="Danger [W|J]ill Sysadmin!"> 607<para> 608 There are <em>big</em> problems with this solution. Having root on a 609 workstation that mounts NFS shares is tantamount to giving the user 610 root on the NFS server! Most NFS servers can and should be 611 configured so that any access to an exported file system by UID 0 is 612 mapped to a user with no privilege whatsoever. But that's not the 613 whole answer. With root, a user can assume any UID in the passwd 614 map. This means that on the NFS server, other users' files and 615 system files not owned by root, are at the mercy of root <em>on the 616 NFS client!</em> The approach I've described works best when the 617 workstations are NFS servers, not clients. There is still an issue 618 with other systems mounting shares from the workstation. If the NFS 619 client implementation doesn't enable you to disallow setuid 620 binaries, a root user on the workstation could place, for example, a 621 setuid root bash binary on the exported file system, then execute 622 that binary on the client and get root privs. 623</para> 624</subsection> 625<subsection title="Philosophy 1001"> 626<para> 627 This is not an exhaustive list of the security problems such a setup 628 could raise. But remember that in my environment, the users had 629 successfully agitated for more-or-less unrestricted sudo 630 access. Since this was so, giving them an audited root shell didn't 631 raise any more security concerns than were already present. So I'm not 632 claiming that this would work everywhere. But, in my small shop, 633 I can look each of my users in the eye every day if I choose. There 634 is not a single unteachable idiot in the bunch. I also don't hand 635 out my scripts to everyone. In short, I rely on the good faith of my 636 users. I give them the tools they say they need, and I try to give 637 them the benefit of the doubt on the question, despite my technical 638 knowledge to the contrary. If they shoot themselves in an extremity 639 with their privilege, I triage and fix the damage, with the benefit 640 of a recent audit trail. 641<pre> 642<tirade mode="self righteous" color="purple"> 643 644 I trust my users in this regard because of one argument in favor of 645 Democracy: If you give people more choices, some will make bad ones. 646 Many more will make good ones, yielding a net benefit. 647 648</tirade> 649</pre> 650 And this principle may well be applicable beyond my environment. 651 How it plays out in yours is up to you, and your users. 652</para> 653</subsection> 654<subsection title="Finally, Documentation"> 655<para> 656 Once I've fixed any problems caused by inexperienced root users 657 blasting off their toes, I try to leverage the occasion to get them 658 to read my documentation. Ah, yes. Documentation. Nobody likes to 659 write it, and nobody likes to read it. I write lots of documentation 660 and perversely perhaps, I enjoy doing it. What I find hard to take 661 is the indifference most of my users show toward what I write. My 662 epiphany regarding the differences between sysadmins and software 663 engineers has provided me with an explanation for that conundrum as 664 well. <em>What's relevant to me and to the systems under my care is 665 not directly relevant to my user's concerns!</em> If I were to write 666 the best-ever UML manual, <em>then</em> they might notice. But when 667 a pretty bright engineer has made some embarrassing error that has 668 clearly resulted in a hit on her productivity, or worse, that of her 669 colleagues, then the docs I write may seem more relevant. You have 670 to be tactful and swift in exploiting these opportunities for 671 education, however. Tactful because these folks are proud, and their 672 pride has just been wounded. Swift, because they'll have their heads 673 completely stuffed full of Java before long, with little room for 674 anything else. 675</para> 676</subsection> 677</section> 678 679</content> 680</egbokdoc> 681