1 2:mod:`lockfile` --- Platform-independent file locking 3===================================================== 4 5.. module:: lockfile 6 :synopsis: Platform-independent file locking 7.. moduleauthor:: Skip Montanaro <skip@pobox.com> 8.. sectionauthor:: Skip Montanaro <skip@pobox.com> 9 10.. warning:: 11 12 This package is **deprecated**. It is highly preferred that instead of 13 using this code base that instead `fasteners`_ or `oslo.concurrency`_ is 14 used instead. For any questions or comments or further help needed 15 please email `openstack-dev`_ and prefix your email subject 16 with ``[oslo][pylockfile]`` (for a faster response). 17 18.. note:: 19 20 This package is pre-release software. Between versions 0.8 and 0.9 it 21 was changed from a module to a package. It is quite possible that the 22 API and implementation will change again in important ways as people test 23 it and provide feedback and bug fixes. In particular, if the mkdir-based 24 locking scheme is sufficient for both Windows and Unix platforms, the 25 link-based scheme may be deleted so that only a single locking scheme is 26 used, providing cross-platform lockfile cooperation. 27 28.. note:: 29 30 The implementation uses the `with` statement, both in the tests and in the 31 main code, so will only work out-of-the-box with Python 2.5 or later. 32 However, the use of the `with` statement is minimal, so if you apply the 33 patch in the included 2.4.diff file you can use it with Python 2.4. It's 34 possible that it will work in Python 2.3 with that patch applied as well, 35 though the doctest code relies on APIs new in 2.4, so will have to be 36 rewritten somewhat to allow testing on 2.3. As they say, patches welcome. 37 ``;-)`` 38 39The :mod:`lockfile` package exports a :class:`LockFile` class which provides 40a simple API for locking files. Unlike the Windows :func:`msvcrt.locking` 41function, the Unix :func:`fcntl.flock`, :func:`fcntl.lockf` and the 42deprecated :mod:`posixfile` module, the API is identical across both Unix 43(including Linux and Mac) and Windows platforms. The lock mechanism relies 44on the atomic nature of the :func:`link` (on Unix) and :func:`mkdir` (On 45Windows) system calls. It also contains several lock-method-specific 46modules: :mod:`lockfile.linklockfile`, :mod:`lockfile.mkdirlockfile`, and 47:mod:`lockfile.sqlitelockfile`, each one exporting a single class. For 48backwards compatibility with versions before 0.9 the :class:`LinkFileLock`, 49:class:`MkdirFileLock` and :class:`SQLiteFileLock` objects are exposed as 50attributes of the top-level lockfile package, though this use was deprecated 51starting with version 0.9 and will be removed in version 1.0. 52 53.. note:: 54 55 The current implementation uses :func:`os.link` on Unix, but since that 56 function is unavailable on Windows it uses :func:`os.mkdir` there. At 57 this point it's not clear that using the :func:`os.mkdir` method would be 58 insufficient on Unix systems. If it proves to be adequate on Unix then 59 the implementation could be simplified and truly cross-platform locking 60 would be possible. 61 62.. note:: 63 64 The current implementation doesn't provide for shared vs. exclusive 65 locks. It should be possible for multiple reader processes to hold the 66 lock at the same time. 67 68The module defines the following exceptions: 69 70.. exception:: Error 71 72 This is the base class for all exceptions raised by the :class:`LockFile` 73 class. 74 75.. exception:: LockError 76 77 This is the base class for all exceptions raised when attempting to lock 78 a file. 79 80.. exception:: UnlockError 81 82 This is the base class for all exceptions raised when attempting to 83 unlock a file. 84 85.. exception:: LockTimeout 86 87 This exception is raised if the :func:`LockFile.acquire` method is 88 called with a timeout which expires before an existing lock is released. 89 90.. exception:: AlreadyLocked 91 92 This exception is raised if the :func:`LockFile.acquire` detects a 93 file is already locked when in non-blocking mode. 94 95.. exception:: LockFailed 96 97 This exception is raised if the :func:`LockFile.acquire` detects some 98 other condition (such as a non-writable directory) which prevents it from 99 creating its lock file. 100 101.. exception:: NotLocked 102 103 This exception is raised if the file is not locked when 104 :func:`LockFile.release` is called. 105 106.. exception:: NotMyLock 107 108 This exception is raised if the file is locked by another thread or 109 process when :func:`LockFile.release` is called. 110 111The following classes are provided: 112 113.. class:: linklockfile.LinkLockFile(path, threaded=True) 114 115 This class uses the :func:`link(2)` system call as the basic lock 116 mechanism. *path* is an object in the file system to be locked. It need 117 not exist, but its directory must exist and be writable at the time the 118 :func:`acquire` and :func:`release` methods are called. *threaded* is 119 optional, but when set to :const:`True` locks will be distinguished 120 between threads in the same process. 121 122.. class:: symlinklockfile.SymlinkLockFile(path, threaded=True) 123 124 This class uses the :func:`symlink(2)` system call as the basic lock 125 mechanism. The parameters have the same meaning and constraints as for 126 the :class:`LinkLockFile` class. 127 128.. class:: mkdirlockfile.MkdirLockFile(path, threaded=True) 129 130 This class uses the :func:`mkdir(2)` system call as the basic lock 131 mechanism. The parameters have the same meaning and constraints as for 132 the :class:`LinkLockFile` class. 133 134.. class:: sqlitelockfile.SQLiteLockFile(path, threaded=True) 135 136 This class uses the :mod:`sqlite3` module to implement the lock 137 mechanism. The parameters have the same meaning as for the 138 :class:`LinkLockFile` class. 139 140.. class:: LockBase(path, threaded=True) 141 142 This is the base class for all concrete implementations and is available 143 at the lockfile package level so programmers can implement other locking 144 schemes. 145 146.. function:: locked(path, timeout=None) 147 148 This function provides a decorator which insures the decorated function 149 is always called with the lock held. 150 151By default, the :const:`LockFile` object refers to the 152:class:`mkdirlockfile.MkdirLockFile` class on Windows. On all other 153platforms it refers to the :class:`linklockfile.LinkLockFile` class. 154 155When locking a file the :class:`linklockfile.LinkLockFile` class creates a 156uniquely named hard link to an empty lock file. That hard link contains the 157hostname, process id, and if locks between threads are distinguished, the 158thread identifier. For example, if you want to lock access to a file named 159"README", the lock file is named "README.lock". With per-thread locks 160enabled the hard link is named HOSTNAME-THREADID-PID. With only per-process 161locks enabled the hard link is named HOSTNAME--PID. 162 163When using the :class:`mkdirlockfile.MkdirLockFile` class the lock file is a 164directory. Referring to the example above, README.lock will be a directory 165and HOSTNAME-THREADID-PID will be an empty file within that directory. 166 167.. seealso:: 168 169 Module :mod:`msvcrt` 170 Provides the :func:`locking` function, the standard Windows way of 171 locking (parts of) a file. 172 173 Module :mod:`posixfile` 174 The deprecated (since Python 1.5) way of locking files on Posix systems. 175 176 Module :mod:`fcntl` 177 Provides the current best way to lock files on Unix systems 178 (:func:`lockf` and :func:`flock`). 179 180LockFile Objects 181---------------- 182 183:class:`LockFile` objects support the `context manager` protocol used by the 184statement:`with` statement. The timeout option is not supported when used in 185this fashion. While support for timeouts could be implemented, there is no 186support for handling the eventual :exc:`Timeout` exceptions raised by the 187:func:`__enter__` method, so you would have to protect the `with` statement with 188a `try` statement. The resulting construct would not be any simpler than just 189using a `try` statement in the first place. 190 191:class:`LockFile` has the following user-visible methods: 192 193.. method:: LockFile.acquire(timeout=None) 194 195 Lock the file associated with the :class:`LockFile` object. If the 196 *timeout* is omitted or :const:`None` the caller will block until the 197 file is unlocked by the object currently holding the lock. If the 198 *timeout* is zero or a negative number the :exc:`AlreadyLocked` exception 199 will be raised if the file is currently locked by another process or 200 thread. If the *timeout* is positive, the caller will block for that 201 many seconds waiting for the lock to be released. If the lock is not 202 released within that period the :exc:`LockTimeout` exception will be 203 raised. 204 205.. method:: LockFile.release() 206 207 Unlock the file associated with the :class:`LockFile` object. If the 208 file is not currently locked, the :exc:`NotLocked` exception is raised. 209 If the file is locked by another thread or process the :exc:`NotMyLock` 210 exception is raised. 211 212.. method:: is_locked() 213 214 Return the status of the lock on the current file. If any process or 215 thread (including the current one) is locking the file, :const:`True` is 216 returned, otherwise :const:`False` is returned. 217 218.. method:: break_lock() 219 220 If the file is currently locked, break it. 221 222.. method:: i_am_locking() 223 224 Returns true if the caller holds the lock. 225 226Examples 227-------- 228 229This example is the "hello world" for the :mod:`lockfile` package:: 230 231 from lockfile import LockFile 232 lock = LockFile("/some/file/or/other") 233 with lock: 234 print lock.path, 'is locked.' 235 236To use this with Python 2.4, you can execute:: 237 238 from lockfile import LockFile 239 lock = LockFile("/some/file/or/other") 240 lock.acquire() 241 print lock.path, 'is locked.' 242 lock.release() 243 244If you don't want to wait forever, you might try:: 245 246 from lockfile import LockFile 247 lock = LockFile("/some/file/or/other") 248 while not lock.i_am_locking(): 249 try: 250 lock.acquire(timeout=60) # wait up to 60 seconds 251 except LockTimeout: 252 lock.break_lock() 253 lock.acquire() 254 print "I locked", lock.path 255 lock.release() 256 257You can also insure that a lock is always held when appropriately decorated 258functions are called:: 259 260 from lockfile import locked 261 @locked("/tmp/mylock") 262 def func(a, b): 263 return a + b 264 265Other Libraries 266--------------- 267 268The idea of implementing advisory locking with a standard API is not new 269with :mod:`lockfile`. There are a number of other libraries available: 270 271* locknix - http://pypi.python.org/pypi/locknix - Unix only 272* mx.MiscLockFile - from Marc André Lemburg, part of the mx.Base 273 distribution - cross-platform. 274* Twisted - http://twistedmatrix.com/trac/browser/trunk/twisted/python/lockfile.py 275* zc.lockfile - http://pypi.python.org/pypi/zc.lockfile 276 277 278Contacting the Author 279--------------------- 280 281If you encounter any problems with ``lockfile``, would like help or want to 282submit a patch, check http://launchpad.net/pylockfile 283 284 285.. _fasteners: http://fasteners.readthedocs.org/ 286.. _openstack-dev: mailto:openstack-dev@lists.openstack.org 287.. _oslo.concurrency: http://docs.openstack.org/developer/oslo.concurrency/ 288