1Documentation for repoze.tm2 (``repoze.tm`` fork) 2================================================= 3 4Overview 5-------- 6 7:mod:`repoze.tm2` is WSGI middleware which uses the ``ZODB`` package's 8transaction manager to wrap a call to its pipeline children inside a 9transaction. 10 11.. note:: :mod:`repoze.tm2` is equivalent to the :mod:`repoze.tm` 12 package (it was forked from :mod:`repoze.tm`), except it has a 13 dependency only on the ``transaction`` package rather than a 14 dependency on the entire ``ZODB3`` package (``ZODB3`` 3.8 ships 15 with the ``transaction`` package right now). It is an error to 16 install both repoze.tm and repoze.tm2 into the same environment, as 17 they provide the same entry points and import points. 18 19Behavior 20-------- 21 22When this middleware is present in the WSGI pipeline, a new 23transaction will be started once a WSGI request makes it to the 24middleware. If any downstream application raises an 25exception, the transaction will be aborted, otherwise the transaction 26will be committed. Any "data managers" participating in the 27transaction will be aborted or committed respectively. A ZODB 28"connection" is an example of a data manager. 29 30Since this is a tiny wrapper around the ZODB transaction module, and 31the ZODB transaction module is "thread-safe" (in the sense that its 32default policy is to create a new transaction for each thread), it 33should be fine to use in either multiprocess or multithread 34environments. 35 36Purpose and Usage 37----------------- 38 39The ZODB transaction manager is a completely generic transaction 40manager. It can be used independently of the actual "object database" 41part of ZODB. One of the purposes of creating :mod:`repoze.tm` was to 42allow for systems other than Zope to make use of two-phase commit 43transactions in a WSGI context. 44 45Let's pretend we have an existing system that places data into a 46relational database when someone submits a form. The system has been 47running for a while, and our code handles the database commit and 48rollback for us explicitly; if the form processing succeeds, our code 49commits the database transaction. If it fails, our code rolls back 50the database transaction. Everything works fine. 51 52Now our customer asks us if we can also place data into another 53separate relational database when the form is submitted as well as 54continuing to place data in the original database. We need to put 55data in both databases, and if we want to ensure that no records exist 56in one that don't exist in the other as a result of a form submission, 57we're going to need to do a pretty complicated commit and rollback 58dance in each place in our code which needs to write to both data 59stores. We can't just blindly commit one, then commit the other, 60because the second commit may fail and we'll be left with "orphan" 61data in the first, and we'll either need to clean it up manually or 62leave it there to trip over later. 63 64A transaction manager helps us ensure that no data is committed to 65either database unless both participating data stores can commit. 66Once the transaction manager determines that both data stores are 67willing to commit, it will commit them both in very quick succession, 68so that there is only a minimal chance that the second data store will 69fail to commit. If it does, the system will raise an error that makes 70it impossible to begin another transaction until the system restarts, 71so the damage is minimized. In practice, this error almost never 72occurs unless the code that interfaces the database to the transaction 73manager has a bug. 74 75Adding :mod:`repoze.tm2` To Your WSGI Pipeline 76---------------------------------------------- 77 78Via ``PasteDeploy`` .INI configuration:: 79 80 [pipeline:main] 81 pipeline = 82 egg:repoze.tm2#tm 83 myapp 84 85Via Python: 86 87.. code-block:: python 88 89 from otherplace import mywsgiapp 90 91 from repoze.tm import TM 92 new_wsgiapp = TM(mywsgiapp) 93 94Using A Commit Veto 95------------------- 96 97If you'd like to veto commits based on the status code returned by the 98downstream application, use a commit veto callback. 99 100First, define the callback somewhere in your application: 101 102.. code-block:: python 103 104 def commit_veto(environ, status, headers): 105 for header_name, header_value in headers: 106 if header_name.lower() == 'x-tm': 107 if header_value.lower() == 'commit': 108 return False 109 return True 110 for bad in ('4', '5'): 111 if status.startswith(bad): 112 return True 113 return False 114 115Then configure it into your middleware. 116 117Via Python: 118 119.. code-block:: python 120 121 from otherplace import mywsgiapp 122 from my.package import commit_veto 123 124 from repoze.tm import TM 125 new_wsgiapp = TM(mywsgiapp, commit_veto=commit_veto) 126 127Via PasteDeploy: 128 129.. code-block:: ini 130 131 [filter:tm] 132 commit_veto = my.package:commit_veto 133 134In the PasteDeploy example, the path is a Python dotted name, where the dots 135separate module and package names, and the colon separates a module from its 136contents. In the above example, the code would be implemented as a 137"commit_veto" function which lives in the "package" submodule of the "my" 138package. 139 140A variant of the commit veto implementation shown above as an example is 141actually present in the ``repoze.tm2`` package as 142``repoze.tm.default_commit_veto``. It's fairly general, so you needn't 143implement one yourself. Instead just use it. 144 145Via Python: 146 147.. code-block:: python 148 149 from otherplace import mywsgiapp 150 from repoze.tm import default_commit_veto 151 152 from repoze.tm import TM 153 new_wsgiapp = TM(mywsgiapp, commit_veto=default_commit_veto) 154 155Via PasteDeploy: 156 157.. code-block:: ini 158 159 [filter:tm] 160 commit_veto = repoze.tm:default_commit_veto 161 162API documentation for ``default_commit_veto`` exists at 163:func:`repoze.tm.default_commit_veto`. 164 165Mocking Up A Data Manager 166------------------------- 167 168The piece of code you need to write in order to participate in ZODB 169transactions is called a 'data manager'. It is typically a class. 170Here's the interface that you need to implement in the code for a data 171manager: 172 173.. code-block:: python 174 175 class IDataManager(zope.interface.Interface): 176 """Objects that manage transactional storage. 177 178 These objects may manage data for other objects, or they 179 may manage non-object storages, such as relational 180 databases. For example, a ZODB.Connection. 181 182 Note that when some data is modified, that data's data 183 manager should join a transaction so that data can be 184 committed when the user commits the transaction. """ 185 186 transaction_manager = zope.interface.Attribute( 187 """The transaction manager (TM) used by this data 188 manager. 189 190 This is a public attribute, intended for read-only 191 use. The value is an instance of ITransactionManager, 192 typically set by the data manager's constructor. """ 193 ) 194 195 def abort(transaction): 196 """Abort a transaction and forget all changes. 197 198 Abort must be called outside of a two-phase commit. 199 200 Abort is called by the transaction manager to abort transactions 201 that are not yet in a two-phase commit. 202 """ 203 204 # Two-phase commit protocol. These methods are called by 205 # the ITransaction object associated with the transaction 206 # being committed. The sequence of calls normally follows 207 # this regular expression: tpc_begin commit tpc_vote 208 # (tpc_finish | tpc_abort) 209 210 def tpc_begin(transaction): 211 212 """Begin commit of a transaction, starting the 213 two-phase commit. 214 215 transaction is the ITransaction instance associated with the 216 transaction being committed. 217 """ 218 219 def commit(transaction): 220 221 """Commit modifications to registered objects. 222 223 Save changes to be made persistent if the transaction 224 commits (if tpc_finish is called later). If tpc_abort 225 is called later, changes must not persist. 226 227 This includes conflict detection and handling. If no 228 conflicts or errors occur, the data manager should be 229 prepared to make the changes persist when tpc_finish 230 is called. """ 231 232 def tpc_vote(transaction): 233 """Verify that a data manager can commit the transaction. 234 235 This is the last chance for a data manager to vote 'no'. A 236 data manager votes 'no' by raising an exception. 237 238 transaction is the ITransaction instance associated with the 239 transaction being committed. 240 """ 241 242 def tpc_finish(transaction): 243 244 """Indicate confirmation that the transaction is done. 245 246 Make all changes to objects modified by this 247 transaction persist. 248 249 transaction is the ITransaction instance associated 250 with the transaction being committed. 251 252 This should never fail. If this raises an exception, 253 the database is not expected to maintain consistency; 254 it's a serious error. """ 255 256 def tpc_abort(transaction): 257 258 """Abort a transaction. 259 260 This is called by a transaction manager to end a 261 two-phase commit on the data manager. Abandon all 262 changes to objects modified by this transaction. 263 264 transaction is the ITransaction instance associated 265 with the transaction being committed. 266 267 This should never fail. 268 """ 269 270 def sortKey(): 271 272 """Return a key to use for ordering registered 273 DataManagers. 274 275 ZODB uses a global sort order to prevent deadlock when 276 it commits transactions involving multiple resource 277 managers. The resource manager must define a 278 sortKey() method that provides a global ordering for 279 resource managers. """ 280 # Alternate version: 281 #"""Return a consistent sort key for this connection. 282 # #This allows ordering multiple connections that use 283 the same storage in #a consistent manner. This is 284 unique for the lifetime of a connection, #which is 285 good enough to avoid ZEO deadlocks. #""" 286 287Let's implement a mock data manager. Our mock data manager will write 288data to a file if the transaction commits. It will not write data to 289a file if the transaction aborts: 290 291.. code-block:: python 292 293 class MockDataManager: 294 295 transaction_manager = None 296 297 def __init__(self, data, path): 298 self.data = data 299 self.path = path 300 301 def abort(self, transaction): 302 pass 303 304 def tpc_begin(self, transaction): 305 pass 306 307 def commit(self, transaction): 308 import tempfile 309 self.tempfn = tempfile.mktemp() 310 temp = open(self.tempfn, 'wb') 311 temp.write(self.data) 312 temp.flush() 313 temp.close() 314 315 def tpc_vote(self, transaction): 316 import os 317 if not os.path.exists(self.tempfn): 318 raise ValueError('%s doesnt exist' % self.tempfn) 319 if os.path.exists(self.path): 320 raise ValueError('file already exists') 321 322 def tpc_finish(self, transaction): 323 import os 324 os.rename(self.tempfn, self.path) 325 326 def tpc_abort(self, transaction): 327 import os 328 try: 329 os.remove(self.tempfn) 330 except OSError: 331 pass 332 333We can create a datamanager and join it into the currently running 334transaction: 335 336.. code-block:: python 337 338 dm = MockDataManager('heres the data', '/tmp/file') 339 import transaction 340 t = transaction.get() 341 t.join(dm) 342 343When the transaction commits, a file will be placed in '/tmp/file' 344containing 'heres the data'. If the transaction aborts, no file will 345be created. 346 347If more than one data manager is joined to the transaction, all of 348them must be willing to commit or the entire transaction is aborted 349and none of them commit. If you can imagine creating two of the mock 350data managers we've made within application code, if one has a problem 351during "tpc_vote", neither will actually write a file to the ultimate 352location, and thus your application consistency is maintained. 353 354Integrating Your Data Manager With :mod:`repoze.tm2` 355---------------------------------------------------- 356 357The :mod:`repoze.tm2` transaction management machinery has an implicit 358policy. When it is in the WSGI pipeline, a transaction is started 359when the middleware is invoked. Thus, in your application code, 360calling "import transaction; transaction.get()" will return the 361transaction object created by the :mod:`repoze.tm2` middleware. You 362needn't call t.commit() or t.abort() within your application code. 363You only need to call t.join, to register your data manager with the 364transaction. :mod:`repoze.tm2` will abort the transaction if an 365exception is raised by your application code or lower middleware 366before it returns a WSGI response. If your application or lower 367middleware raises an exception, the transaction is aborted. 368 369Cleanup 370------- 371 372When the :mod:`repoze.tm2` middleware is in the WSGI pipeline, a boolean 373key is present in the environment (``repoze.tm.active``). A utility 374function named :func:`repoze.tm.isActive` can be imported and passed the 375WSGI environment to check for activation: 376 377.. code-block:: python 378 379 from repoze.tm import isActive 380 tm_active = isActive(wsgi_environment) 381 382If an application needs to perform an action after a transaction ends, the 383:attr:`repoze.tm.after_end` registry may be used to register a callback. 384This object is an instance fo the :class:`repoze.tm.AfterEnd` class. The 385:meth:`repoze.tm.AfterEnd.register` method accepts a callback (accepting no 386arguments) and a transaction instance: 387 388.. code-block:: python 389 390 from repoze.tm import after_end 391 import transaction 392 t = transaction.get() # the current transaction 393 def func(): 394 pass # close a connection, etc 395 after_end.register(func, t) 396 397"after_end" callbacks should only be registered when the transaction 398manager is active, or a memory leak will result (registration cleanup 399happens only on transaction commit or abort, which is managed by 400:mod:`repoze.tm2` while in the pipeline). 401 402Further Documentation 403--------------------- 404 405Many database adapters written for Zope (e.g. for Postgres, MySQL, 406etc) use this transaction manager, so it should be possible to take a 407look in these places to see how to implement a more real-world 408transaction-aware database connector that uses this module in non-Zope 409applications: 410 411- http://www.zodb.org/en/latest/documentation/guide/transactions.html 412 413- http://mysql-python.sourceforge.net/ (ZMySQLDA) 414 415- http://www.initd.org/svn/psycopg/psycopg2/trunk/ (ZPsycoPGDA) 416 417Contacting 418---------- 419 420The `repoze-dev maillist 421<http://lists.repoze.org/mailman/listinfo/repoze-dev>`_ should be used 422for communications about this software. 423 424Report bugs on Github: https://github.com/repoze/repoze.tm2/issues 425 426Fork it on Github: https://github.com/repoze/repoze.tm2/ 427 428API Docs 429------------------------ 430 431.. toctree:: 432 :maxdepth: 3 433 434 api 435 436 437Change Logs 438----------- 439 440.. toctree:: 441 :maxdepth: 2 442 443 changes 444 445Indices and tables 446------------------ 447 448* :ref:`genindex` 449* :ref:`modindex` 450* :ref:`search` 451