1import collections 2import contextlib 3import os 4import pathlib 5import weakref 6 7import notmuch2._base as base 8import notmuch2._capi as capi 9import notmuch2._errors as errors 10import notmuch2._tags as tags 11 12 13__all__ = ['Message'] 14 15 16class Message(base.NotmuchObject): 17 """An email message stored in the notmuch database retrieved via a query. 18 19 This should not be directly created, instead it will be returned 20 by calling methods on :class:`Database`. A message keeps a 21 reference to the database object since the database object can not 22 be released while the message is in use. 23 24 Note that this represents a message in the notmuch database. For 25 full email functionality you may want to use the :mod:`email` 26 package from Python's standard library. You could e.g. create 27 this as such:: 28 29 notmuch_msg = db.get_message(msgid) # or from a query 30 parser = email.parser.BytesParser(policy=email.policy.default) 31 with notmuch_msg.path.open('rb) as fp: 32 email_msg = parser.parse(fp) 33 34 Most commonly the functionality provided by notmuch is sufficient 35 to read email however. 36 37 Messages are considered equal when they have the same message ID. 38 This is how libnotmuch treats messages as well, the 39 :meth:`pathnames` function returns multiple results for 40 duplicates. 41 42 :param parent: The parent object. This is probably one off a 43 :class:`Database`, :class:`Thread` or :class:`Query`. 44 :type parent: NotmuchObject 45 :param db: The database instance this message is associated with. 46 This could be the same as the parent. 47 :type db: Database 48 :param msg_p: The C pointer to the ``notmuch_message_t``. 49 :type msg_p: <cdata> 50 :param dup: Whether the message was a duplicate on insertion. 51 :type dup: None or bool 52 """ 53 _msg_p = base.MemoryPointer() 54 55 def __init__(self, parent, msg_p, *, db): 56 self._parent = parent 57 self._msg_p = msg_p 58 self._db = db 59 60 @property 61 def alive(self): 62 if not self._parent.alive: 63 return False 64 try: 65 self._msg_p 66 except errors.ObjectDestroyedError: 67 return False 68 else: 69 return True 70 71 def __del__(self): 72 self._destroy() 73 74 def _destroy(self): 75 if self.alive: 76 capi.lib.notmuch_message_destroy(self._msg_p) 77 self._msg_p = None 78 79 @property 80 def messageid(self): 81 """The message ID as a string. 82 83 The message ID is decoded with the ignore error handler. This 84 is fine as long as the message ID is well formed. If it is 85 not valid ASCII then this will be lossy. So if you need to be 86 able to write the exact same message ID back you should use 87 :attr:`messageidb`. 88 89 Note that notmuch will decode the message ID value and thus 90 strip off the surrounding ``<`` and ``>`` characters. This is 91 different from Python's :mod:`email` package behaviour which 92 leaves these characters in place. 93 94 :returns: The message ID. 95 :rtype: :class:`BinString`, this is a normal str but calling 96 bytes() on it will return the original bytes used to create 97 it. 98 99 :raises ObjectDestroyedError: if used after destroyed. 100 """ 101 ret = capi.lib.notmuch_message_get_message_id(self._msg_p) 102 return base.BinString(capi.ffi.string(ret)) 103 104 @property 105 def threadid(self): 106 """The thread ID. 107 108 The thread ID is decoded with the surrogateescape error 109 handler so that it is possible to reconstruct the original 110 thread ID if it is not valid UTF-8. 111 112 :returns: The thread ID. 113 :rtype: :class:`BinString`, this is a normal str but calling 114 bytes() on it will return the original bytes used to create 115 it. 116 117 :raises ObjectDestroyedError: if used after destroyed. 118 """ 119 ret = capi.lib.notmuch_message_get_thread_id(self._msg_p) 120 return base.BinString(capi.ffi.string(ret)) 121 122 @property 123 def path(self): 124 """A pathname of the message as a pathlib.Path instance. 125 126 If multiple files in the database contain the same message ID 127 this will be just one of the files, chosen at random. 128 129 :raises ObjectDestroyedError: if used after destroyed. 130 """ 131 ret = capi.lib.notmuch_message_get_filename(self._msg_p) 132 return pathlib.Path(os.fsdecode(capi.ffi.string(ret))) 133 134 @property 135 def pathb(self): 136 """A pathname of the message as a bytes object. 137 138 See :attr:`path` for details, this is the same but does return 139 the path as a bytes object which is faster but less convenient. 140 141 :raises ObjectDestroyedError: if used after destroyed. 142 """ 143 ret = capi.lib.notmuch_message_get_filename(self._msg_p) 144 return capi.ffi.string(ret) 145 146 def filenames(self): 147 """Return an iterator of all files for this message. 148 149 If multiple files contained the same message ID they will all 150 be returned here. The files are returned as instances of 151 :class:`pathlib.Path`. 152 153 :returns: Iterator yielding :class:`pathlib.Path` instances. 154 :rtype: iter 155 156 :raises ObjectDestroyedError: if used after destroyed. 157 """ 158 fnames_p = capi.lib.notmuch_message_get_filenames(self._msg_p) 159 return PathIter(self, fnames_p) 160 161 def filenamesb(self): 162 """Return an iterator of all files for this message. 163 164 This is like :meth:`pathnames` but the files are returned as 165 byte objects instead. 166 167 :returns: Iterator yielding :class:`bytes` instances. 168 :rtype: iter 169 170 :raises ObjectDestroyedError: if used after destroyed. 171 """ 172 fnames_p = capi.lib.notmuch_message_get_filenames(self._msg_p) 173 return FilenamesIter(self, fnames_p) 174 175 @property 176 def ghost(self): 177 """Indicates whether this message is a ghost message. 178 179 A ghost message if a message which we know exists, but it has 180 no files or content associated with it. This can happen if 181 it was referenced by some other message. Only the 182 :attr:`messageid` and :attr:`threadid` attributes are valid 183 for it. 184 185 :raises ObjectDestroyedError: if used after destroyed. 186 """ 187 ret = capi.lib.notmuch_message_get_flag( 188 self._msg_p, capi.lib.NOTMUCH_MESSAGE_FLAG_GHOST) 189 return bool(ret) 190 191 @property 192 def excluded(self): 193 """Indicates whether this message was excluded from the query. 194 195 When a message is created from a search, sometimes messages 196 that where excluded by the search query could still be 197 returned by it, e.g. because they are part of a thread 198 matching the query. the :meth:`Database.query` method allows 199 these messages to be flagged, which results in this property 200 being set to *True*. 201 202 :raises ObjectDestroyedError: if used after destroyed. 203 """ 204 ret = capi.lib.notmuch_message_get_flag( 205 self._msg_p, capi.lib.NOTMUCH_MESSAGE_FLAG_EXCLUDED) 206 return bool(ret) 207 208 @property 209 def date(self): 210 """The message date as an integer. 211 212 The time the message was sent as an integer number of seconds 213 since the *epoch*, 1 Jan 1970. This is derived from the 214 message's header, you can get the original header value with 215 :meth:`header`. 216 217 :raises ObjectDestroyedError: if used after destroyed. 218 """ 219 return capi.lib.notmuch_message_get_date(self._msg_p) 220 221 def header(self, name): 222 """Return the value of the named header. 223 224 Returns the header from notmuch, some common headers are 225 stored in the database, others are read from the file. 226 Headers are returned with their newlines stripped and 227 collapsed concatenated together if they occur multiple times. 228 You may be better off using the standard library email 229 package's ``email.message_from_file(msg.path.open())`` if that 230 is not sufficient for you. 231 232 :param header: Case-insensitive header name to retrieve. 233 :type header: str or bytes 234 235 :returns: The header value, an empty string if the header is 236 not present. 237 :rtype: str 238 239 :raises LookupError: if the header is not present. 240 :raises NullPointerError: For unexpected notmuch errors. 241 :raises ObjectDestroyedError: if used after destroyed. 242 """ 243 # The returned is supposedly guaranteed to be UTF-8. Header 244 # names must be ASCII as per RFC x822. 245 if isinstance(name, str): 246 name = name.encode('ascii') 247 ret = capi.lib.notmuch_message_get_header(self._msg_p, name) 248 if ret == capi.ffi.NULL: 249 raise errors.NullPointerError() 250 hdr = capi.ffi.string(ret) 251 if not hdr: 252 raise LookupError 253 return hdr.decode(encoding='utf-8') 254 255 @property 256 def tags(self): 257 """The tags associated with the message. 258 259 This behaves as a set. But removing and adding items to the 260 set removes and adds them to the message in the database. 261 262 :raises ReadOnlyDatabaseError: When manipulating tags on a 263 database opened in read-only mode. 264 :raises ObjectDestroyedError: if used after destroyed. 265 """ 266 try: 267 ref = self._cached_tagset 268 except AttributeError: 269 tagset = None 270 else: 271 tagset = ref() 272 if tagset is None: 273 tagset = tags.MutableTagSet( 274 self, '_msg_p', capi.lib.notmuch_message_get_tags) 275 self._cached_tagset = weakref.ref(tagset) 276 return tagset 277 278 @contextlib.contextmanager 279 def frozen(self): 280 """Context manager to freeze the message state. 281 282 This allows you to perform atomic tag updates:: 283 284 with msg.frozen(): 285 msg.tags.clear() 286 msg.tags.add('foo') 287 288 Using This would ensure the message never ends up with no tags 289 applied at all. 290 291 It is safe to nest calls to this context manager. 292 293 :raises ReadOnlyDatabaseError: if the database is opened in 294 read-only mode. 295 :raises UnbalancedFreezeThawError: if you somehow managed to 296 call __exit__ of this context manager more than once. Why 297 did you do that? 298 :raises ObjectDestroyedError: if used after destroyed. 299 """ 300 ret = capi.lib.notmuch_message_freeze(self._msg_p) 301 if ret != capi.lib.NOTMUCH_STATUS_SUCCESS: 302 raise errors.NotmuchError(ret) 303 self._frozen = True 304 try: 305 yield 306 except Exception: 307 # Only way to "rollback" these changes is to destroy 308 # ourselves and re-create. Behold. 309 msgid = self.messageid 310 self._destroy() 311 with contextlib.suppress(Exception): 312 new = self._db.find(msgid) 313 self._msg_p = new._msg_p 314 new._msg_p = None 315 del new 316 raise 317 else: 318 ret = capi.lib.notmuch_message_thaw(self._msg_p) 319 if ret != capi.lib.NOTMUCH_STATUS_SUCCESS: 320 raise errors.NotmuchError(ret) 321 self._frozen = False 322 323 @property 324 def properties(self): 325 """A map of arbitrary key-value pairs associated with the message. 326 327 Be aware that properties may be used by other extensions to 328 store state in. So delete or modify with care. 329 330 The properties map is somewhat special. It is essentially a 331 multimap-like structure where each key can have multiple 332 values. Therefore accessing a single item using 333 :meth:`PropertiesMap.get` or :meth:`PropertiesMap.__getitem__` 334 will only return you the *first* item if there are multiple 335 and thus are only recommended if you know there to be only one 336 value. 337 338 Instead the map has an additional :meth:`PropertiesMap.all` 339 method which can be used to retrieve all properties of a given 340 key. This method also allows iterating of a a subset of the 341 keys starting with a given prefix. 342 """ 343 try: 344 ref = self._cached_props 345 except AttributeError: 346 props = None 347 else: 348 props = ref() 349 if props is None: 350 props = PropertiesMap(self, '_msg_p') 351 self._cached_props = weakref.ref(props) 352 return props 353 354 def replies(self): 355 """Return an iterator of all replies to this message. 356 357 This method will only work if the message was created from a 358 thread. Otherwise it will yield no results. 359 360 :returns: An iterator yielding :class:`Message` instances. 361 :rtype: MessageIter 362 """ 363 # The notmuch_messages_valid call accepts NULL and this will 364 # become an empty iterator, raising StopIteration immediately. 365 # Hence no return value checking here. 366 msgs_p = capi.lib.notmuch_message_get_replies(self._msg_p) 367 return MessageIter(self, msgs_p, db=self._db) 368 369 def __hash__(self): 370 return hash(self.messageid) 371 372 def __eq__(self, other): 373 if isinstance(other, self.__class__): 374 return self.messageid == other.messageid 375 376 377class OwnedMessage(Message): 378 """An email message owned by parent thread object. 379 380 This subclass of Message is used for messages that are retrieved 381 from the notmuch database via a parent :class:`notmuch2.Thread` 382 object, which "owns" this message. This means that when this 383 message object is destroyed, by calling :func:`del` or 384 :meth:`_destroy` directly or indirectly, the message is not freed 385 in the notmuch API and the parent :class:`notmuch2.Thread` object 386 can return the same object again when needed. 387 """ 388 389 @property 390 def alive(self): 391 return self._parent.alive 392 393 def _destroy(self): 394 pass 395 396 397class FilenamesIter(base.NotmuchIter): 398 """Iterator for binary filenames objects.""" 399 400 def __init__(self, parent, iter_p): 401 super().__init__(parent, iter_p, 402 fn_destroy=capi.lib.notmuch_filenames_destroy, 403 fn_valid=capi.lib.notmuch_filenames_valid, 404 fn_get=capi.lib.notmuch_filenames_get, 405 fn_next=capi.lib.notmuch_filenames_move_to_next) 406 407 def __next__(self): 408 fname = super().__next__() 409 return capi.ffi.string(fname) 410 411 412class PathIter(FilenamesIter): 413 """Iterator for pathlib.Path objects.""" 414 415 def __next__(self): 416 fname = super().__next__() 417 return pathlib.Path(os.fsdecode(fname)) 418 419 420class PropertiesMap(base.NotmuchObject, collections.abc.MutableMapping): 421 """A mutable mapping to manage properties. 422 423 Both keys and values of properties are supposed to be UTF-8 424 strings in libnotmuch. However since the uderlying API uses 425 bytestrings you can use either str or bytes to represent keys and 426 all returned keys and values use :class:`BinString`. 427 428 Also be aware that ``iter(this_map)`` will return duplicate keys, 429 while the :class:`collections.abc.KeysView` returned by 430 :meth:`keys` is a :class:`collections.abc.Set` subclass. This 431 means the former will yield duplicate keys while the latter won't. 432 It also means ``len(list(iter(this_map)))`` could be different 433 than ``len(this_map.keys())``. ``len(this_map)`` will correspond 434 with the length of the default iterator. 435 436 Be aware that libnotmuch exposes all of this as iterators, so 437 quite a few operations have O(n) performance instead of the usual 438 O(1). 439 """ 440 Property = collections.namedtuple('Property', ['key', 'value']) 441 _marker = object() 442 443 def __init__(self, msg, ptr_name): 444 self._msg = msg 445 self._ptr = lambda: getattr(msg, ptr_name) 446 447 @property 448 def alive(self): 449 if not self._msg.alive: 450 return False 451 try: 452 self._ptr 453 except errors.ObjectDestroyedError: 454 return False 455 else: 456 return True 457 458 def _destroy(self): 459 pass 460 461 def __iter__(self): 462 """Return an iterator which iterates over the keys. 463 464 Be aware that a single key may have multiple values associated 465 with it, if so it will appear multiple times here. 466 """ 467 iter_p = capi.lib.notmuch_message_get_properties(self._ptr(), b'', 0) 468 return PropertiesKeyIter(self, iter_p) 469 470 def __len__(self): 471 iter_p = capi.lib.notmuch_message_get_properties(self._ptr(), b'', 0) 472 it = base.NotmuchIter( 473 self, iter_p, 474 fn_destroy=capi.lib.notmuch_message_properties_destroy, 475 fn_valid=capi.lib.notmuch_message_properties_valid, 476 fn_get=capi.lib.notmuch_message_properties_key, 477 fn_next=capi.lib.notmuch_message_properties_move_to_next, 478 ) 479 return len(list(it)) 480 481 def __getitem__(self, key): 482 """Return **the first** peroperty associated with a key.""" 483 if isinstance(key, str): 484 key = key.encode('utf-8') 485 value_pp = capi.ffi.new('char**') 486 ret = capi.lib.notmuch_message_get_property(self._ptr(), key, value_pp) 487 if ret != capi.lib.NOTMUCH_STATUS_SUCCESS: 488 raise errors.NotmuchError(ret) 489 if value_pp[0] == capi.ffi.NULL: 490 raise KeyError 491 return base.BinString.from_cffi(value_pp[0]) 492 493 def keys(self): 494 """Return a :class:`collections.abc.KeysView` for this map. 495 496 Even when keys occur multiple times this is a subset of set() 497 so will only contain them once. 498 """ 499 return collections.abc.KeysView({k: None for k in self}) 500 501 def items(self): 502 """Return a :class:`collections.abc.ItemsView` for this map. 503 504 The ItemsView treats a ``(key, value)`` pair as unique, so 505 dupcliate ``(key, value)`` pairs will be merged together. 506 However duplicate keys with different values will be returned. 507 """ 508 items = set() 509 props_p = capi.lib.notmuch_message_get_properties(self._ptr(), b'', 0) 510 while capi.lib.notmuch_message_properties_valid(props_p): 511 key = capi.lib.notmuch_message_properties_key(props_p) 512 value = capi.lib.notmuch_message_properties_value(props_p) 513 items.add((base.BinString.from_cffi(key), 514 base.BinString.from_cffi(value))) 515 capi.lib.notmuch_message_properties_move_to_next(props_p) 516 capi.lib.notmuch_message_properties_destroy(props_p) 517 return PropertiesItemsView(items) 518 519 def values(self): 520 """Return a :class:`collecions.abc.ValuesView` for this map. 521 522 All unique property values are included in the view. 523 """ 524 values = set() 525 props_p = capi.lib.notmuch_message_get_properties(self._ptr(), b'', 0) 526 while capi.lib.notmuch_message_properties_valid(props_p): 527 value = capi.lib.notmuch_message_properties_value(props_p) 528 values.add(base.BinString.from_cffi(value)) 529 capi.lib.notmuch_message_properties_move_to_next(props_p) 530 capi.lib.notmuch_message_properties_destroy(props_p) 531 return PropertiesValuesView(values) 532 533 def __setitem__(self, key, value): 534 """Add a key-value pair to the properties. 535 536 You may prefer to use :meth:`add` for clarity since this 537 method usually implies implicit overwriting of an existing key 538 if it exists, while for properties this is not the case. 539 """ 540 self.add(key, value) 541 542 def add(self, key, value): 543 """Add a key-value pair to the properties.""" 544 if isinstance(key, str): 545 key = key.encode('utf-8') 546 if isinstance(value, str): 547 value = value.encode('utf-8') 548 ret = capi.lib.notmuch_message_add_property(self._ptr(), key, value) 549 if ret != capi.lib.NOTMUCH_STATUS_SUCCESS: 550 raise errors.NotmuchError(ret) 551 552 def __delitem__(self, key): 553 """Remove all properties with this key.""" 554 if isinstance(key, str): 555 key = key.encode('utf-8') 556 ret = capi.lib.notmuch_message_remove_all_properties(self._ptr(), key) 557 if ret != capi.lib.NOTMUCH_STATUS_SUCCESS: 558 raise errors.NotmuchError(ret) 559 560 def remove(self, key, value): 561 """Remove a key-value pair from the properties.""" 562 if isinstance(key, str): 563 key = key.encode('utf-8') 564 if isinstance(value, str): 565 value = value.encode('utf-8') 566 ret = capi.lib.notmuch_message_remove_property(self._ptr(), key, value) 567 if ret != capi.lib.NOTMUCH_STATUS_SUCCESS: 568 raise errors.NotmuchError(ret) 569 570 def pop(self, key, default=_marker): 571 try: 572 value = self[key] 573 except KeyError: 574 if default is self._marker: 575 raise 576 else: 577 return default 578 else: 579 self.remove(key, value) 580 return value 581 582 def popitem(self): 583 try: 584 key = next(iter(self)) 585 except StopIteration: 586 raise KeyError 587 value = self.pop(key) 588 return (key, value) 589 590 def clear(self): 591 ret = capi.lib.notmuch_message_remove_all_properties(self._ptr(), 592 capi.ffi.NULL) 593 if ret != capi.lib.NOTMUCH_STATUS_SUCCESS: 594 raise errors.NotmuchError(ret) 595 596 def getall(self, prefix='', *, exact=False): 597 """Return an iterator yielding all properties for a given key prefix. 598 599 The returned iterator yields all peroperties which start with 600 a given key prefix as ``(key, value)`` namedtuples. If called 601 with ``exact=True`` then only properties which exactly match 602 the prefix are returned, those a key longer than the prefix 603 will not be included. 604 605 :param prefix: The prefix of the key. 606 """ 607 if isinstance(prefix, str): 608 prefix = prefix.encode('utf-8') 609 props_p = capi.lib.notmuch_message_get_properties(self._ptr(), 610 prefix, exact) 611 return PropertiesIter(self, props_p) 612 613 614class PropertiesKeyIter(base.NotmuchIter): 615 616 def __init__(self, parent, iter_p): 617 super().__init__( 618 parent, 619 iter_p, 620 fn_destroy=capi.lib.notmuch_message_properties_destroy, 621 fn_valid=capi.lib.notmuch_message_properties_valid, 622 fn_get=capi.lib.notmuch_message_properties_key, 623 fn_next=capi.lib.notmuch_message_properties_move_to_next) 624 625 def __next__(self): 626 item = super().__next__() 627 return base.BinString.from_cffi(item) 628 629 630class PropertiesIter(base.NotmuchIter): 631 632 def __init__(self, parent, iter_p): 633 super().__init__( 634 parent, 635 iter_p, 636 fn_destroy=capi.lib.notmuch_message_properties_destroy, 637 fn_valid=capi.lib.notmuch_message_properties_valid, 638 fn_get=capi.lib.notmuch_message_properties_key, 639 fn_next=capi.lib.notmuch_message_properties_move_to_next, 640 ) 641 642 def __next__(self): 643 if not self._fn_valid(self._iter_p): 644 self._destroy() 645 raise StopIteration 646 key = capi.lib.notmuch_message_properties_key(self._iter_p) 647 value = capi.lib.notmuch_message_properties_value(self._iter_p) 648 capi.lib.notmuch_message_properties_move_to_next(self._iter_p) 649 return PropertiesMap.Property(base.BinString.from_cffi(key), 650 base.BinString.from_cffi(value)) 651 652 653class PropertiesItemsView(collections.abc.Set): 654 655 __slots__ = ('_items',) 656 657 def __init__(self, items): 658 self._items = items 659 660 @classmethod 661 def _from_iterable(self, it): 662 return set(it) 663 664 def __len__(self): 665 return len(self._items) 666 667 def __contains__(self, item): 668 return item in self._items 669 670 def __iter__(self): 671 yield from self._items 672 673 674collections.abc.ItemsView.register(PropertiesItemsView) 675 676 677class PropertiesValuesView(collections.abc.Set): 678 679 __slots__ = ('_values',) 680 681 def __init__(self, values): 682 self._values = values 683 684 def __len__(self): 685 return len(self._values) 686 687 def __contains__(self, value): 688 return value in self._values 689 690 def __iter__(self): 691 yield from self._values 692 693 694collections.abc.ValuesView.register(PropertiesValuesView) 695 696 697class MessageIter(base.NotmuchIter): 698 699 def __init__(self, parent, msgs_p, *, db, msg_cls=Message): 700 self._db = db 701 self._msg_cls = msg_cls 702 super().__init__(parent, msgs_p, 703 fn_destroy=capi.lib.notmuch_messages_destroy, 704 fn_valid=capi.lib.notmuch_messages_valid, 705 fn_get=capi.lib.notmuch_messages_get, 706 fn_next=capi.lib.notmuch_messages_move_to_next) 707 708 def __next__(self): 709 msg_p = super().__next__() 710 return self._msg_cls(self, msg_p, db=self._db) 711