1:mod:`nntplib` --- NNTP protocol client 2======================================= 3 4.. module:: nntplib 5 :synopsis: NNTP protocol client (requires sockets). 6 7**Source code:** :source:`Lib/nntplib.py` 8 9.. index:: 10 pair: NNTP; protocol 11 single: Network News Transfer Protocol 12 13-------------- 14 15This module defines the class :class:`NNTP` which implements the client side of 16the Network News Transfer Protocol. It can be used to implement a news reader 17or poster, or automated news processors. It is compatible with :rfc:`3977` 18as well as the older :rfc:`977` and :rfc:`2980`. 19 20Here are two small examples of how it can be used. To list some statistics 21about a newsgroup and print the subjects of the last 10 articles:: 22 23 >>> s = nntplib.NNTP('news.gmane.io') 24 >>> resp, count, first, last, name = s.group('gmane.comp.python.committers') 25 >>> print('Group', name, 'has', count, 'articles, range', first, 'to', last) 26 Group gmane.comp.python.committers has 1096 articles, range 1 to 1096 27 >>> resp, overviews = s.over((last - 9, last)) 28 >>> for id, over in overviews: 29 ... print(id, nntplib.decode_header(over['subject'])) 30 ... 31 1087 Re: Commit privileges for Łukasz Langa 32 1088 Re: 3.2 alpha 2 freeze 33 1089 Re: 3.2 alpha 2 freeze 34 1090 Re: Commit privileges for Łukasz Langa 35 1091 Re: Commit privileges for Łukasz Langa 36 1092 Updated ssh key 37 1093 Re: Updated ssh key 38 1094 Re: Updated ssh key 39 1095 Hello fellow committers! 40 1096 Re: Hello fellow committers! 41 >>> s.quit() 42 '205 Bye!' 43 44To post an article from a binary file (this assumes that the article has valid 45headers, and that you have right to post on the particular newsgroup):: 46 47 >>> s = nntplib.NNTP('news.gmane.io') 48 >>> f = open('article.txt', 'rb') 49 >>> s.post(f) 50 '240 Article posted successfully.' 51 >>> s.quit() 52 '205 Bye!' 53 54The module itself defines the following classes: 55 56 57.. class:: NNTP(host, port=119, user=None, password=None, readermode=None, usenetrc=False, [timeout]) 58 59 Return a new :class:`NNTP` object, representing a connection 60 to the NNTP server running on host *host*, listening at port *port*. 61 An optional *timeout* can be specified for the socket connection. 62 If the optional *user* and *password* are provided, or if suitable 63 credentials are present in :file:`/.netrc` and the optional flag *usenetrc* 64 is true, the ``AUTHINFO USER`` and ``AUTHINFO PASS`` commands are used 65 to identify and authenticate the user to the server. If the optional 66 flag *readermode* is true, then a ``mode reader`` command is sent before 67 authentication is performed. Reader mode is sometimes necessary if you are 68 connecting to an NNTP server on the local machine and intend to call 69 reader-specific commands, such as ``group``. If you get unexpected 70 :exc:`NNTPPermanentError`\ s, you might need to set *readermode*. 71 The :class:`NNTP` class supports the :keyword:`with` statement to 72 unconditionally consume :exc:`OSError` exceptions and to close the NNTP 73 connection when done, e.g.: 74 75 >>> from nntplib import NNTP 76 >>> with NNTP('news.gmane.io') as n: 77 ... n.group('gmane.comp.python.committers') 78 ... # doctest: +SKIP 79 ('211 1755 1 1755 gmane.comp.python.committers', 1755, 1, 1755, 'gmane.comp.python.committers') 80 >>> 81 82 .. audit-event:: nntplib.connect self,host,port nntplib.NNTP 83 84 .. audit-event:: nntplib.putline self,line nntplib.NNTP 85 86 All commands will raise an :ref:`auditing event <auditing>` 87 ``nntplib.putline`` with arguments ``self`` and ``line``, 88 where ``line`` is the bytes about to be sent to the remote host. 89 90 .. versionchanged:: 3.2 91 *usenetrc* is now ``False`` by default. 92 93 .. versionchanged:: 3.3 94 Support for the :keyword:`with` statement was added. 95 96.. class:: NNTP_SSL(host, port=563, user=None, password=None, ssl_context=None, readermode=None, usenetrc=False, [timeout]) 97 98 Return a new :class:`NNTP_SSL` object, representing an encrypted 99 connection to the NNTP server running on host *host*, listening at 100 port *port*. :class:`NNTP_SSL` objects have the same methods as 101 :class:`NNTP` objects. If *port* is omitted, port 563 (NNTPS) is used. 102 *ssl_context* is also optional, and is a :class:`~ssl.SSLContext` object. 103 Please read :ref:`ssl-security` for best practices. 104 All other parameters behave the same as for :class:`NNTP`. 105 106 Note that SSL-on-563 is discouraged per :rfc:`4642`, in favor of 107 STARTTLS as described below. However, some servers only support the 108 former. 109 110 .. audit-event:: nntplib.connect self,host,port nntplib.NNTP_SSL 111 112 .. audit-event:: nntplib.putline self,line nntplib.NNTP_SSL 113 114 All commands will raise an :ref:`auditing event <auditing>` 115 ``nntplib.putline`` with arguments ``self`` and ``line``, 116 where ``line`` is the bytes about to be sent to the remote host. 117 118 .. versionadded:: 3.2 119 120 .. versionchanged:: 3.4 121 The class now supports hostname check with 122 :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see 123 :data:`ssl.HAS_SNI`). 124 125.. exception:: NNTPError 126 127 Derived from the standard exception :exc:`Exception`, this is the base 128 class for all exceptions raised by the :mod:`nntplib` module. Instances 129 of this class have the following attribute: 130 131 .. attribute:: response 132 133 The response of the server if available, as a :class:`str` object. 134 135 136.. exception:: NNTPReplyError 137 138 Exception raised when an unexpected reply is received from the server. 139 140 141.. exception:: NNTPTemporaryError 142 143 Exception raised when a response code in the range 400--499 is received. 144 145 146.. exception:: NNTPPermanentError 147 148 Exception raised when a response code in the range 500--599 is received. 149 150 151.. exception:: NNTPProtocolError 152 153 Exception raised when a reply is received from the server that does not begin 154 with a digit in the range 1--5. 155 156 157.. exception:: NNTPDataError 158 159 Exception raised when there is some error in the response data. 160 161 162.. _nntp-objects: 163 164NNTP Objects 165------------ 166 167When connected, :class:`NNTP` and :class:`NNTP_SSL` objects support the 168following methods and attributes. 169 170Attributes 171^^^^^^^^^^ 172 173.. attribute:: NNTP.nntp_version 174 175 An integer representing the version of the NNTP protocol supported by the 176 server. In practice, this should be ``2`` for servers advertising 177 :rfc:`3977` compliance and ``1`` for others. 178 179 .. versionadded:: 3.2 180 181.. attribute:: NNTP.nntp_implementation 182 183 A string describing the software name and version of the NNTP server, 184 or :const:`None` if not advertised by the server. 185 186 .. versionadded:: 3.2 187 188Methods 189^^^^^^^ 190 191The *response* that is returned as the first item in the return tuple of almost 192all methods is the server's response: a string beginning with a three-digit 193code. If the server's response indicates an error, the method raises one of 194the above exceptions. 195 196Many of the following methods take an optional keyword-only argument *file*. 197When the *file* argument is supplied, it must be either a :term:`file object` 198opened for binary writing, or the name of an on-disk file to be written to. 199The method will then write any data returned by the server (except for the 200response line and the terminating dot) to the file; any list of lines, 201tuples or objects that the method normally returns will be empty. 202 203.. versionchanged:: 3.2 204 Many of the following methods have been reworked and fixed, which makes 205 them incompatible with their 3.1 counterparts. 206 207 208.. method:: NNTP.quit() 209 210 Send a ``QUIT`` command and close the connection. Once this method has been 211 called, no other methods of the NNTP object should be called. 212 213 214.. method:: NNTP.getwelcome() 215 216 Return the welcome message sent by the server in reply to the initial 217 connection. (This message sometimes contains disclaimers or help information 218 that may be relevant to the user.) 219 220 221.. method:: NNTP.getcapabilities() 222 223 Return the :rfc:`3977` capabilities advertised by the server, as a 224 :class:`dict` instance mapping capability names to (possibly empty) lists 225 of values. On legacy servers which don't understand the ``CAPABILITIES`` 226 command, an empty dictionary is returned instead. 227 228 >>> s = NNTP('news.gmane.io') 229 >>> 'POST' in s.getcapabilities() 230 True 231 232 .. versionadded:: 3.2 233 234 235.. method:: NNTP.login(user=None, password=None, usenetrc=True) 236 237 Send ``AUTHINFO`` commands with the user name and password. If *user* 238 and *password* are ``None`` and *usenetrc* is true, credentials from 239 ``~/.netrc`` will be used if possible. 240 241 Unless intentionally delayed, login is normally performed during the 242 :class:`NNTP` object initialization and separately calling this function 243 is unnecessary. To force authentication to be delayed, you must not set 244 *user* or *password* when creating the object, and must set *usenetrc* to 245 False. 246 247 .. versionadded:: 3.2 248 249 250.. method:: NNTP.starttls(context=None) 251 252 Send a ``STARTTLS`` command. This will enable encryption on the NNTP 253 connection. The *context* argument is optional and should be a 254 :class:`ssl.SSLContext` object. Please read :ref:`ssl-security` for best 255 practices. 256 257 Note that this may not be done after authentication information has 258 been transmitted, and authentication occurs by default if possible during a 259 :class:`NNTP` object initialization. See :meth:`NNTP.login` for information 260 on suppressing this behavior. 261 262 .. versionadded:: 3.2 263 264 .. versionchanged:: 3.4 265 The method now supports hostname check with 266 :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see 267 :data:`ssl.HAS_SNI`). 268 269.. method:: NNTP.newgroups(date, *, file=None) 270 271 Send a ``NEWGROUPS`` command. The *date* argument should be a 272 :class:`datetime.date` or :class:`datetime.datetime` object. 273 Return a pair ``(response, groups)`` where *groups* is a list representing 274 the groups that are new since the given *date*. If *file* is supplied, 275 though, then *groups* will be empty. 276 277 >>> from datetime import date, timedelta 278 >>> resp, groups = s.newgroups(date.today() - timedelta(days=3)) 279 >>> len(groups) # doctest: +SKIP 280 85 281 >>> groups[0] # doctest: +SKIP 282 GroupInfo(group='gmane.network.tor.devel', last='4', first='1', flag='m') 283 284 285.. method:: NNTP.newnews(group, date, *, file=None) 286 287 Send a ``NEWNEWS`` command. Here, *group* is a group name or ``'*'``, and 288 *date* has the same meaning as for :meth:`newgroups`. Return a pair 289 ``(response, articles)`` where *articles* is a list of message ids. 290 291 This command is frequently disabled by NNTP server administrators. 292 293 294.. method:: NNTP.list(group_pattern=None, *, file=None) 295 296 Send a ``LIST`` or ``LIST ACTIVE`` command. Return a pair 297 ``(response, list)`` where *list* is a list of tuples representing all 298 the groups available from this NNTP server, optionally matching the 299 pattern string *group_pattern*. Each tuple has the form 300 ``(group, last, first, flag)``, where *group* is a group name, *last* 301 and *first* are the last and first article numbers, and *flag* usually 302 takes one of these values: 303 304 * ``y``: Local postings and articles from peers are allowed. 305 * ``m``: The group is moderated and all postings must be approved. 306 * ``n``: No local postings are allowed, only articles from peers. 307 * ``j``: Articles from peers are filed in the junk group instead. 308 * ``x``: No local postings, and articles from peers are ignored. 309 * ``=foo.bar``: Articles are filed in the ``foo.bar`` group instead. 310 311 If *flag* has another value, then the status of the newsgroup should be 312 considered unknown. 313 314 This command can return very large results, especially if *group_pattern* 315 is not specified. It is best to cache the results offline unless you 316 really need to refresh them. 317 318 .. versionchanged:: 3.2 319 *group_pattern* was added. 320 321 322.. method:: NNTP.descriptions(grouppattern) 323 324 Send a ``LIST NEWSGROUPS`` command, where *grouppattern* is a wildmat string as 325 specified in :rfc:`3977` (it's essentially the same as DOS or UNIX shell wildcard 326 strings). Return a pair ``(response, descriptions)``, where *descriptions* 327 is a dictionary mapping group names to textual descriptions. 328 329 >>> resp, descs = s.descriptions('gmane.comp.python.*') 330 >>> len(descs) # doctest: +SKIP 331 295 332 >>> descs.popitem() # doctest: +SKIP 333 ('gmane.comp.python.bio.general', 'BioPython discussion list (Moderated)') 334 335 336.. method:: NNTP.description(group) 337 338 Get a description for a single group *group*. If more than one group matches 339 (if 'group' is a real wildmat string), return the first match. If no group 340 matches, return an empty string. 341 342 This elides the response code from the server. If the response code is needed, 343 use :meth:`descriptions`. 344 345 346.. method:: NNTP.group(name) 347 348 Send a ``GROUP`` command, where *name* is the group name. The group is 349 selected as the current group, if it exists. Return a tuple 350 ``(response, count, first, last, name)`` where *count* is the (estimated) 351 number of articles in the group, *first* is the first article number in 352 the group, *last* is the last article number in the group, and *name* 353 is the group name. 354 355 356.. method:: NNTP.over(message_spec, *, file=None) 357 358 Send an ``OVER`` command, or an ``XOVER`` command on legacy servers. 359 *message_spec* can be either a string representing a message id, or 360 a ``(first, last)`` tuple of numbers indicating a range of articles in 361 the current group, or a ``(first, None)`` tuple indicating a range of 362 articles starting from *first* to the last article in the current group, 363 or :const:`None` to select the current article in the current group. 364 365 Return a pair ``(response, overviews)``. *overviews* is a list of 366 ``(article_number, overview)`` tuples, one for each article selected 367 by *message_spec*. Each *overview* is a dictionary with the same number 368 of items, but this number depends on the server. These items are either 369 message headers (the key is then the lower-cased header name) or metadata 370 items (the key is then the metadata name prepended with ``":"``). The 371 following items are guaranteed to be present by the NNTP specification: 372 373 * the ``subject``, ``from``, ``date``, ``message-id`` and ``references`` 374 headers 375 * the ``:bytes`` metadata: the number of bytes in the entire raw article 376 (including headers and body) 377 * the ``:lines`` metadata: the number of lines in the article body 378 379 The value of each item is either a string, or :const:`None` if not present. 380 381 It is advisable to use the :func:`decode_header` function on header 382 values when they may contain non-ASCII characters:: 383 384 >>> _, _, first, last, _ = s.group('gmane.comp.python.devel') 385 >>> resp, overviews = s.over((last, last)) 386 >>> art_num, over = overviews[0] 387 >>> art_num 388 117216 389 >>> list(over.keys()) 390 ['xref', 'from', ':lines', ':bytes', 'references', 'date', 'message-id', 'subject'] 391 >>> over['from'] 392 '=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?= <martin@v.loewis.de>' 393 >>> nntplib.decode_header(over['from']) 394 '"Martin v. Löwis" <martin@v.loewis.de>' 395 396 .. versionadded:: 3.2 397 398 399.. method:: NNTP.help(*, file=None) 400 401 Send a ``HELP`` command. Return a pair ``(response, list)`` where *list* is a 402 list of help strings. 403 404 405.. method:: NNTP.stat(message_spec=None) 406 407 Send a ``STAT`` command, where *message_spec* is either a message id 408 (enclosed in ``'<'`` and ``'>'``) or an article number in the current group. 409 If *message_spec* is omitted or :const:`None`, the current article in the 410 current group is considered. Return a triple ``(response, number, id)`` 411 where *number* is the article number and *id* is the message id. 412 413 >>> _, _, first, last, _ = s.group('gmane.comp.python.devel') 414 >>> resp, number, message_id = s.stat(first) 415 >>> number, message_id 416 (9099, '<20030112190404.GE29873@epoch.metaslash.com>') 417 418 419.. method:: NNTP.next() 420 421 Send a ``NEXT`` command. Return as for :meth:`.stat`. 422 423 424.. method:: NNTP.last() 425 426 Send a ``LAST`` command. Return as for :meth:`.stat`. 427 428 429.. method:: NNTP.article(message_spec=None, *, file=None) 430 431 Send an ``ARTICLE`` command, where *message_spec* has the same meaning as 432 for :meth:`.stat`. Return a tuple ``(response, info)`` where *info* 433 is a :class:`~collections.namedtuple` with three attributes *number*, 434 *message_id* and *lines* (in that order). *number* is the article number 435 in the group (or 0 if the information is not available), *message_id* the 436 message id as a string, and *lines* a list of lines (without terminating 437 newlines) comprising the raw message including headers and body. 438 439 >>> resp, info = s.article('<20030112190404.GE29873@epoch.metaslash.com>') 440 >>> info.number 441 0 442 >>> info.message_id 443 '<20030112190404.GE29873@epoch.metaslash.com>' 444 >>> len(info.lines) 445 65 446 >>> info.lines[0] 447 b'Path: main.gmane.org!not-for-mail' 448 >>> info.lines[1] 449 b'From: Neal Norwitz <neal@metaslash.com>' 450 >>> info.lines[-3:] 451 [b'There is a patch for 2.3 as well as 2.2.', b'', b'Neal'] 452 453 454.. method:: NNTP.head(message_spec=None, *, file=None) 455 456 Same as :meth:`article()`, but sends a ``HEAD`` command. The *lines* 457 returned (or written to *file*) will only contain the message headers, not 458 the body. 459 460 461.. method:: NNTP.body(message_spec=None, *, file=None) 462 463 Same as :meth:`article()`, but sends a ``BODY`` command. The *lines* 464 returned (or written to *file*) will only contain the message body, not the 465 headers. 466 467 468.. method:: NNTP.post(data) 469 470 Post an article using the ``POST`` command. The *data* argument is either 471 a :term:`file object` opened for binary reading, or any iterable of bytes 472 objects (representing raw lines of the article to be posted). It should 473 represent a well-formed news article, including the required headers. The 474 :meth:`post` method automatically escapes lines beginning with ``.`` and 475 appends the termination line. 476 477 If the method succeeds, the server's response is returned. If the server 478 refuses posting, a :class:`NNTPReplyError` is raised. 479 480 481.. method:: NNTP.ihave(message_id, data) 482 483 Send an ``IHAVE`` command. *message_id* is the id of the message to send 484 to the server (enclosed in ``'<'`` and ``'>'``). The *data* parameter 485 and the return value are the same as for :meth:`post()`. 486 487 488.. method:: NNTP.date() 489 490 Return a pair ``(response, date)``. *date* is a :class:`~datetime.datetime` 491 object containing the current date and time of the server. 492 493 494.. method:: NNTP.slave() 495 496 Send a ``SLAVE`` command. Return the server's *response*. 497 498 499.. method:: NNTP.set_debuglevel(level) 500 501 Set the instance's debugging level. This controls the amount of debugging 502 output printed. The default, ``0``, produces no debugging output. A value of 503 ``1`` produces a moderate amount of debugging output, generally a single line 504 per request or response. A value of ``2`` or higher produces the maximum amount 505 of debugging output, logging each line sent and received on the connection 506 (including message text). 507 508 509The following are optional NNTP extensions defined in :rfc:`2980`. Some of 510them have been superseded by newer commands in :rfc:`3977`. 511 512 513.. method:: NNTP.xhdr(hdr, str, *, file=None) 514 515 Send an ``XHDR`` command. The *hdr* argument is a header keyword, e.g. 516 ``'subject'``. The *str* argument should have the form ``'first-last'`` 517 where *first* and *last* are the first and last article numbers to search. 518 Return a pair ``(response, list)``, where *list* is a list of pairs ``(id, 519 text)``, where *id* is an article number (as a string) and *text* is the text of 520 the requested header for that article. If the *file* parameter is supplied, then 521 the output of the ``XHDR`` command is stored in a file. If *file* is a string, 522 then the method will open a file with that name, write to it then close it. 523 If *file* is a :term:`file object`, then it will start calling :meth:`write` on 524 it to store the lines of the command output. If *file* is supplied, then the 525 returned *list* is an empty list. 526 527 528.. method:: NNTP.xover(start, end, *, file=None) 529 530 Send an ``XOVER`` command. *start* and *end* are article numbers 531 delimiting the range of articles to select. The return value is the 532 same of for :meth:`over()`. It is recommended to use :meth:`over()` 533 instead, since it will automatically use the newer ``OVER`` command 534 if available. 535 536 537.. method:: NNTP.xpath(id) 538 539 Return a pair ``(resp, path)``, where *path* is the directory path to the 540 article with message ID *id*. Most of the time, this extension is not 541 enabled by NNTP server administrators. 542 543 .. deprecated:: 3.3 544 The XPATH extension is not actively used. 545 546 547.. XXX deprecated: 548 549 .. method:: NNTP.xgtitle(name, *, file=None) 550 551 Process an ``XGTITLE`` command, returning a pair ``(response, list)``, where 552 *list* is a list of tuples containing ``(name, title)``. If the *file* parameter 553 is supplied, then the output of the ``XGTITLE`` command is stored in a file. 554 If *file* is a string, then the method will open a file with that name, write 555 to it then close it. If *file* is a :term:`file object`, then it will start 556 calling :meth:`write` on it to store the lines of the command output. If *file* 557 is supplied, then the returned *list* is an empty list. This is an optional NNTP 558 extension, and may not be supported by all servers. 559 560 :rfc:`2980` says "It is suggested that this extension be deprecated". Use 561 :meth:`descriptions` or :meth:`description` instead. 562 563 564Utility functions 565----------------- 566 567The module also defines the following utility function: 568 569 570.. function:: decode_header(header_str) 571 572 Decode a header value, un-escaping any escaped non-ASCII characters. 573 *header_str* must be a :class:`str` object. The unescaped value is 574 returned. Using this function is recommended to display some headers 575 in a human readable form:: 576 577 >>> decode_header("Some subject") 578 'Some subject' 579 >>> decode_header("=?ISO-8859-15?Q?D=E9buter_en_Python?=") 580 'Débuter en Python' 581 >>> decode_header("Re: =?UTF-8?B?cHJvYmzDqG1lIGRlIG1hdHJpY2U=?=") 582 'Re: problème de matrice' 583