Obscure netrc parser "bug" #78313

Activity

1. smontanaro commented on Jul 16, 2018

   

   smontanaro

   on Jul 16, 2018

   ContributorAuthor

   More actions

   Not sure I can really call this a bug, however there is a behavioral change between 2.7 and at least 3.6 and 3.7 (probably earlier versions of the 3.x series as well). There is no spec for .netrc files that I can find, certainly nothing which mentions comment or blank lines. Still, Python's netrc file parser seems happy with both.

   However, in 3.x a blank line followed immediately by a comment line containing actual comment text causes the parser to raise a parse error. I've attached two netrc files, netrc-comment-blank, and netrc-blank-comment, identical save for the ordering of a blank line and a comment line. Here's what a 2.7.14 session looks like:

   Python 2.7.14 |Anaconda, Inc.| (default, Mar 27 2018, 17:29:31) 
   [GCC 7.2.0] on linux2
   Type "help", "copyright", "credits" or "license" for more information.
   >>> import netrc
   >>> rc = netrc.netrc(file="/home/skip/tmp/netrc-comment-blank")
   >>> rc = netrc.netrc(file="/home/skip/tmp/netrc-blank-comment")

   Here's 3.7.0:

   Python 3.7.0 (default, Jun 28 2018, 13:15:42) 
   [GCC 7.2.0] :: Anaconda, Inc. on linux
   Type "help", "copyright", "credits" or "license" for more information.
   >>> import netrc
   >>> rc = netrc.netrc(file="/home/skip/tmp/netrc-comment-blank")
   >>> rc = netrc.netrc(file="/home/skip/tmp/netrc-blank-comment")
   Traceback (most recent call last):
     File "<stdin>", line 1, in <module>
     File "/home/skip/miniconda3/envs/python3/lib/python3.7/netrc.py", line 30, in __init__
       self._parse(file, fp, default_netrc)
     File "/home/skip/miniconda3/envs/python3/lib/python3.7/netrc.py", line 63, in _parse
       "bad toplevel token %r" % tt, file, lexer.lineno)
   netrc.NetrcParseError: bad toplevel token 'Comment' (/home/skip/tmp/netrc-blank-comment, line 2)
2. smontanaro

   added

   3.7 (EOL)end of lifeend of life

   3.8 (EOL)end of lifeend of life

   stdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directory

   type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

   on Jul 16, 2018
3. tirkarthi commented on Jul 17, 2018

   

   tirkarthi

   on Jul 17, 2018

   Member

   More actions

   This seems to have been introduced with commit : 1df0f21 before which it was working fine.

   Testing shows the error from 1df0f21 and above. I have checked out only versions of Lib/netrc.py and I think this doesn't require a rebuild of Python.

   ➜ cpython git:(master) $ git log --oneline --format="%h" Lib/netrc.py > commits.txt # Get all commits to test the changes
   ➜ cpython git:(master) ✗ ./python netrc_foo_tester.py

   netrc_foo_tester.py is attached with the message which basically does the below using Python for every commit. netrc_foo.py contains the actual test of parsing two files.

   ➜ cpython git:(master) ✗ ./python
   Python 3.8.0a0 (heads/master:35c0809, Jul 16 2018, 10:29:23)
   [GCC 5.4.0 20160609] on linux
   Type "help", "copyright", "credits" or "license" for more information.

   >>

   ➜ cpython git:(master) ✗ cat netrc_foo.py
   import netrc

   rc = netrc.netrc(file="netrc-comment-blank")
   rc = netrc.netrc(file="netrc-blank-comment")

   ➜  cpython git:(master) ✗ git reset --quiet HEAD . && git checkout . && git checkout be19ed7 Lib/netrc.py && ./python netrc_foo.py
   netrc_foo.py:2: ResourceWarning: unclosed file <_io.TextIOWrapper name='netrc-comment-blank' mode='r' encoding='UTF-8'>
     rc = netrc.netrc(file="netrc-comment-blank")
   netrc_foo.py:3: ResourceWarning: unclosed file <_io.TextIOWrapper name='netrc-blank-comment' mode='r' encoding='UTF-8'>
     rc = netrc.netrc(file="netrc-blank-comment")
   ➜  cpython git:(master) ✗ git reset --quiet HEAD . && git checkout . && git checkout c12a813 Lib/netrc.py && ./python netrc_foo.py
   ➜  cpython git:(master) ✗ git reset --quiet HEAD . && git checkout . && git checkout 78a1a15 Lib/netrc.py && ./python netrc_foo.py
   ➜  cpython git:(master) ✗ git reset --quiet HEAD . && git checkout . && git checkout 1df0f21 Lib/netrc.py && ./python netrc_foo.py
   Traceback (most recent call last):
     File "netrc_foo.py", line 3, in <module>
       rc = netrc.netrc(file="netrc-blank-comment")
     File "/home/cpython/Lib/netrc.py", line 32, in __init__
       self._parse(file, fp)
     File "/home/cpython/Lib/netrc.py", line 65, in _parse
       "bad toplevel token %r" % tt, file, lexer.lineno)
   netrc.NetrcParseError: bad toplevel token 'Comment' (netrc-blank-comment, line 2)

   I hope the above approach is correct.

   Thanks
4. bbayles commented on Jul 20, 2018

   

   bbaylesmannequin

   on Jul 20, 2018

   Mannequin

   More actions

   I took a shot at this, but closed my PR when I found that it introduced a different regression.

   After playing with it for a bit, I think that this commit [1] is a bit problematic because it uses lexer.instream.readline(), which can make the line number in lexer.lineno incorrect.

   I may take another look if I have time, but someone else is free to tackle this.

   [1] 1df0f21#diff-e6896f6d68ca3fd8094a933533f8b2ed
5. bbayles commented on Jul 21, 2018

   

   bbaylesmannequin

   on Jul 21, 2018

   Mannequin

   More actions

   I realized that the Python 2 solution was adapt-able after all; I've re-submitted as GitHub PR 8360.

   #8360
6. ezio-melotti

   transferred this issue fromon Apr 10, 2022
7. diegogimenez04 commented on Jun 9, 2026

   

   diegogimenez04

   on Jun 9, 2026

   Last edited by diegogimenez04

   More actions

   Hello everyone. Is this still an issue in newer versions? If not, I suggest closing it.
8. hugovk commented on Oct 25, 2026

   

   hugovk

   on Oct 25, 2026

   Member

   More actions

   Likely duplicate of #79089, fixed by #26330.
9. hugovk

   closed this as completedon Oct 25, 2026