Bug of the Day: issue 19626 (closed)ΒΆ

I’m going try out something here. I’ve long had the ambition to make sure I close at least one Python bug a day (or at least make some significant progress on one), but I seem to rarely manage it. So let’s see if making a commitment in public to blog the bug I work on/close makes a difference in my ability to actually do the work :)

The blog entries will probably be very brief, since adding a bunch of additional overhead to closing a bug isn’t likely to allow me to close more bugs.

On the other hand, I like writing, so you may alternatively get more details than you want. Or this little experiment may crash and burn, we’ll see.

Today’s bug is a simple one: the report was that test_email was failing if the test suite was run with the python -OO flag asserted:

./python -OO -m test test_email

The cause (shown clearly in the traceback in the issue) is that the Policy metaclass copies the base class docstring and adds to it. The exception showed the _append_doc function trying to do a split on None. What -OO does is to remove docstrings, so that made sense, and I figured I needed to harden that code to handle a None docstring.

I couldn’t get the test to fail. At first I was thinking that -OO wasn’t actually removing docstrings, because I could still see them in the interactive interpreter...but I forgot when checking that to supply -OO when starting the shell, so of course it was reading the .pyc that had the docstrings instead of the .pyo file that didn’t.

While I was figuring that out I was chatting with another dev on IRC, and he was having the same problem, but he was supplying -OO when starting the python shell. Since I in the meantime had proven to myself that -OO was working, we quickly figured out that his problem was that he had previously run the test suite with -O, which only removes asserts but leaves the docstrings alone. And, due to a quirk in the way python handles .pyo files, if a .pyo with just assert strips exists, Python happily uses it even if -OO was specified on the command line. (This was reported a long time ago (in Web years) in issue 1538778 (closed), but it was closed as “won’t fix”, a decision that IMO needs to be revisited at some point.)

So...I still couldn’t get the test to fail. Looking at the code, it in fact checks for a docstring that isn’t set (if cls.__doc__ and ...), so it does in fact handle -OO. So why did the reporter get a test failure?

Again, looking at the code, somehow the reporter must have wound up in a situation where the _policybase.pyo file was created under the -OO flag (no docstrings), while the policy.py file (which contains the Policy subclasses) was imported using -O. So the _append_doc code thought that docstrings existed (it found one on the subclass), and blithely tried to use the one from the base class (which didn’t exist).

I’m not at all sure how this situation could arise, especially since the reporter said he was running the test using -OO. But, (1) I couldn’t get it to fail myself, and (2) the real bug in the situation in any case is issue 1538778 (closed). So I’m recommending that issue 19626 (closed) be closed as “works for me”, with issue 1538778 (closed) as a superseder.