Python Performance: Why 'if not list' is 2x Faster Than Using len()

isn't the expected behaviour exactly identical on any object that has len defined:

"By default, an object is considered true unless its class defines either a bool() method that returns False or a len() method that returns zero, when called with the object."

ps: well your objection is I guess that we cant know in advance if that said object has len defined such as being a collection so this question does not really apply to your post I guess.

type safe python is a myth

Sure, but type hints provide a ton of value in documenting for your users what the code expects. I use type hints everywhere, and it's fantastic! Yes, there's no guarantee that the types are correct, but with static analysis and the assumption that your users want their code to work correctly, there's a very high chance that the types are correct.

That said, I lie about types all the time. For example, if my function accepts a class instance as an argument, the intention is that the code accept any class that implements the same methods as the one I've defined in the parameter list, and you don't necessarily have to pass an instance of that class in (or one of its sub-classes). But I feel like putting something reasonable in there makes a lot more sense than nothing, and I can clarify in the docstring that I really just need something that looks like that object. One of these days I'll get around to switching that to Protocol classes to reduce type errors.

That said, I don't type hint everything. A lot of private methods and private functions don't have types, because they're usually short and aren't used outside the class/file anyway, so what's the point?

It's all about trade-offs. Here are a few reasons why one might care about performance in their Python code:

1. Performance is often more tied to the code than to the interpreter - an O(n³) algorithm in blazing fast C won't necessarily perform any better than an O(nlogn) algorithm in Python.
2. Just because this particular Python code isn't particularly performance constrained doesn't mean you're okay with it taking twice as long.
3. Rewriting a large code base can be very expensive and error-prone. Converting small, very performance-sensitive parts of the code to a compiled language while keeping the bulk of the business logic in Python is often a much better value proposition.

These are also performance benefits one can get essentially for free with linter rules.

Anecdotally: in my final year of university I took a computational physics class. Many of my classmates wrote their simulations in C or C++. I would rotate between Matlab, Octave and Python. During one of our labs where we wrote particle simulations, I wrote and ran Octave and Python simulations in the time it took my classmates to write their C/C++ versions, and the two fastest simulations in the class were my Octave and Python ones, respectively. (The professor's own sim came in third place). The overhead my classmates had dealing with poorly optimised code that caused constant cache misses was far greater than the interpreter overhead in my code (though at the time I don't think I could have explained why their code was so slow compared to mine).

Honestly most people use Python because it has fantastic libraries. They optimize it because the language is middling, but the libraries are gorgeous

ETA: This might double post because my Internet sucks right now, will fix when I have a chance

This is my two cents as someone in the industry.

Because, while you don't want to nitpick on each instruction cycle, sometimes the code runs millions of times and each microsecond adds up.

Keep in mind that people use this kind of things for work, serving real world customers who are doing their work.

Yes, the language itself is not optimal even by design, but its easy to work with, so they are making it worth a while. There's no shortage of people who can work with it. It is easy to develop and maintain stuff with it, cutting development cost. Yes, we're talking real businesses with real resource constraints.

You may want to beneficiate from little performance boost even though you mostly don't need it and still need python's advantages. Being interested in performance isnt always looking for the very best performance there is out of any language, it can also be using little tips to go a tiny bit faster when you can.

Yes, Python is the wrong choice if performance is your top priority.

But here's another perspective: why leave easy performance wins on the table? Especially if the cost is simpler code that works as you probably wanted anyway with both None and []?

Python is great if you want a really fast development cycle, because the code is generally quite simple and it's "fast enough." Any wins for "fast enough" is appreciated, because it delays me needing to actually look into little performance issues. It's pretty easy for me to write a simple regex to fix this cose (s/if len\((\w+)\) == 0:/if not \1:/), and my codebase will be slightly faster. That's awesome! I could even write up a quick pylint or ruff rule to catch these cases for developers going forward (if there isn't one already).

If I'm actively tweaking things in my Python code to get a little better performance, you're right, I should probably just use something else (writing a native module is probably a better use of time). But the author isn't arguing that you should do that, just that, in this case, if not foo is preferred over if len(foo) == 0 for technical reasons, and I'll add that it makes a ton of sense for readability reasons as well.

Here are some other simple wins:

• [] and {} instead of list() and dict() - the former copy constants, whereas the latter actually constructs things; oh, and you save a few chars
• use list comprehensions instead of regular loops - list comprehensions seem to be faster due to not needing to call append (and less code)
• use built-ins when you can - they're often implemented in native code

I consider each of those cleaner Python code anyway, because they're less code, just as explicit, and use built-in language features instead of reinventing the wheel.

It comes down to the question "Is YOUR C++ code faster than Python?" (and of course the reverse).

I've built a SCADA from scratch and performance requirements are low to begin with, seeing as it's all network bound and real world objects take time to react, but I'm finding everything is very timely.

A colleague used SQLAlchemy for a similar task and got abysmal performance. No wonder, it's constantly querying the DB for single results.

Alternatively, why wait twice as long for your python code to execute as you have to?

I’ve worked on a library that’s Python because the users of said library are used to Python.

The original version of the project made heavy use of numpy, so the actual performance sensitive code was effectively C++ and fourtran, which is what numpy is under the hood.

We eventually replaced the performance sensitive part of the code with Rust (and still some fourtran because BLAS) which ended up being about 10x faster.

The outermost layer of code is still Python though.

I have the same question. I prefer other languages. I use G'MIC for image processing over Python and C++.

Telemetry

if isinstance(mylist, list) and not mylist

Problem solved.

Or if not mylist # check if list is empty

Python likes giving lists.

else: # not my list, it is ourlist

Strongly disagree that not x implies to programmers that x is a bool.

That's why we use type-hinting at my company:

 undefined

    
def do_work(foo: list | None):
    if not foo:
        return
    ...

  

Boom, self-documenting, faster, and very simple.

len(foo) == 0 also doesn't imply it's a list, it could be a dict or any other type that implements the __len__. That matters a lot in most cases, so I highly recommend using type hints instead of relying on assumptions like len(foo) == 0 is probably a list operation.

In my experience, if you didn't write the function that creates the list, there's a solid chance it could be None too, and if you try to check the length of None, you get an error. This is also why returning None when a function fails is bad practice IMO, but that doesn't seem to stop my coworkers.

I really dislike using boolean operators on anything that is not a boolean. I recently made an esception to my rule and got punished... Yeah it is skill issue on my part that I tried to check that a variable equal to 0 was not None using "if variable...". But many programming rules are there to avoid bugs caused by this kind of inattention.

if you're worried about readability you can leave a comment.

I think there's a good chance of that:

• -2x instead of ~2x - a human is unlikely to make that mistake
• no space here: ==0 - there's a space every other time it's done, including the screenshot
• the numbers are wrong - the screenshot has different data than the image
• why are there three bars? A naive approach would have two.

Looks like it. It's a complete fever dream graph. I really don't get how someone can use an image like that. Personally I don't really like AI art anyways, but I could somewhat understand it as a sort of "filler" image to make your article a bit more interesting. But a graph that is supposed to convey actual information? No idea why anyone would AI gen that without checking

yeah I got angry just looking at it

My ad blocker has blocked all pictures on this article, so I can't say. 😄

except for c++ it was just to avoid a single function call, not extra indirection. also on modern compilers size() will get inlined and ultimate instructions generated by the compiler will likely be the same

This is honestly the worst version regarding readability. Don't rely on implicit coercion, people.

That's worse. IMO, solve this problem with two things:

• type hint mylist as list | None or just list
• use if not mylist:

The first documents intent and gives you static analysis tools some context to check for type consistency/compatibility, and the second shows that None vs empty isn't an important distinction here.

That woulb be 0.5x. −2x implies negative duration, which makes no sense. Neither does the layout of anything else in the image.

I think it was supposed to be a ~, since they use that in a paragraph below the image.

Maybe they mean up to?

Oh, there are plenty of other terrible ways:

 undefined

    
for _ in mylist:
    break
else:
    # whatever you'd do if mylist was empty

  

 undefined

    
if not any(True for _ in mylist):

  

 undefined

    
try:
    def do_raise(): raise ValueError

    _ = [do_raise() for _ in mylist]
except ValueError:
    pass
else:
    # whatever you'd do i mylist was empty

  

I could probably come up with a few others as well.

Please note that none of these handles the TypeError if mylist is None.

You'd need to explicitly check for None if using the len() construct as well, so this doesn't change the point of the article.

You get the length of the stringified length of the array. And a warning if those are enabled.

That's absolutely cursed.

I really liked unless in perl; especially as I get older !length or something makes that bang really easy to miss. I use !(length) or something instead to visually set it aside. unless made this much more visually clear.

Empty sequences being false goes back a lot further than perl, it was already a thing in the first lisp (in fact the empty list was the cannonical false).

There's probably an "import * from relativity" in there somewhere.

why comment if you don't even want to read the article? python lists are not linked lists, they're contiguous with a smart growth strategy.

No, len is a constant time operation, at least in most cases I believe.

So… it has to iterate over the whole empty list is what you’re saying? like once for every of the zero items in the list?

Then please be less lazy next time.