Python Will Literally Get Faster Soon
Python 3.11 will bear the fruits of CPython's multi-year effort to make Python a faster programming language.
Hey Guys,
This is Data Science Learning Premium,
I know, I know, Data Science Learning Center topics have been a little all over the place. This one is for the software engineers and programing enthusiasts.
So you know it, I know it, Python is incredibly popular because it's easy to learn, versatile, and has thousands of useful libraries for data science. But one thing it is not, is fast.
The next version of the standard Python interpreter, CPython, is expected in October, 2022. It will include significant performance improvements and support for running inside the browser.
If you’d like to support the channel and receive all the content, you can do so here.
Python Upgraded?
Let’s get into it:
Tracking the progress of software and code is really interesting. Like Shopify’s recent Engineering blog about investing in Ruby at Scale. Shopify has further it further recently by funding high-profile academics to focus their work towards Ruby and the needs of the Ruby community. Over the past year they have given nearly half a million dollars in gifts to influential researchers that we trust to make a significant impact on the Ruby community for the long term.
But this is not that story, Python is a way bigger community!
2022 and Python
For the first time in its history, Python has a real chance to hit first place in the TIOBE Index and become the most searched programming language in the world.
The other evidence of Python’s popularity is the results of the StackOverflow survey. 66% of developers mentioned that they enjoyed working with the language. 30% of engineers who have never before worked with Python replied that they would like to apply the language for their next project. Clearly Python is a fan favorite for engineers and coding enthusiasts alike.
Last week, the first Python language summit since 2019 took place in Salt Lake City. At the event, the language's development team announced various changes for the forthcoming version of the language, as well as its near future.
Core Python (CPython) developer Mark Shannon shared details about the project to make Python faster at the PyCon 2022 conference this week, where developers also showed off progress on the goal of running Python code in the browser.
Thanks Microsoft!
Last year, Microsoft funded a project for the Python Software Foundation (PSF), led by Python creator Guido van Rossum and Shannon, to make Python twice as fast as the current stable 3.10 series. The vision is to nudge Python towards the performance of C.
Although it is hard to count the exact number of Python developers operating in 2022, we can make a rough estimation. According to SlashData, in 2018, there were 8.2 million Python developers in the world. The forecast for the next three years is the infusion of 6 more million new developers into the Python community. So technically, we could be near 15 million global developers using Python in mid 2022.
Although the language was released 30 years ago, it is still a relevant tool for the development of applications. Python in 2022 is widely used by many IT giants, for example, by Google. 33% of company’s job advertisements mention Python which makes knowledge of the language the most required skill by Google.
The On-Ramp to Machine Learning
Performance, perhaps, hasn't been a top priority for Python as adoption has been fueled by machine learning and data science thanks to Tensor Flow, Numpy, Pandas and many more platforms, such as AWS's Boto3 SDK for Python. These platforms are downloaded tens of millions of times a month and used in environments that are often not constrained by hardware.
There are multiple editions of Python out there, including interpreters for the JVM and .NET CLR, as well as compilers, but the core implementation of the language is the CPython interpreter.
Apparently, Meta owned Instagram uses Python very heavily and runs on an internal version called Cinder, but this is specialized for the company's needs and isn't for general consumption. The new effort should be more widely applicable. Thanks IG!
How Much Faster? At least 25%
With Microsoft paying the wages of Van Rossum and Mark Shannon, along with some money from Bloomberg and others, the HotPy project is making progress. So far, the beta of CPython 3.11 is on average about 25 percent faster in benchmarking. That’s seriously impressive!
Popularity aside, Python is no speed demon. Optimized for easy syntax and ease of coding, Python is far slower than native C code; the latter has been demonstrated to run over a hundred times faster, depending on the task at hand. The much added boost could at least make it more useful.
Shannon is realistic about the project's ability to improve Python performance, but believes the improvements can extend Python's viable use to more virtual machines. 1.25 times faster is a lot.
"Python is widely acknowledged as slow. Whilst Python will never attain the performance of low-level languages like C, Fortran, or even Java, we would like it to be competitive with fast implementations of scripting languages, like V8 for Javascript or luajit for lua," he wrote last year in the Python Enhancement Proposal (PEP) 659.
The key approach detailed in PEP 659 is a "specializing, adaptive interpreter that specializes code aggressively, but over a very small region, and is able to adjust to mis-specialization rapidly and at low cost."
On the question of a just-in-time (JIT) compiler for Python's performance, Shannon suggested it was not a priority and would likely not arrive until Python 3.13, according to the Python Software Foundation's coverage of the event.
Led by Python creator Guido van Rossum and Shannon, work that began at least a year ago is close to fruition and could see a significant increase in Python performance before the end of the year. It’s looking like guys that 2022 will be the year Python got a bit faster.
Anaconda, the maker of the Anaconda Python distribution for data science, is backing the Pyston project, an implementation of Python that promises speed improvements over Python.
It is understood that work on the improved CPython interpreter is almost complete, though “major parts” such as dynamic specialization for loops and binary operations are yet to be completed.
Of course, Python can be sped up in various ways, including the use of alternate runtimes to wrap modules written in C/C++, as observed by a report on InfoWorld.
One of Anaconda's older efforts to speed up Python was the Numba project, an LLVM-based JIT compiler for CPython, which accelerates numerical Python functions running on the CPU or GPU, but can't optimize entire programs and doesn't address wider Python use cases.
Even a slow version of Python is still super accessible though as we have seen through its increased adoption in the 2016 to 2021 time frame. Why do so many people like Python today? The popularity of the language is guaranteed by its close to the natural English syntax and a good catalog of libraries and frameworks.
Suffice to say it’s reason to celebrate.
According to the Faster Python implementation plan, CPython 3.12 might gain a "simple JIT compiler for small regions" that compiles small regions of specialized code, while 3.13 would enhance the compiler to extend the regions for compilation. All in due time I guess.
If you want to follow this Newsletter on the app I recommend it.
The most popular machine-learning library in 2021 was Scikit-learn. Released in 2007 by David Cournapeau, it quickly gained popularity among scientists of different fields. Built on top of NumPy, the library allows to execution of such classification algorithms as random forests, support vector machines, k-means, gradient boosting, and others.
The Future of Python
Want to run Python code in a browser? Soon you might be able to according to Zdnet. At PyCon 2022, the annual conference for its community of "Pythonistas" – and the first in-person meet-up for Python contributors since 2019 due to the pandemic – developers revisited the idea of running Python code in the browser.
Even as technology is evolving, so is the code that builds and supports it. Corporate sponsorship and open-source movements really do add up.
By all regards, in Python 3.11, the programming language continues to improve functionality, exception handling, and usage.
Faster Processing Speed
Enhanced Error Messages
Use of Except* Keyword for Enabling Exception Handling
Introduction of the tomllib Library
Python 3.11’s the Newest Python Variant on the Block
In some ways it’s expected that Python 3.11 will radically change the essence of programming for beginners and advanced users alike. Do you agree with that?