Visit our site to listen to past episodes, support the show, join our community, and sign up for our mailing list.
Summary
In an attempt to improve the performance characteristics of the CPython implementation, Dino Viehland began work on a patch to allow for a pluggable interface to a JIT (Just In Time) compiler. His employer, Microsoft, decided to sponsor his efforts and the result is the Pyjion project. In this episode we spoke with Dino Viehland and Brett Cannon about the goals of the project, the progress they have made so far, and the issues they have encountered along the way. We also made an interesting detour to discuss the general state of performance in the Python ecosystem and why the GIL isn’t the bogeyman it’s made out to be.
Brief Introduction
- Hello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.
- Subscribe on iTunes, Stitcher, TuneIn or RSS
- Follow us on Twitter or Google+
- Give us feedback! Leave a review on iTunes, Tweet to us, send us an email or leave us a message on Google+
- Join our community! Visit discourse.pythonpodcast.com for your opportunity to find out about upcoming guests, suggest questions, and propose show ideas.
- I would like to thank everyone who has donated to the show. Your contributions help us make the show sustainable. For details on how to support the show you can visit our site at pythonpodcast.com
- Linode is sponsoring us this week. Check them out at linode.com/podcastinit and get a $20 credit to try out their fast and reliable Linux virtual servers for your next project
- I would also like to thank Hired, a job marketplace for developers and designers, for sponsoring this episode of Podcast.__init__. Use the link hired.com/podcastinit to double your signing bonus.
- Your hosts as usual are Tobias Macey and Chris Patti
- Open Data Science Conference, Boston MA May 21st - 22nd, use the discount code EP at registration for 20% off
- Today we are interviewing Brett Cannon and Dino Viehland about their work on Pyjion, a CPython extension that provides an API to allow for plugging a JIT compilation engine into the CPython runtime.
Use the promo code podcastinit20 to get a $20 credit when you sign up!
On Hired software engineers & designers can get 5+ interview requests in a week and each offer has salary and equity upfront. With full time and contract opportunities available, users can view the offers and accept or reject them before talking to any company. Work with over 2,500 companies from startups to large public companies hailing from 12 major tech hubs in North America and Europe. Hired is totally free for users and If you get a job you’ll get a $2,000 “thank you” bonus. If you use our special link to signup, then that bonus will double to $4,000 when you accept a job. If you’re not looking for a job but know someone who is, you can refer them to Hired and get a $1,337 bonus when they accept a job.
Interview with Brett Cannon and Dino Viehland
- Introductions
- How did you get introduced to Python? - Chris
- What was the inspiration for the Pyjion project and what are its goals? - Tobias
- The FAQ mentions that Pyjion could easily be made cross platform, but this being a Microsoft project it was bootstrapped on Windows. Have any of the discrete tasks required to get Pyjion running under OSX or Linux been laid out even in outline form? - Chris
- Given that this is a Microsoft backed project it makes sense that the first JIT engine to be implemented is for the CoreCLR. What would an alternative implementation provide and in what ways can a JIT framework be tuned for particular workloads? - Tobias
- What kinds of use cases and problem domains that were previously impractical will be enabled by this? - Tobias
- Does Microsoft’s recent acquisition of Xamarin and the Mono project change things for the Pyjion project at all? - Chris
- What are the challenges associated with your work on Pyjion? Are there certain aspects of the Python language and the CPython implementation that make the work more difficult than it might be otherwise? - Tobias
- When I think of Microsoft and programming languages I generally think of C++ and C#. Did your team have to go through an approval process in order to utilize Python, and further to open source your work on Pyjion? - Chris
- How does Pyjion hook into the CPython runtime and what kinds of primitives does it expose to JIT engines for them to be able to work with? - Tobias
- Would an entire project be run through the JIT engine during runtime or is it possible to target a subset of the code being executed? - Tobias
- In what ways can a JIT compiler implementation be purpose-built for a given workload and how would someone go about creating one? - Tobias
- Could a JIT plugin be designed with different trade-offs, like no C API compatibility, but that worked around the GIL to provide real concurrency in Python? - Chris
- One of the most notable benefits of having a JIT implementation for the CPython runtime is the fact that modules with C extensions can be used, such as NumPy. Does that pose any difficulties in the compilation methods used for optimizing the Python portion of the code? - Tobias
- What kinds of performance improvements have you seen in your experimentation? - Tobias
- Which release of Python do you hope to have Pyjion incorporated into? - Tobias
- Has any thought been given to making Python a first class citizen in Visual Studio Code? - Chris
- What areas of the project could use some help from our listeners? - Chris
Keep In Touch
Picks
- Tobias
- Chris
- Brett
- Dino
The intro and outro music is from Requiem for a Fish The Freak Fandango Orchestra / CC BY-SA
Visit our site to listen to past episodes, support the show, join our community, and sign up for our mailing list.Summary In an attempt to improve the performance characteristics of the CPython implementation, Dino Viehland began work on a patch to allow for a pluggable interface to a JIT (Just In Time) compiler. His employer, Microsoft, decided to sponsor his efforts and the result is the Pyjion project. In this episode we spoke with Dino Viehland and Brett Cannon about the goals of the project, the progress they have made so far, and the issues they have encountered along the way. We also made an interesting detour to discuss the general state of performance in the Python ecosystem and why the GIL isn't the bogeyman it's made out to be.Brief IntroductionHello and welcome to Podcast.__init__, the podcast about Python and the people who make it great.Subscribe on iTunes, Stitcher, TuneIn or RSSFollow us on Twitter or Google+Give us feedback! Leave a review on iTunes, Tweet to us, send us an email or leave us a message on Google+Join our community! Visit discourse.pythonpodcast.com for your opportunity to find out about upcoming guests, suggest questions, and propose show ideas.I would like to thank everyone who has donated to the show. Your contributions help us make the show sustainable. For details on how to support the show you can visit our site at pythonpodcast.comLinode is sponsoring us this week. Check them out at linode.com/podcastinit and get a $20 credit to try out their fast and reliable Linux virtual servers for your next projectI would also like to thank Hired, a job marketplace for developers and designers, for sponsoring this episode of Podcast.__init__. Use the link hired.com/podcastinit to double your signing bonus.Your hosts as usual are Tobias Macey and Chris PattiOpen Data Science Conference, Boston MA May 21st - 22nd, use the discount code EP at registration for 20% offToday we are interviewing Brett Cannon and Dino Viehland about their work on Pyjion, a CPython extension that provides an API to allow for plugging a JIT compilation engine into the CPython runtime. Use the promo code podcastinit20 to get a $20 credit when you sign up! On Hired software engineers designers can get 5+ interview requests in a week and each offer has salary and equity upfront. With full time and contract opportunities available, users can view the offers and accept or reject them before talking to any company. Work with over 2,500 companies from startups to large public companies hailing from 12 major tech hubs in North America and Europe. Hired is totally free for users and If you get a job you’ll get a $2,000 “thank you” bonus. If you use our special link to signup, then that bonus will double to $4,000 when you accept a job. If you’re not looking for a job but know someone who is, you can refer them to Hired and get a $1,337 bonus when they accept a job.Interview with Brett Cannon and Dino ViehlandIntroductionsHow did you get introduced to Python? - ChrisWhat was the inspiration for the Pyjion project and what are its goals? - TobiasThe FAQ mentions that Pyjion could easily be made cross platform, but this being a Microsoft project it was bootstrapped on Windows. Have any of the discrete tasks required to get Pyjion running under OSX or Linux been laid out even in outline form? - ChrisGiven that this is a Microsoft backed project it makes sense that the first JIT engine to be implemented is for the CoreCLR. What would an alternative implementation provide and in what ways can a JIT framework be tuned for particular workloads? - TobiasWhat kinds of use cases and problem domains that were previously impractical will be enabled by this? - TobiasDoes Microsoft's recent acquisition of Xamarin and the Mono project change things for the Pyjion project at all? - ChrisWhat are the challenges associated with your work on Pyjion? Are there certain aspects of the Python language and the CPython implementation that make the work