Graham Dumpleton joined us last year to bring New Relic to the Python world. He hails out of Sydney, Australia and has been programming for over 25 years. Graham is the author of mod-wsgi, the leading Python web hosting solution for Apache, and is a member of both the Python and Apache Software Foundations. He will be leading a talk on Web Server Bottlenecks and Performance Tuning at PyCon next week.
Why did you decide to join New Relic?
New Relic gave me the opportunity to work on a product which would help answer a question I received many times as the author of Apache/mod_wsgi: “How can I configure Apache/mod_wsgi for my website?”
My answer to this question was always that it depends on your specific web application. Plus, the real bottlenecks are generally going to be with your web application, database or call outs to external services, and not in your web server.
At the time there weren’t really any tools that could be used in production to give deep introspection into what was going on inside a Python web application. I thus saw working at New Relic as a great opportunity to address that lack of production application performance monitoring tools available to the Python web community.
I have been so excited about the opportunities here that I have even blogged previously about it.
What are you working on now?
As Python is the fifth and most recent language supported by New Relic, it is still to a degree playing catch up with the existing agents. My main focus is still bringing it up to feature parity with the existing agents.
There is still a lot to be done as far as implementing instrumentation for the most popular Python web frameworks and component libraries. We acknowledge that there are some frameworks which are notably missing in the list of what we do support. We hope to get to them soon.
As well as the web frameworks and component libraries, we are also interested in seeing what we can do in the way of special support for high-level products such as Plone. We also keep getting questions about whether we can support Twisted.Web. We aren’t committing to anything related to Twisted at this point, but we are investigating how such async systems can fit in with the existing product.
What does a typical day look like for you?
We work hard for our customers, whether they are on our Pro subscription level or using the free Lite product. As such, support requests are always the first order of the day.
Because the Python agent is still in its ramp up phase, we know we can learn a lot from what data is being reported by real world customer applications. I am, therefore, always keeping a close eye on new deployments and seeing what interesting things crop up. Often we are spotting issues and fixing them before customers even realize there is something wrong. Where appropriate, we will reach out to customers, provide suggestions, and see if they would like to test a new version of the agent where it should help to resolve an issue we identify.
Then it’s back to working on the Python agent itself, adding new features and additional instrumentation for third-party Python web frameworks and component libraries where we can see value in data we can extract from them.
I saw you last year at PyCon Australia? Where will you be speaking next?
This will be my first year talking at US PyCon. (Being held March 7 – 15 in Santa Clara, CA.) I have talked previously at PyCon’s in Australia, New Zealand and Singapore, but they are minnows compared to the expected attendance of over 1,500 at this year’s US PyCon.
While I said above that most real performance bottlenecks are not going to be in your web server, my PyCon talk will discuss how the web server can still be the issue. It will look at issues such as how a web server is architected can be a factor, especially in relation to the capacity the web server has for serving a large request throughput. The intention is to show that there is a lot more to the issue than just the speed at which a server can respond to requests.
See the PyCon website for more information.
Do you have any advice for Python developers?
Solutions for hosting Python web applications are still not ideal and more work is needed, especially as far as making deployment easier. We have come a long way though from where we were a few years ago, with a number of good solutions available such as Apache/mod_wsgi, gunicorn, uWSGI and Tornado.
Therefore, the challenge going forward is no longer with web hosting, but how you monitor what your web application running on top of it is doing. Production application performance monitoring which gives deep introspection into your Python web app is going to become more and more important. As tools such as New Relic grow into the Python web space and start providing more and more detail, answering the question of “how can I configure Apache/mod_wsgi for my website?” will become easier to answer. That and they will also help to highlight all the other bottlenecks in your web application and backend services.
I really encourage Python web developers to give New Relic a try. Make the most of the initial free Pro trial to get some deep insight into what your web application is doing. Even if you decide the full product isn’t for you, the Lite subscription level still provides a wealth of information you simply aren’t going to have now. So stop seeing your web app as a black box for which you know nothing about what is going on inside — dive in and find out, you might be quite amazed with what you find.