This timeline describes how Pylons evolved into its current form, and the influences from other web frameworks or projects.
It's not a WSGI or all-framework timeline but could be the basis of one.
The World Wide Web came into existence in 1990. Graphical web browsers soon followed, and Netscape incorporated Java and Javascript in 1995. The following year the Python core developers released Grail, a web browser written in Python that can execute Python applets. Grail was abandoned in 1999 because its paying sponsors felt it could never keep up with Netscape's speed or innovations enough to gain market share. But Grails lives on in spirit because most of the web and Internet modules in the Python standard library came from the Grails project.
Zope was released in 1998, a web framework written by Digital Creations for its consulting work. I think it was the first Python web framework. Previous developers wrote CGI scripts using Python's cgi module. Zope is more than a framework, it's a complete application environment, a kind of Emacs for web applications. Many Python developers rebelled against its monolithic "kitchen sink" approach, especially because it was almost a new programming language and also because the early docs were targeted toward content managers who didn't know Python rather than application developers who did. These problems were later addressed in Zope, but in the meantime the dissenters wrote several other web frameworks that were less ambitious and worked more like a typical Python library. We'll look at three of these now, Webware, Quixote, and Twisted. CherryPy comes from this era too, but we'll look at it a little later. These descriptions may seem long but we'll show how all these aspects influenced Pylons.
Webware was released in 2000 and follows Java's servlet model. URL components map to filesystem directories, and the final component is either a static file or a Python module containing a same-name subclass of Servlet. The class has a certain method the application server calls to get the content of the response. An Apache module mod_webkit delegates a site directory to a Webware daemon.
At the time there were no templating engines for Python (besides Zope's DTML and TAL), so Cheetah was written to fill this need in Webware. Like Webware, Cheetah was based on a Java library, Velocity. Cheetah's Webware origins can be seen in its special support for servlets: a Cheetah template can serve as a drop-in replacement for a Webware servlet. Any calculations the servlet had to do were done in a preamble at the top of the template. However, the Cheetah developers decided early on to make it usable in non-Webware applications too.
Webware is an umbrella term for several independent components. WebKit is the application server; UserKit does authentication; MiddleKit is an object-relational mapper (ORM) for certain SQL databases. The name "Webware" was also used by another company, so Webware's official name was changed to "Webware for Python". Nowadays a third product called WebWare has appeared, the rendering engine in the Safari web browser. This was forked from KDE's KHTML renderer and is now being re-merged under the WebWare name. Konqueror, wxPython, and GNOME all use it or soon will.
Quixote was created by Zope developers as a minimalistic framework that followed Zope's philosophy without the lock-in. It was also released in 2000. Unlike Webware it maps URL components not to physical directories but to attributes and subattributes of a Python object. Each attribute in the chain is a Directory instance, and the final one is a method which returns the result content. Any Directory may optionally have a ._q_traverse() method which totally takes control of the request and can override the subattribute lookup if it wishes. This provides a hook to add authentication, session initialization, preprocessing or postprocessing, or any other feature you might want to add. Access to the request, response, session, and user is via functions in the top-level quixote package. Session and user are do-nothing stubs; a production site would need homegrown or third-party libraries to use them. This reflects Quixote's minimalist philosophy of providing only the essentials, but frustrates users who want more out of the box. However, one of Quixote's strengths is the far-sightedness of its developers, who provided very flexible components that can be used in a wide variety of situations.
Note: the earliest versions of Quixote allowed any Python object to be a "directory" attribute, including modules. The switch to Directory classes was made to provide more flexibility: the standard dispatch mechanism is now built into Directory._q_traverse rather than in the publisher. This can be seen as precursor to WSGI because any application (with its own dispatch mechanism) can be linked to any Quixote-compliant server, with just a stripped-down Publisher object in between.
Twisted is an asynchronous framework, very different than any of the others. It's really a general Internet framework than a web framework per se, and its web features aren't as strong as some of the other frameworks. However, its proponents claim its asynchronous paradigm is much faster and less resource-hungry than other frameworks, which multithreaded or multiprocess. How true this is is a subject of debate, but what's not in debate is that Twisted's asynchronous nature requires a different programming style than other frameworks, which limits the ability of applications or frameworks to be compatible with it.
Mike Orr humorously described the situation that had arisen in his PyCon 2005 article:
The PyWebOff and WSGI
In the beginning there was Zope. Zope was a web application framework and the basis for several Content Management
Systems, but it had some discontents who dared to call it "monolithic" and "unpythonic". And behold, then there came
Webware, and it was Modular and didn't impose New Programming Languages on site developers, and there was much
rejoicing. But others rebelled at even Webware's Heavy-Handedness and arbitrary Conventions and wanted something even
Simpler, and a Ton of frameworks appeared: Quixote (which calls itself "lightweight Zope"), SkunkWEB ("Smell the
power!"), CherryPy ("fun to work with"), and some thirteen others. Meanwhile, Twisted had released its own
Whole Earth
Catalog of asynchronous Internet libraries including Nevow. Trying to find the forest through the trees, Ian Bicking
held a Shootout at PyCon 2004, comparing several frameworks against each other.
This year, Michelle Levesque went a step further and said we've forgotten about "Brian". "Brian" is the typical
non-techie developer who just wants to get a simple dynamic site up. The Python frameworks have now mushroomed to
forty [slide showing a montage of logos]. Experienced Pythoneers know that Zope is easy if it does what you want out
of the box, Quixote is good for sites that are big on calculations and small on eye candy, Twisted is good for
high-demand sites, etc.; but Brian doesn't know this. Brian sees forty apparently equal frameworks and chooses this:
[slide with the word "PHP"], or maybe this: [slide with the word "Java"]. Python is about having One Obvious Way To Do
It, but in the web framework world it's Ruby and Java that have a unified model, not Python. Quoting Moshe Zadka,
"You're not a Real Python Programmer until you've written your own web templating toolkit." But, Michelle said, there
are a lot of Brians in the world; they form by far the biggest potential "market".
Michelle's plea to developers is, "Stop writing kewl new frameworks! Help improve the top few frameworks so they can
become a best of breed. And put documentation on python.org telling Brian, "For heavyweight use A, for lightweight use
B, for performance use C, for XML use D, for no XML use E." Of course, this means the Python community must come to
consensus on which are the top frameworks. Some might think "when hell freezes over", but Michelle has a plan.
She issued herself a challenge to implement a typical Brian application (in this case, a book checkout system) in each
of seven frameworks, and compare the experiences (i.e., compare the grief).
The specific results of Michelle's study don't matter now. What matters is the paradigm shift it caused among Python web developers, why now tried to standardize and unify their frameworks. For that they had a new tool:
Ian Bicking followed Michelle's talk with a remarkably similar topic: "WSGI Middleware and WSGIKit (for Webware)"... .
WSGI (the Web Server Gateway Interface) is a proposed standard for Python (PEP 333). It's a protocol for web servers
to communicate with application frameworks....
WSGI allows each framework to need only a single virtual adapter, and the webservers can provide "best of breed"
adapters that plug into any WSGI-compliant framework. You can also plug in single-purpose "middleware" objects that
look like an application to the webserver, and like a server to the framework, or even chain middleware objects
together. This could allow alternate URL-parsing and Session modules to be plugged in and out, for instance,
eliminating the need for each framework to reinvent the wheel, and allowing applications to mix and match which
coding styles they prefer.
Over the next year there was a frenzy of activity making existing frameworks compatible with WSGI, and developing WSGI servers and middleware. Ian put his thinking cap on and came up with Paste, a way to sling WSGI tools together. Pylons' paster command and config file come from Paste. Paste was not a web framework but provided the foundation for a framework. Actually it included a couple frameworks, including a reimplementation of Webware called Wareweb. (I guess Erawbew was too difficult to pronounce.) Ian was one of the Webware developers in its early days.
2005 also saw the release of three new frameworks -- TurboGears, Django, and Pylons -- which ushered in the "megaframework" era. Megaframeworks combine the traditional services provided by earlier Python web frameworks with add-ons like template engines, SQL ORMs, Javascript libraries, and user/role paradigms to provide "one-stop shopping" for web developers. Of course we'd seen some of these add-ons in Zope and Webware, but Zope was still monolithic and Webware's add-ons were minimal. The megaframeworks provided most of Zope's capabilities in a modular manner that could compete directly with Rails.
2006 was mostly consumed with the Python web community absorbing Django and TurboGears, and getting the latter up to 1.0. Several developments occurred simultaneously:
- Django had been created by a newspaper company for in-house use. When it was developed in the mid-2000s the components
it needed didn't exist yet (e.g., SQL ORM), so Django developed its own integrated suite of tools. With its tight
integration, Rails-like functionality, and superior documentation and marketing thanks to its corporate backer, it
became Python's most popular framework and the one most often chosen by "Brian". However, its critics didn't like
its "not invented here" nature, which seemed to encourage the kind of lock-in many Python developers were trying to
get away from.
- TurboGears was also created for news publishing but its author Kevin Dangor took a different route. It started
development a little later than Django, so more third-party components were available or on the horizon. Kevin chose
several third-party components as "best of breed" in their niche, and built a web framework around them:
CherryPy for URL dispatching, its HTTP server, and configuration system; Kid for templating; SQLObject for database;
MochiKit/Scriptaculous for Javascript; etc. There were no suitable form libraries, admin interfaces, or authorization
libraries, so TG developed their own, and then started spinning them off so other frameworks could use them, a way to
give back to the Python community they had taken from.
- Both of these frameworks implemented the Model-View-Controller (MVC) pattern for separating content (view) from logic
(controller), and application-specific logic (model) from the framework (controller). They both had a "create project"
command that built a do-nothing application in a directory, preconfigured with stubs for the application developer to
fill in their controllers, templates, models, deployment configuration, documentation, and other stuff. Previous
frameworks required users to built their project directory manually or copy a demo, and did not include a built-in
MVC structure. These were some of the "mega" features of the megaframeworks.
- SQLAlchemy appeared as another SQL ORM, with a non-ORM layer that allowed lower-level access to the database.
However, it was still very young so it wasn't yet as stable or documented as SQLObject was.
- Ben Bangert started a new framework on top of Paste, called Pylons. Unlike the other frameworks which had to retrofit
WSGI in a kludgy manner, Pylons was WSGI at the core. It also took TurboGears' modularity one step further, because
CherryPy was a monolithic server and dispatcher, whereas Paste was merely glue to link together any server or
application (its own or third-party). Ben also ported the Routes dispatcher from Rails, making even URL mapping
autonomous and switchable. Routes installs as a middleware, and its only interaction with Pylons is through WSGI
environment variables. Pylons apps come with a default middleware stack, but the user is free to modify
middleware.py to disable some components and add others.
(Update from Ben: Pylons started in mid 2005. Originally he was running Myghty as a standalone framework under mod_python. Routes came first, sometime before August, and hooked into Myghty. Then Ben got into WSGI and Paste, and Pylons was born 2005-09-17. The pylons-discuss list was started two months later. Here's the second thread, where Ben summarized Pylons' status and goals.)
2006 was mostly consumed with the Python community absorbing Django and TurboGears. Pylons remained small but continued development.
Pylons switched from the Myghty template engine to Mako, which were both written by Mike Bayer who also wrote SQLAlchemy. Mako generally followed the Cheetah tradition but was much simpler, and included innovations that had appeared more recently in Kid and Buffet. Myghty was ported from Perl's HTML::Mason and had several framework-like features not appropriate to a Python template engine. The session and cache features were spun off into Beaker, written by Ben, which used MyghtyUtils, a spin-off of the core caching API used in Myghty. Beaker installs as middleware.
TurboGears' dependency on third-party components became a liability when its components started evolving out from under it. Kid became unmaintained and was succeeded by Genshi. SQLAlchemy won market share over SQLObject. Shortcomings were discovered in the Javascript libraries. And some users were clamoring for WSGI, Routes, and Cheetah. TG responded to the templating problem by creating Buffet, a common front-end for template engines, and spun it off as an automomous package. SQLAlchemy HOWTOs were written. CherryPy 3 finally supported WSGI and Routes as options, but TG surprised everybody by deciding to jump ship and build TurboGears 2 on top of Pylons. Yes, that's a framework on top of a framework on top of a WSGI library. Ian blogged that suddenly, for the first time in history, the number of Python web frameworks had gone down rather than up.
This wasn't strictly true because TurboGears kept its own identity and unique features, and the absolute number of frameworks continued to increase. But the number of frameworks that most developers were choosing had shrunk to six: Django, TurboGears, Pylons, CherryPy, Zope, and Twisted. And all of them supported WSGI.
TurboGears' cooperation with Pylons meant more developers supporting each framework. It also allowed a division of labor between the two. TurboGears wants to continue providing all megaframework services using a choice of third-party components. Unlike before the specific components would be expected to change over time as the state of the art advanced, but the scope of what TG provides would continue to expand. Pylons, in contrast, withdrew from some megaframework areas and started concentrating on a small core of essential components, a larger set of components that would be supported only in the documentation (the Official Docs or the unofficial Cookbook), and started sending those who wanted more to TurboGears. Meanwhile Pylons and TG try to choose common components for common needs. For instance, Pylons defaults to Mako but includes Genshi support, which TG leverages to default to Genshi. Pylons includes a form validator (FormEndode) and some basic form-tag functions (WebHelpers), while TG created its own library for form building (ToscaWidgets) and uses FormEncode for validation. It also spun off ToscaWidgets so Pylons users and other frameworks can use it. Pylons offers minimum SQL support in the form of a default SQLAlchemy configuration (which is optional), while TG offers implicit commit/rollback on top of it.
Meanwhile Paste spun off WebOb, a framework-neutral pair of high-level Request and Response objects. Pylons 0.9.7 (to be released in 2008) adopted these, and by extension TurboGears.
Repoze is a project to port Zope services to WSGI middleware, allowing Zope and non-Zope features to be combined into a WSGI website. For Pylons, it means the highly-regarded Plone content management system can be embedded in a Pylons application. repoze.who has been adopted by TurboGears as its authentication system, and there has been some interest in merging it with AuthKit for Pylons, or at least making the two compatible with each other.)
Pylons 0.9.7 jettisoned Buffet in favor of simple render_mako and render_genshi functions, which use native engine calls and can easily be replaced by the user for customization or to use a different template engine. As TurboGears 2 switched to this system too, Buffet lost its only known users and appears defunct.
|
|
