Comments and impressions of what is going on around in the world of Plone Open Source CMS, written by Myroslav Opyr. Blog entries include analysis of open source technologies current state as well as overview of some interesting open source services and techniques.
Decoupled Python Refcounters
Memory page that is not tackled (is not written into but only data are read from) is a good computer citizen. If such page was already paged out to Swap, in case of Memory pressure it is just discarded. The computer bus between CPU cache and RAM will not be occupied by writes out. In case of such page being shared by multiple processes, it makes huge saving of RAM and without copy-on-write operations (as page is not written into) the use of such pages has no CPU/bus overhead, and improves utilization of CPU cache memory.
In Python every PyObject has refcounter in its header. Every "touch" of python object makes inc/dec of that structure member. Each such operation makes CPU caches dirty, occupies bus to RAM with flushes, and makes no sense in sharing the page between processes, as single incref will make the page to do copy-on-write operation.
What if refcounters are decoupled from PyObjects into their own arenas? To shift hotpoint from sparse refcounter fields of PyObjecs into tightly packed counter arrays (arenas) with PyObject having only largely immutable index into arena. The approach would have slightly larger RAM footprint (due to refcounter arenas), slightly slower incref/decref operation (another couple of pointer resolutions), and small overhead during construction/destroy operations of PyObject (as refcounter arena free/busy bitmap should be maintained).
What would such approach bring to and what it'll break in Python?
Document Actions
Zope at Google AppEngine
When Google released GAE, a lot of folks in Zope world turned their sights to Google because Python, the heart of Zope was the language GAE was talking in then. Preliminary experiments showed that Zope and namely ZODB, persistence machinery behind Zope had little chances to survive at GAE. Folks succeeded into porting component architecture there, in running with non-ZODB storage, and in many other initiatives.
We set the goal high: Run Zope3 at GAE with ZODB and persistency. After 9 month of growth together with GAE, its SDK and service we've had basic zope3, worldcookery and grok' adder running at Google. We've learned a lot about GAE at its limits, were puzzled with a lot of obscure moments. Experienced unexpected and unreported kills. Rewrote launch and processing machinery several times.
There are many details to what was accomplished and how it was accomplished, you can see some from slides below. Zope3 is far from being production ready for GAE, but it works! We've even came with quick-n-dirty buildout that transforms Zope into GAE deliverable with single bin/buildout. Comments are welcome.
Topical discussion at http://chatterous.com/zope-gae.
Document Actions
Tendencies in Multi-core computing
They've produced healthy discussion upon the topic:
- Mr Rossum, tear down that GIL - request of Juergen Brendel
- It isn't Easy to Remove the GIL by Guido van Rossum
I was pointed to a good writeup upon Hardware/Software overview of tendencies in multi-core computing. It sheds some light upon progress made in the area touched by Guido in different languages/systems/platforms.
If we’re going to live in a world where CPUs are cheap and parallelism is the norm, then we need to think in those terms. If we need Perl/Python/Ruby/etc. programs to interact with parallelized libraries written in C/C++/Fortran, where’s the harm in spawning another process? Let the two halves of the program communicate over some IPC mechanism (sockets, or perhaps HTTP + REST). That model is well known, well tested, well-understood, widely deployed and has been shipping for decades. Plus, it is at least as language-agnostic as Parrot hopes to become. (+2 points if the solution uses JSON instead of XML.)
Fourth, there’s Patrick Logan, who rightly points out the issue simply isn’t about a multi-core future, but also a multi-node future. Some applications will run in parallel on a single machine, others will run across multiple nodes on a network, and still others will be a hybrid of both approaches. Running programs across a network of nodes is done today, with tools like MapReduce, Hadoop and their kin.
If you have a grid of dual-core machines today, and need to plan out how to best use the network of 64-core machines you will have a decade from now, here’s a very simple migration plan for you: run 32x as many processes on each node!
With that said, here is my recipe for taming the multi-core dust bunny:
- Determine what kind of parallelism makes sense for you: none, flyweight, fine-grained or coarse grained.
- Avoid troublesome low-level concurrency primitives wherever possible.
- Use tools like GHC’s Nested Data Parallelism for flyweight concurrency (one program, lots of data, spread out over multiple CPUs on a single machine).
- Use tools like GHC’s Software Transactional Memory for lightweight concurrency (many interoperating processes managing shared data on a single machine).
- Use tools like MapReduce and friends for heavyweight concurrency (work spread out across multiple cooperating processes, running on one or many machines).
Document Actions
Open/close events of KSS Opener
Besides defining ":opener-init" even that initializes opener, developer has ability to extend/override open/close events:
.term:opener-init{
evt-init-elementSelector: '.details';
}
.term:opener-open{
action-client: executeCommand;
executeCommand-commandName: replaceInnerHTML;
executeCommand-commandSelector: '#status';
executeCommand-html: 'expanded';
}
.term:opener-close{
action-client: executeCommand;
executeCommand-commandName: replaceInnerHTML;
executeCommand-commandSelector: '#status';
executeCommand-html: 'collapsed';
}
and HTML for the samle is:
<hr />
<div id="status">Current status</div>
<hr />
<div class="term">
History
<div class="details">
Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Vestibulum
iaculis eros eu purus. Integer accumsan leo id lorem viverra vulputate.
Donec feugiat nunc molestie massa nonummy pulvinar. Proin porta pede
sit amet lectus. Duis leo urna, tempor non, condimentum condimentum,
commodo non, libero. Integer feugiat, pede at.
</div>
</div>
The resulting behaviour on page is:
If you do not see a flash movie above, see original post.
The important thing is that on open and close you can have client-side or server-side actions. You can disable default expand/collapse behaviour, with preventDefault parameter:
.term:opener-open{
preventDefault: true;
}
Document Actions
Controlling Controller of KSS Opener
In the case you need your opener controller to be inserted into specific place, you can use pair of controllerPlacementSelector parameters:
dl:opener-init{
evt-init-elementSelector: '> dd';
evt-init-controllerPlacementSelector: '> dt';
}
if the above sample is applied to following HTML code:
<dl>
<dt>History</dt>
<dd>
Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Vestibulum
iaculis eros eu purus. Integer accumsan leo id lorem viverra vulputate.
Donec feugiat nunc molestie massa nonummy pulvinar. Proin porta pede
sit amet lectus. Duis leo urna, tempor non, condimentum condimentum,
commodo non, libero. Integer feugiat, pede at.
</dd>
</dl>
And the result is:
If you do not see a flash movie above, see original post.
The whole definition list has opener assigned. Controller (clickable handle) is inserted into <dt> element, not in root of list (that can confuse some browsers). This is why I avoided dl/dt/dd example in my introduction post.
Document Actions
Introduction of KSS Opener
After installation product offers following event:
opener-selector:opener-init {
evt-init-elementSelector: element-selector;
}
All elements that matches opener-selector will get controller that will offer expand/collapse functionality with mouse click.
Real-life example is:
.term:opener-init {
evt-init-elementSelector: '.details';
}
and having HTML code:
<div class="term">
History
<div class="details">
Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Vestibulum
iaculis eros eu purus. Integer accumsan leo id lorem viverra vulputate.
Donec feugiat nunc molestie massa nonummy pulvinar. Proin porta pede
sit amet lectus. Duis leo urna, tempor non, condimentum condimentum,
commodo non, libero. Integer feugiat, pede at.
</div>
</div>
will produce into something like:
If you do not see a flash movie above, see original post.
Document Actions
History of KSS Opener
In Plone1 the navigation tree, nested structure by its nature and semantics, was rendered into ugly list of <a> tags that had just "navlevelX" class that made it possible for designers to style by some extent.
Then we had proper <ul><li> nested structure that was huge step forward.
Then we had icons eliminated, that is another big step (not perfect yet, complex hover/active styles still makes life of designer hard).
And now we have Plone-3.0 knocking our doors. The most prominent feature everybody will be talking about is Ajax in Plone. Some will be mentioning KSS that made thing happen for complex CMS.
We had some efforts to address each of the items above, but none of them has matured enough to find its way into Plone code-base. And now we are going to address Ajax aspect of navigation tree.


