Davide Moro: Introducing substancek. A Kotti project

Let me introduce substancek, a Kotti (http://kotti.pylonsproject.org) project.

What it is substancek?

substancek is:

• Kotti (web application framework) + decoupled admin interface

It is only an additional layer upon the following opinionated stack:

• kotti_backend, a plugin that turns Kotti to a private admin interface (backend). See https://github.com/Kotti/kotti_backend
• Kotti, web development framework based on Pylons/Pyramid with workflows support, content types, security). See http://kotti.pylonsproject.org
• Pylons/Pyramid, one of the most flexible Python web frameworks. See http://www.pylonsproject.org
• SQLAlchemy, the Python SQL Toolkit and Object Relational Mapper. See http://www.sqlalchemy.org
• Python. See https://www.python.org

with the following motto:

"""(even) better development experience and complete frontend freedom"""

and introduces (or better promotes) the concept of private admin area (backend) decoupled from the public side (frontend) of your web applications built with Kotti.

In other words it is a set of technologies addressed under the substancek brand that let you extend Kotti in order to use it just as a private backend administration area for your application data.

So you are still using plain Kotti with an additional package (at least kotti_backend depending on what you need).

If you want to know more I've discussed here benefits and why frontend decoupled from the backend pattern. See http://davidemoro.blogspot.it/2015/07/kotti-cms-successful-story-part-1.html

substancek name

Tribute to:

• substanced (http://www.substanced.net). Carefully designed, well documented Python web development framework based on Pyramid/ZODB. It shares the concept of management views decoupled from the retail views for each published resource
• kotti (http://kotti.pylonsproject.org). The Kotti framework

When substancek is for you

Any project of any size (from micro to XXL) involving content management that needs:

• RDBMS. Kotti is an opinionated framework opposed to plain Pyramid. The persistence layer is managed by SQLAlchemy (http://www.sqlalchemy.org)
• user friendly and production ready admin/editing interface for content producers so they can immediately start adding content to your web application with copy/cut & paste, clean urls, etc
• rapid development. Pyramid and Kotti are easy to learn and let you become productive quickly and speed up your development. Even more with the decoupled admin interface.
• stability. Pyramid and Kotti exist since many years and they are solid rock solutions production ready
• frontend freedom. The admin interface comes for free while it is completely up to you add what your application needs (retail views). Since they are completely two different applications there are no css/js conflicts and you can integrate your preferred frontend tool chain without get influenced by how the admin interface is built.
• flexibility. Pyramid won't hold you back when your application is small and won't get in your way when your application becomes large.
  Even if you start small, using Pyramid you anticipate what customers will need later avoiding expensive rewrites and this can make the difference. In other words this also means successful projects in the long term. Same for Kotti. See What makes Pyramid unique.

So if you project needs (or in future iterations) one or more:

• complex security policies
• workflows
• hierarchical data support
• or even intranet/extranet like collaboration areas

you might consider even more substancek (kotti_backend + Kotti + Pyramid + SQLAlchemy).

For example:

• very small applications. For example a just one view public json endpoint for published news-like resources consumed by a third party app with an admin interface for editing contents
• heavy Javascript based applications with modern frontend tools (eg: SPA + REST) with a decoupled admin interface
• content management solutions
• blog
• ecommerce
• intranets
• large CMS-ish applications

Note well: if you don't need workflows don't be scared because there is no overkill. You can use a one state workflow or no workflow at all for example. No hierarchical data? Use not nestable resources and so on. If it comes out later that you need them it will be quite easy converting your code.

Alternatives

You can use plain Kotti, without the substancek's kotti_backend setup. Or if you prefer noSQL try the excellent substanced (substanced + Pyramid + ZODB). Both solutions are lightweight, tested, well documented and easy to learn. Alternatively if you need really a minimal and unopinionated solution you might use plain Pyramid.

Do you need something more? You might consider to use Plone (https://plone.org) as a framework.

Anyway the good news is that Python is plenty of good options.

substancek architecture details

As already told you the private admin area (backend) and the rest of the application (frontend) are two complete different applications with different settings, different views and shared authentication.

Assuming you are going to use PasteDeploy to run your application, let's consider the following configuration files setup:

• backend-dev.ini, configures the private admin interface based on Kotti thanks to the kotti_backend plugin
• frontend-dev.ini, configures your application you are developing (a full CMS frontend implementation or a microapp with just one retail view).
• development.ini (optional), mount the backend-dev.ini and frontend-dev.ini applications in the same process (/admin for the private admin interface and / for your application). Alternatively you can run the frontend and backend using two processes waiting for requests on different ports and play with rewrite rules.

backend-dev.ini

[app:kotti]
use = egg:kotti

...
pyramid.includes =
    pyramid_debugtoolbar
    pyramid_tm
kotti_backend.views.override_root_view


kotti.configurators = kotti_tinymce.kotti_configure
kotti_backend.kotti_configure

kotti.use_workflow = kotti_backend:workflows/simple_backend.zcml

kotti_backend.goto_frontend = 1

This is a normal Kotti setup with:

• enabled the (optional) root view override for our admin interface. In other words the default root view will be @@contents instead of the standard Kotti's hello page (see pyramid.includes)
• added kotti_backend to kotti.configurators
• override the Kotti's default workflow with the one provided by kotti_backend (kotti.use_workflow). Playing with the additional pview permission you can decide your resources visibility on the frontend. See the workflow definition here https://github.com/Kotti/kotti_backend/blob/master/kotti_backend/workflows/simple_backend.zcml
• enable a "Goto frontend" link (kotti_backend.goto_frontend) for easy switching from admin interface and frontend 

See more options on the kotti_backend's README file:

• https://github.com/Kotti/kotti_backend

frontend-dev.ini

[app:main]
use = egg:Kotti
...

kotti.use_workflow = kotti_backend:workflows/simple_backend.zcml

kotti.configurators =
    your_package.kotti_configure

kotti.base_includes =
    kotti
    kotti.views

On the frontend configuration file we share the same workflow in use on the admin interface (kotti.use_workflow).

One of the most important configuration is the kotti.base_includes override: here we decide what will be loaded on our application. We omit all the Kotti views loaded by default in the standard setup and we load what we want to include where:

• kotti, loads the kotti "core"
• kotti.views (optional), load some view discriminators and utils defined by Kotti if you need them
• your_plugin.your_includes, load your includes registering the views needed by your application

The kotti.configurators typically auto includes your package and tell what should be included in your application (pyramid.includes). See the Kotti documentation for more info.

In other words:

"what is not loaded, it doesn't exist"

so the final result is that there is nothing exposed on the frontend except what you decide to load and extreme control. You can register just only one view application or configure a complex setup for a CMS-like application: it's up to you registering only the views your application needs and no more. This way you can use completely different frontend frameworks, different versions of Javascript libraries, you have no css/js conflicts and no need to hide unneeded things and you decide which resources will be published on the frontend.

See also another advance usage pattern "Using Kotti as a library"http://kotti.readthedocs.org/en/latest/developing/advanced/as-a-library.html

development.ini

# See http://pythonpaste.org/deploy/#paste-composite-factory
[composite:main]
use = egg:Paste#urlmap
/ = config:frontend-dev.ini
/admin = config:backend-dev.ini

[server:main]
use = egg:waitress#main
host = 127.0.0.1
port = 5000

The (optional) development.ini shows how to configure a composite application with different mount points. You can change /admin with /cms or /whateveryouwant depending on your needs.

Examples

You can checkout the https://github.com/substancek/substancek_cms_theme package if you want to see in action a (quite complex) example.

I'm going to provide more and simpler examples (eg: a pretend micro application), see the roadmap.

What are the substancek related packages

Here you can see the whole substancek ecosystem:

• kotti_backend, generic package that turns Kotti to a private admin area. This is the keystone for every substancek like project
• CMS-like applications
• substancek_cms (roadmap), a Kotti CMS distribution with an alternative frontend theme based on SASS, html/templates minification and assets optimization based on Yeoman tools. It will be based on the kotti_project experiment available here https://github.com/davidemoro/kotti_project
• substancek_cms_theme, an example public side CMS implementation based on Kotti built with SASS and html/templates minification
• kotti_actions, link actions backend implementation for header, footer and main navigation links
• kotti_boxes, portlets and box backend implementation
• kotti_es, elastic search support (to be refactored)
• common utilities
• pyramid_html_minifier, generic Pyramid package that introduces Chameleon templates minification with no overhead. Required by substancek_cms_theme.
• build_commands, generic Python package that defines a set of initialization commands on install (eg: python setup.py npm/bower/gulp) for frontend stuff. Required by substancek_cms_theme.

Who's using substancek technologies

MIP - graduate school of business

The MIP (Politecnico di Milano graduate school of business - www.mip.polimi.it/en) uses substancek technology inside for the private admin interface. This approach is so flexible that let you use Kotti as a private admin content management area and even implement your public views using other frameworks or non Python languages (for example PHP+Symfony2).

See:

• http://davidemoro.blogspot.com/2015/07/kotti-cms-successful-story-part-1.html
• http://davidemoro.blogspot.com/2015/07/kotti-cms-frontend-decoupled-backend-part-2.html

substancek_cms_theme

This is a work in progress opinionated CMS frontend implementation that reuses existing Kotti templates and logics.

The look and feel is the same you get with a standard Kotti installation but it shows how to integrate and distribute a Python package integrated with a Yeoman setup (http://yeoman.io) that provides:

• production vs development setup. In production mode will be used assets and templates from the dist
• Chameleon templates minification with no overhead thanks to pyramid_html_minifier when you are in "production" mode
• SASS
• image/assets optimization
• build commands for npm/bower/gulp initialization thanks to the generic package build_commands

Once installed you'll see the admin interface visiting http://localhost:5000/cms.
See the code here: https://github.com/substancek/substancek_cms_theme

Next steps

If you want to contribute there is a lot to do:

• contributions to the Kotti core (for example @@contents pagination, REST, etc and new ideas)
• creation of the substancek_cms package that puts together the substancek_cms_theme default theme plus the most useful third party Kotti plugins like news, events, etc with decoupled admin interface and Vagrant/Ansible automated setup based on kotti_project for easy installation/evaluation
• creation of simple pretend packages, add more examples (microapp, blog)
• a toolbar available on the frontend just of editor users for improved usability (the same shown in the MIP case history) implemented with a Pyramid tween
• more work on substancek_cms_theme, implement advanced features shown in the MIP case history like link managers and portlets and refactor the Yeoman folder removing what is not strictly needed
• scaffolding, help people create new projects with decoupled admin interface using Kotti as a framework
• substancek dedicated github page

Contributions, feedback or pings like "hey, I'm going to use Pyramid/Kotti for the my next project" will be very appreciated!

Documentation

• http://kotti.readthedocs.org/en/latest
• http://pylons.readthedocs.org/en/latest

All Kotti posts published by @davidemoro

• http://davidemoro.blogspot.it/search/label/kotti

Twitter links

• https://twitter.com/pylons
• https://twitter.com/KottiCMS
• https://twitter.com/davidemoro