Composer Extra Assets 1.1

I'm happy to announce a new release of composer-extra-assets, the composer plugin for installing bower and npm dependencies.

New in this version is the integration of the nodejs-installer composer plugin by moufmouf. This plugin downloads and installs nodejs binaries locally (in vendor) if not found globally. The installed nodejs can also be used in other php scripts, the installer will put it into vendor/bin/node.

This update basically removes any dependencies from composer-extra-assets, it should be able to be used in any environment.

If you found a bug or have troubles with the plugin please file a bug on github.

This entry was posted in Koala Framework, Web Development by Niko Sams

Koala Framework 3.8

This koala might be resting... We aren't!

Koala Framework branch 3.8 is now officially considered as stable and ready to be used. That basically means that our demos use 3.8 now and it's the recommended branch to work with.

What changed since 3.7?

A lot.

Dependency installation:

  • We now install our dependencies using composer. Read this blog post why this is a great thing.
  • We use composer-extra-assets for additional npm and bower based dependencies. Read this blog post to get more about this very great (obviously as I wrote it) composer plugin

Build Step:

  • Every modern web software needs to be built nowadays - we make no exception. Read the obligatory blog post explaining the benefits.

Development Dependencies

While the the above mentioned changes are awesome, they came at a cost: additional dependencies.

But installing all of that is pretty easy, see the updated installation instructions.

Runtime Dependencies

All these new dependencies are easy to install on the local development environment, but the production server... This crappy hoster might not even have Php 5.3 - and doesn't even know what the hack nodejs is.

Don't worry - runtime dependencies didn't change. You can still run Koala Framework on the 8-year-old (yes, Php 5.2 came out 2006) Webserver. (BTW: nodejs released first 2009, npm 2011 and bower 2013)

What else?

  • The most important first: a new styled backend login!
  • twig template engine integrated
  • Rewritten, now modular, User Model Authentication
  • Frontend Code implenentation changed from ExtJS to jQuery (more to come)
  • Improved Frontend Performance with deferred JavaScript

See the full changelog for details.

This entry was posted in Koala Framework by Niko Sams

Source Maps Utilities Php Library

Public Announcement of a new php library for working with source maps: koala-framework/sourcemaps

Source maps help with debugging minified javascript or css files, see the html5rocks article for a short introduction.

This library originated from koala framework where we support source maps created by sass and uglifyjs.

Features

  • creating mappings
  • reading mappings
  • source map aware string replace for existing mapping
  • concat maps (optimized for best performance)

Use cases

Concat into a single file

A common pattern is the creating of a single javascript and css file out of many source files for better performance (as the browser has to make less requests).This library allows concating multiple source maps and keeps the mappings intact.

During development performance is key, so we implemented this concat as performant as possible: we don't parse and write the whole mapping information, instead we only work with the strings and put them together - with an offset correction injected - so the maps stay intact.

String replacements

When different versions of a file need to be generated (eg. translation or replacing a base path) it makes sense to cache the uglifyjs compiled file and do required string replacements on that.

Example usage

//read
$map = new Kwf_SourceMaps_SourceMap($mapContents, $minimiedFileContents);
$map->getMappings()
 
//create new map
$map = Kwf_SourceMaps_SourceMap::createEmptyMap($minimiedFileContents);
$map->addMapping(2, 3, 10, 12, 'foo.js'); //$generatedLine, $generatedColumn, $originalLine, $originalColumn, $originalSource, $originalName = null)
$map->getMapContents();
 
//merge two maps
$map1->concat($map2);
 
//perform string replacement
$map->stringReplace('foo', 'bar');

Found any issue, bug or have a question? Create an issue on github.

This entry was posted in Koala Framework, Web Development by Niko Sams

composer+npm+bower: Assets installation using composer-extra-assets

Short Version: a new composer plugin composer-extra-assets can help with installing bower & npm assets.


Long Version:

Composer is a great piece of software, it solves the problem of installing third party software for php quite nicely.

However a typical web application doesn't exist of php code only. You have got some javascript code that uses jQuery, some carousel, lightbox and whatever jQuery plugin. There might even be some server side running grunt task eg. for building modernizr.

But composer is for php code only, so it won't help us with installing those assets out of the box.

Oh, and there is one more thing that makes our live hard: it must be possible for packages to require assets indirectly, eg. a theme or a module (which is used by a project) must be able to require assets.

What are my options?

Ok, now that we know the requirements, how could this problem be solved?

1. Add assets to version control

While that is common practice for a number of projects it's not really a good idea:

  • Updating libraries is cumbersome
  • Different packages can't share a asset package

2. add asset package repository to root composer.json and install using composer

You can add any git repository in the root composer.json to use as composer package:

   "require": {
        "jquery/jquery": "1.11.1"
    },
    "repositories": [
        {
            "type": "package",
            "package": {
                "version": "1.11.1",
                "name": "jquery/jquery",
                "source": {
                    "url": "https://github.com/jquery/jquery.git",
                    "type": "git",
                    "reference": "1.11.1"
                }
            }
        }
    ]

That solution is quite simple, but:

  • Works for root composer.json only (you can't add repositories in other packages)
  • packages can't specify dependencies to other packages

3. use shim repositories and install them using composer

Shim repositories are built & stripped down versions of packages. You can try to find existing packages or even create your own and add them to packagist.org.

There is even a component-installer plugin and a number of commonly used packages on github.

That solution already works pretty well.

  • includes manual work to replicate the contents of bower & npm repositories to packagist (especially indirect dependencies get cumbersome)
  • new releases have to be pushed by someone
  • not-so-common packages have to be created by yourself
  • this won't help for npm packages

Luckily there are already existing tools - like Php has composer & packagist, there is npm for node and bower for web assets - which both have their own package repositories. So how can we use those repositories in a composer project?

4. create own satis/packagist server containing all bower & npm packages and install them using composer

Re-use the existing npm and bower repositories and build a composer compatible repository out of it's contents.

There is event a proof of concept implementation.

What I think is difficult for this to work properly:

  • That are too many packages that need to be updated regularly
  • Updates will appear with delay
  • This repository needs to be hosted somewhere
  • The repository needs to be added to the root composer.json although it's possibly only required by eg. a theme package

5. create composer plugin that converts all bower & npm packages on the fly to composer packages

To solve some of the issues of (4) this can also be implemented as composer plugin that queries the npm & bower repositories directly.

This also already has a working implementation. (Requires this composer PR to be accepted)

This plugin works pretty good in theory, but in practice is has a few issues:

  • Performance: many requests to github are required to read version information. Problem is that composer is internally a lot different than bower or npm. Caching is possible though
  • Larger dependency chains don't work reliable
  • it's basically a re-implementation of npm and bower clients, so it will always be "behind" (bugs & features)

6. use native bower & npm manually

Or, why not simply use the existing tools (npm, bower) instead of inventing something new?

Obviously this requires those tool being installed.

  • You need to create a bower.json and package.json manually containing all requirements of used packages. Not very convenient.
  • bower install and npm install needs to be called after every composer install, this can be automated by adding scripts to the root composer.json though:
    "scripts": {
        "post-install-cmd": [
            "bower install"
        ],
        "post-update-cmd": [
            "bower install"
        ]
    }

7. use native bower & npm with composer-extra-assets

And finally... drum roll... a solution I have come up with:

composer-extra-assets, a composer plugin that automatically calls bower & npm after package installation and gets required asset packages from all composer packages.

Npm packages will be installed in package folder, bower will be installed in root - with all dependencies merged.

Requirements:

npm needs to be in your path. (no need for bower, that's installed using npm)

Example Usage:

$ cat composer.json 
{
    "require": {
        "koala-framework/composer-extra-assets": "dev-master"
    },
    "extra": {
        "require-bower": {
            "jquery": "*"
        }
    }
}
 
$ composer install 
Loading composer repositories with package information
Installing dependencies (including require-dev)
  - Installing koala-framework/composer-extra-assets (dev-master 0d3324a)
    Cloning 0d3324aeafc33fe21de98aa1ff4dd834ebe32eca
 
Writing lock file
Generating autoload files
 
installing npm dependencies in 'vendor/koala-framework/composer-extra-assets'...
bower@1.3.10 node_modules/bower
├── is-root@1.0.0
├── junk@1.0.0
├── stringify-object@1.0.0
├── which@1.0.5
├── abbrev@1.0.5
├── chmodr@0.1.0
├── osenv@0.1.0
├── opn@1.0.0
├── archy@0.0.2
├── graceful-fs@3.0.2
├── rimraf@2.2.8
├── bower-logger@0.2.2
├── lru-cache@2.5.0
├── bower-endpoint-parser@0.2.2
├── lockfile@1.0.0
├── nopt@3.0.1
├── retry@0.6.1
├── tmp@0.0.23
├── q@1.0.1
├── semver@2.3.2
├── p-throttler@0.1.0 (q@0.9.7)
├── shell-quote@1.4.2 (array-filter@0.0.1, array-red
├── bower-json@0.4.0 (intersect@0.0.3, deep-extend@0
├── request-progress@0.3.1 (throttleit@0.0.2)
├── chalk@0.5.1 (escape-string-regexp@1.0.2, ansi-st
├── fstream@1.0.2 (inherits@2.0.1)
├── mkdirp@0.5.0 (minimist@0.0.8)
├── promptly@0.2.0 (read@1.0.5)
├── fstream-ignore@1.0.1 (inherits@2.0.1, minimatch@
├── glob@4.0.5 (once@1.3.0, inherits@2.0.1, minimatc
├── tar-fs@0.5.0 (mkdirp@0.3.5, pump@0.3.5, tar-stre
├── decompress-zip@0.0.8 (nopt@2.2.1, mkpath@0.1.0, 
├── request@2.42.0 (caseless@0.6.0, json-stringify-s0, node-uuid@1.4.1, qs@1.2.2, mime-types@1.0.2, bl@0
├── bower-registry-client@0.2.1 (graceful-fs@2.0.3, 
├── cardinal@0.4.4 (ansicolors@0.2.1, redeyed@0.4.4)
├── mout@0.10.0
├── bower-config@0.5.2 (osenv@0.0.3, graceful-fs@2.0
├── update-notifier@0.2.1 (string-length@1.0.0, semv
├── inquirer@0.7.1 (figures@1.3.3, mute-stream@0.0.4
├── handlebars@2.0.0 (optimist@0.3.7, uglify-js@2.3.
└── insight@0.4.3 (object-assign@1.0.0, async@0.9.0,
 
installing bower dependencies...
bower jquery#*                  cached git://github.
bower jquery#*                validate 2.1.1 against
bower jquery#*                 install jquery#2.1.1
 
jquery#2.1.1 vendor/bower_components/jquery

...and you have jquery installed in vendor/bower_components/jquery!

Issues

No lock. Bower has no feature like composer's lock file. To work around that always specify tags.

Multiple versions of the same package. If composer package A and package B both require jquery but each with a different version this is not handled correctly yet - you only get a warning currently.

Dependencies. You think having npm/node as a dependency is not a good thing? Well, that is true for production environment, but not for development environment. What we do is building locally and deploying the result (i.e. the vendor folder) to production.


If you have any other issues with this plugin, create an issue or fork me on github.

This entry was posted in Koala Framework, Web Development by Niko Sams

Composer powered Koala Framework Dependencies

"Composer is a application-level dependency manager for the PHP programming language that provides a standard format for managing dependencies of PHP software and required libraries. It was developed by Nils Adermann and Jordi Boggiano, who continue to manage the project. They began development in April 2011 and first released it on March 1, 2012. Composer is strongly inspired by Node.js's "npm" and Ruby's "bundler""
(Source: Wikipedia)

Koala Framework will use composer starting with version 3.8. This is a huge step that completely changes how deployment is done.

So why are we switching to composer?

  • it's the de-facto standard for modern php development
  • it allows easy usage of any required external package that can be found on packagist.org
  • it allows updating libraries without huge effort
  • it replaces the huge, monolitic library repository

Using Composer

First, you need to install composer, follow the official instructions - on Linux I would run:

curl -sS https://getcomposer.org/installer | php
sudo mv composer.phar /usr/local/bin/composer

Now that composer is installed (and in your $PATH) you can start using it in any composer enabled project by changing working directory into the project and executing:

composer install

This will download and install all requirements to run the project into the vendor folder. This includes koala-framework itself which can now be found on the official composer package repository packagist.org

As Koala Framework does nothing unusual or special with composer refer to the composer documentation for further usage.

Deployment

One question remains though: how to deploy the vendor folder?

1. You can run composer install on the production server which automatically downloads all dependencies.

2. You can copy the locally generated vendor folder to the production server.

I would recommend (2) as that has less dependencies on the production server, you can even use it without shell access at all.

Koala Framework contains a simple script that creates an archive containing all required files to run a project (that is project code + vendor + build), start the following script to generate deploy.tar.gz:

./vendor/koala-framework/scripts/build-deploy-package
This entry was posted in Koala Framework by Niko Sams

Redesigned Koala CMS Demo

Thanks to the amazing work of Daniel we have now a decent looking demo for our CMS. Needless to say it's fully responsive.

Check out the code on github.

Follow our installation instructions to try it out for yourself.

Screenshots

This entry was posted in Koala Framework by Niko Sams

Build Step

Koala Framework now has a build/compile/generate step.

This got merged into master recently and will be part of verison 3.8.

Why?

With the build step we can use external tools we don't want to depend on on production locally:

  • libsass (for much faster (10x) sass to css compilation
  • uglifyjs (for proper minification of javascript assets)
  • sourcemaps generation (for sass, minified javascript and concated css & js)

Additionally we don't have to rebuild assets on every clear-cache on production servers which caused a downtime of (depending on the server) several minutes.

And one important point is performance - we now can rely on the existence of build files on production and don't have to lazy generate caches.

What?

Currently part of the build are:

  • asset packages (compiled, minified, concated assets)
  • trl (parsed trl.xml files ready for usage in Kwf_Trl, for improved performance)
  • component settings (all static component settings)
  • component events (event listeners aggregated from all components)

Usage

When running a Koala Framework Application you now need the build subfolder. Run the following command to generate it:

php bootstrap.php build

Dependencies

For a successful build several libraries node packages are required: 

  • node-sass (for compiling scss)
  • uglifyjs (for minifying js)
  • grunt-modernizr (for creating modernizr build)
But don't worry - all are installed automatically using composer-extra-assets.

Deployment to production

The idea behind the build is that you can share the generated files across servers. You should not build on production server, instead the locally build files should be copied to the production server.

Build Server

Creating a build Server is relatively easy as you just have download all dependencies and start the build. No database or local config required.

This entry was posted in Koala Framework by Niko Sams

Koala Framework 3.7 Branched

Koala Framework 3.7 branch just got created. See the changelog.

Here a short overview what we added since 3.6:

Responsive Images

Image components now are fully responsive, meaning different resolutions of the image are automatically created and optimally loaded for the screen size.

The desktop (=contentWidth) version of the image is used as base, other sizes are created in 100px steps down to 0 and up to 2x contentWidth (to support displays with dpr up to 2).

Reworked Columns Component

The now third implementation of the columns component now supports defining multiple rows which should give more flexibility when editing content.

Performance

Like in every release we worked on improving performance. This time we concentrated on Frontend JavaScript performance:

  • A new benchmark box for profiling JS in frontend makes it possible to find bottlenecks
  • JavaScript initialization of components in frontend can now be deferred. For example installing a click handler on an element can be deferred for a few ms which makes the initial page load much faster

Ext4 MVC Controllers

Starting with Koala Framework you can now create Ext4 based web applications. This has been introduced in an earlier blog post.

This entry was posted in Koala Framework by Niko Sams

Kwf ExtJS 4 MVC Programming

As outlined in the Ext4 Update Plan we now almost completed Step 2.

That means Kwf master branch provides now the standards how ExtJS 4 MVC applications should be written.

This also includes various helper controllers that allow rapid development of complex applications.

Read the documentation here:

http://www.koala-framework.org/documentation/kwf_ext4_mvc

and take a look at the ported kwf-app-demo (work in progress):

https://github.com/vivid-planet/kwf-app-demo/tree/ext4

Example

Below you can see an example of what is possible with the Kwf Controllers and how easy they are to use. The example creates a binding between grid and form without having to write actual code - other than creating the controllers.

this.mainPanel = this.getView('Example').create();
this.mainPanel.bindStore(this.getExamplesStore());
 
this.formController = new Kwf.Ext4.Controller.Form({
    form: this.mainPanel.down('form')
});
this.gridController = new Kwf.Ext4.Controller.Grid({
    grid: this.mainPanel.down('grid'),
    autoLoad: true
});
 
//this connects the two controllers
new Kwf.Ext4.Controller.Binding.BindableToGrid({
    gridController: this.gridController,
    bindable: new Kwf.Ext4.Controller.Bindable.Form({
        formController: this.formController
    })
});

» see full example.


Want to try yourself?

You can start now writing new MVC applications using Koala Framework, it's one of the best documented parts of Koala Framework!

This entry was posted in Koala Framework by Niko Sams

Koala Framework Installation Documentation updated

The Koala Framework Documentation now contains an Installation section with guides how to install Koala.

Using that it should be possible for everyone to try out Koala.

» Read the installer docs here.

I also documented the maintenance scripts we have.

This entry was posted in Koala Framework by Niko Sams