composer+npm+bower: Assets installation using composer-extra-assets

Short Version: a new composer plugin composer-extra-assets can help with installing bower & npm assets.


Long Version:

Composer is a great piece of software, it solves the problem of installing third party software for php quite nicely.

However a typical web application doesn't exist of php code only. You have got some javascript code that uses jQuery, some carousel, lightbox and whatever jQuery plugin. There might even be some server side running grunt task eg. for building modernizr.

But composer is for php code only, so it won't help us with installing those assets out of the box.

Oh, and there is one more thing that makes our live hard: it must be possible for packages to require assets indirectly, eg. a theme or a module (which is used by a project) must be able to require assets.

What are my options?

Ok, now that we know the requirements, how could this problem be solved?

1. Add assets to version control

While that is common practice for a number of projects it's not really a good idea:

  • Updating libraries is cumbersome
  • Different packages can't share a asset package

2. add asset package repository to root composer.json and install using composer

You can add any git repository in the root composer.json to use as composer package:

   "require": {
        "jquery/jquery": "1.11.1"
    },
    "repositories": [
        {
            "type": "package",
            "package": {
                "version": "1.11.1",
                "name": "jquery/jquery",
                "source": {
                    "url": "https://github.com/jquery/jquery.git",
                    "type": "git",
                    "reference": "1.11.1"
                }
            }
        }
    ]

That solution is quite simple, but:

  • Works for root composer.json only (you can't add repositories in other packages)
  • packages can't specify dependencies to other packages

3. use shim repositories and install them using composer

Shim repositories are built & stripped down versions of packages. You can try to find existing packages or even create your own and add them to packagist.org.

There is even a component-installer plugin and a number of commonly used packages on github.

That solution already works pretty well.

  • includes manual work to replicate the contents of bower & npm repositories to packagist (especially indirect dependencies get cumbersome)
  • new releases have to be pushed by someone
  • not-so-common packages have to be created by yourself
  • this won't help for npm packages

Luckily there are already existing tools - like Php has composer & packagist, there is npm for node and bower for web assets - which both have their own package repositories. So how can we use those repositories in a composer project?

4. create own satis/packagist server containing all bower & npm packages and install them using composer

Re-use the existing npm and bower repositories and build a composer compatible repository out of it's contents.

There is event a proof of concept implementation.

What I think is difficult for this to work properly:

  • That are too many packages that need to be updated regularly
  • Updates will appear with delay
  • This repository needs to be hosted somewhere
  • The repository needs to be added to the root composer.json although it's possibly only required by eg. a theme package

5. create composer plugin that converts all bower & npm packages on the fly to composer packages

To solve some of the issues of (4) this can also be implemented as composer plugin that queries the npm & bower repositories directly.

This also already has a working implementation. (Requires this composer PR to be accepted)

This plugin works pretty good in theory, but in practice is has a few issues:

  • Performance: many requests to github are required to read version information. Problem is that composer is internally a lot different than bower or npm. Caching is possible though
  • Larger dependency chains don't work reliable
  • it's basically a re-implementation of npm and bower clients, so it will always be "behind" (bugs & features)

6. use native bower & npm manually

Or, why not simply use the existing tools (npm, bower) instead of inventing something new?

Obviously this requires those tool being installed.

  • You need to create a bower.json and package.json manually containing all requirements of used packages. Not very convenient.
  • bower install and npm install needs to be called after every composer install, this can be automated by adding scripts to the root composer.json though:
    "scripts": {
        "post-install-cmd": [
            "bower install"
        ],
        "post-update-cmd": [
            "bower install"
        ]
    }

7. use native bower & npm with composer-extra-assets

And finally... drum roll... a solution I have come up with:

composer-extra-assets, a composer plugin that automatically calls bower & npm after package installation and gets required asset packages from all composer packages.

Npm packages will be installed in package folder, bower will be installed in root - with all dependencies merged.

Requirements:

npm needs to be in your path. (no need for bower, that's installed using npm)

Example Usage:

$ cat composer.json 
{
    "require": {
        "koala-framework/composer-extra-assets": "dev-master"
    },
    "extra": {
        "require-bower": {
            "jquery": "*"
        }
    }
}
 
$ composer install 
Loading composer repositories with package information
Installing dependencies (including require-dev)
  - Installing koala-framework/composer-extra-assets (dev-master 0d3324a)
    Cloning 0d3324aeafc33fe21de98aa1ff4dd834ebe32eca
 
Writing lock file
Generating autoload files
 
installing npm dependencies in 'vendor/koala-framework/composer-extra-assets'...
bower@1.3.10 node_modules/bower
├── is-root@1.0.0
├── junk@1.0.0
├── stringify-object@1.0.0
├── which@1.0.5
├── abbrev@1.0.5
├── chmodr@0.1.0
├── osenv@0.1.0
├── opn@1.0.0
├── archy@0.0.2
├── graceful-fs@3.0.2
├── rimraf@2.2.8
├── bower-logger@0.2.2
├── lru-cache@2.5.0
├── bower-endpoint-parser@0.2.2
├── lockfile@1.0.0
├── nopt@3.0.1
├── retry@0.6.1
├── tmp@0.0.23
├── q@1.0.1
├── semver@2.3.2
├── p-throttler@0.1.0 (q@0.9.7)
├── shell-quote@1.4.2 (array-filter@0.0.1, array-red
├── bower-json@0.4.0 (intersect@0.0.3, deep-extend@0
├── request-progress@0.3.1 (throttleit@0.0.2)
├── chalk@0.5.1 (escape-string-regexp@1.0.2, ansi-st
├── fstream@1.0.2 (inherits@2.0.1)
├── mkdirp@0.5.0 (minimist@0.0.8)
├── promptly@0.2.0 (read@1.0.5)
├── fstream-ignore@1.0.1 (inherits@2.0.1, minimatch@
├── glob@4.0.5 (once@1.3.0, inherits@2.0.1, minimatc
├── tar-fs@0.5.0 (mkdirp@0.3.5, pump@0.3.5, tar-stre
├── decompress-zip@0.0.8 (nopt@2.2.1, mkpath@0.1.0, 
├── request@2.42.0 (caseless@0.6.0, json-stringify-s0, node-uuid@1.4.1, qs@1.2.2, mime-types@1.0.2, bl@0
├── bower-registry-client@0.2.1 (graceful-fs@2.0.3, 
├── cardinal@0.4.4 (ansicolors@0.2.1, redeyed@0.4.4)
├── mout@0.10.0
├── bower-config@0.5.2 (osenv@0.0.3, graceful-fs@2.0
├── update-notifier@0.2.1 (string-length@1.0.0, semv
├── inquirer@0.7.1 (figures@1.3.3, mute-stream@0.0.4
├── handlebars@2.0.0 (optimist@0.3.7, uglify-js@2.3.
└── insight@0.4.3 (object-assign@1.0.0, async@0.9.0,
 
installing bower dependencies...
bower jquery#*                  cached git://github.
bower jquery#*                validate 2.1.1 against
bower jquery#*                 install jquery#2.1.1
 
jquery#2.1.1 vendor/bower_components/jquery

...and you have jquery installed in vendor/bower_components/jquery!

Issues

No lock. Bower has no feature like composer's lock file. To work around that always specify tags.

Multiple versions of the same package. If composer package A and package B both require jquery but each with a different version this is not handled correctly yet - you only get a warning currently.

Dependencies. You think having npm/node as a dependency is not a good thing? Well, that is true for production environment, but not for development environment. What we do is building locally and deploying the result (i.e. the vendor folder) to production.


If you have any other issues with this plugin, create an issue or fork me on github.

This entry was posted in Koala Framework, Web Development by Niko Sams. Bookmark the permalink.

3 Thoughts on "composer+npm+bower: Assets installation using composer-extra-assets"

Interesting post, but I would like to clarify some points about fxp/composer-asset-plugin (option 5). Indeed:

- the PR indicated for Composer is not required if the plugin is installed in "global" mode,
- Concerning the performance: for the install or update (without vendor directory), the plugin will retrieve only the indicated version. However, for an update, the plugin will retrieve the full list of versions, but it is the native mechanism of the VCS Repositories of Composer. Of course, all files (package infos and contents) are cached, which allows the plugin to be much faster with the next updates,
- "Larger dependency chains don't work reliable" ? Normally this works, so if you have a use case that does not work, I'd be happy to fix it,
- The plugin is not a re-implementation of NPM and Bower, but a simple conversion of information packages of bower.json and package.json in a Composer version (composer.json). It's like if you add a file composer.json to dependency and add the Composer VCS Repository in the composer.json project.

To be a little clearer: the plugin simply automates the search for the dependencies on the registries NPM and Bower, automatically adding the VCS Repositories of all dependencies (and sub dependencies) of NPM and Bower, and automatically add the Composer Package with the informations included in files and package.json bower.json.
François, thank your for your reply. Your plugin is indeed one of the best options to use - mine just gives you one more.

Problems I had with npm packages:
gist.github.com/nsams/d4a32938828e58874a9c
For now Francois plugin need to be installed globally, it's kinda drawback.
I'm currently evaluating Niko's plugin and I really like that it uses native bower.
Works fine for small project, haven't tested it for larger applications.

Leave a Reply

Your email address will not be published.