Grails and Eclipse: Not So Groovy

In fact I struggled to get the Groovy-Eclipse-Plugin installed those days, and after finally having done that I found out that developing Grails applications in Eclipse was really painful for me.

But now with the first pre-M1 release of the next version of the Groovy-Eclipse plugin many things are better than ever. I finally got around using Grails and Eclipse for some of my projects at work but on the Mac at home I still prefer the combination of TextMate and Console.app. I’m looking forward to the final release which should improve UI performance in Eclipse, especially when using Code Assist.

Instruments now sports a core Java library which takes care of caching the states of system usage for the last time. This is included in an all new “InstrumentsService” which will provide the necessary methods to controllers.

At the moment this is all trimmed for using flot. There are no other “export options” for now.

I decided to make this publicly available in my Subversion repository. You can checkout the source tree of the Grails application here:

http://creatified.com/svn/creatified/Instruments/

The trunk includes the latest development version. Don’t forget the “grails upgrade”-command after checking out the source tree.

When you run the app, all you’ll find there for the moment is the following site which will be located unter http://localhost:8080/instruments/

You will then be able to see something like this:

for the CPU usage and this:

for the Memory usage

This will be all for now, be sure to check out your own version – I’ll also be testing this on the Google AppEngine soon.

But one aspect that’s often annoying me is the Eclipse integration. Like many other guys I’d like to use Eclipse because for me TextMate is no full-blown IDE. But the default integration into Eclipse is “incomplete” at best.

Here are some things I encountered with some possible solutions:

Unfortunately the generated Eclipse project files do not include a source link to the plugins directory of your project. Since Grails 1.1 this will be in your home-directory (at least on Unix-based systems) like ~/.grails/{GRAILS_VERSION}/projects/{PROJECT}/plugins, you will have to link this as a source folder when you’re using any plugins (which will often be the case)

The other thing is: if you’re including jars in the “lib” directory under your Grails project you will either add these jars manually to your build path or once again add this folder as a linked source folder.

I hope these issues will be addressed in a future release of Grails, hopefully as soon as 1.1.2 since these should not be too complicated to fix.

So what’s this about? In a few words: The PMT is a tool which will allow us to keep track of many quality-relevant metrics our company-website “produces” every day.

Problem

The sites we use for selling products are changed very often due to marketing campaigns. Not only do we have to change some images for that instead these are huge changes to the affected frontends. This of course results in the fact that many parts of our codebase are from older versions which may not be needed anymore.

At the moment we’re in the uncomfortable situation that we often don’t know how our pages really perform e.g. how many of the resources we deliver or keep are really necessary. We don’t know how many kilobytes (or even megabytes?) the average page weighs nor do we know where we could optimize easily.

Possible solution: the PMT

The PMT is designed as flexible as we think it can be. All it basically does is receive data from all different kinds of sources by offering a defined interface (a webservice). This will include information from

• the Jasmin servlet as mentioned by my colleague Nico Steiner (both in German)
• our Pustefix framework which renders the sites
• some other emerging projects we will use to make our websites even faster

In the end we will have information not only like “how many of the CSS selectors we deliver are really used by our pages?” but also “how does this change over time”. So we will be able to track quality with our predefined metrics and check back every week or so whether we have improved in those fields.

Architecture

I chose Grails to build this application – which I’m already pretty familiar with – but we’re also using some core Java libraries to add some features specific to the frameworks and technologies we use. Grails seemed like an ideal candidate for me because it already ships with the really powerful GORM abstraction layer. There are also plugins for other enterprise applications like the Quartz enterprise job scheduler. Grails makes it very easy to run all this stuff so I got done with the first steps pretty soon.

Frontend

At the moment I’m also drawing the first drafts of how the user interface of the PMT will look like. It will probably use Flot which I also mentioned several times.

Conclusion

For me this is a really great project. Not only in its size – it’s definitely the biggest project I’ve been working on until today. But its also a project in which I spend all my effort and time of work this time. I think it will help a lot to improve the overall technical quality of our web frontends.

I’ll keep you updated with all major changes and milestones to the PMT. So what do you think of our idea?

Here’s the screen:

Screen of Grails Instruments' cpu monitor

The cool thing is, the chart is rendered by flot (which I named before). At first I had planned to log the system metrics into a simple csv-file on the server. The logger would get the metrics every 500ms and then dump them to the csv. Not bad at all. It only turned very slow when I was then parsing the csv file in Groovy for the data I needed for drawing the chart – and that was a major performance killer. After caching about 10sec of system metrics the page would refresh after about 1-2sec which is just not fun…

So I went back and created a “specialized” cache class (for performance reasons: in Java) which will hold the metrics for the last 3 minutes (set the interval to 1000ms) in memory. This turned out to be much faster – for later versions there could be some “report functionality” which makes use of the csv-dumps (which will then be re-introduced ;-) ). For the moment the last 3minutes should be enough.

And the last thing I set up for today is the thing transmitting the cached results to the frontend (using json, what else?) and fills the data series for flot – resulting in something like the screen above.

There are plenty of things to be done yet. One thing is adding markings to the chart when certain events happen, … stay tuned, and as always: I’d appreciate your comments!

First things first: Fortunately Hyperic had placed an exception-statement into its GPL-licensed SIGAR – which allows to include the SIGAR binaries in – for example – Apache-licensed works. For me that’s really good news, because this means GInstruments could also be used in productive environments which I think often would not be GPL-compliant.

Second: One of the basic things I always wanted to have seems really close now. At the moment GInstruments only displays your current system usage – not too useful, because most times you’d be interested in the usage over time, say for the past 30 seconds or so. So that’s one of the core features I’ll be implementing before releasing anything to the public – sorry for the delay, I’m very busy in other projects right now, too.

For the next few weeks I’ll also be testing on various kinds of systems (Windows, Linux, …) and of course testing on Google AppEngine whether they support all the necessary features.

And for a great user experience I’ll also be working very hard on a prototype of the UI.

Some things that will probably make it into the first public release include:

• logging system usage and events to a set of CSV-files (configurable in later versions)
• usage graphs on the client-side using flot
• more? yeah, sure!

In the past weeks I’ve been thinking of, designing and developing on a new plugin for the growing framework Grails. It will be called GInstruments and it aims to provide some useful clues for Grails developers who want to know how their application is performing.

The plugin uses Hyperic’s (now SpringSource) SIGAR (System Information Gatherer And Reporter) to report vital system information such as CPU usage or memory comsumption overall or for the current process.

The first release (0.1) is only going to be a snapshot of my first ideas, hence I’ll not release it to the official Grails plugin repositories. It will show you some information about your system like name, version, vendor or architecture, the current CPU and memory consumption. Especially for your current process (which will be the relevant information to know). Take a look at this:

First draft of GInstruments 0.1

All these features are subject to change (I first wanted to know whether there could be a future for this). Possible features for the future may include

• logging of metrics (cpu, memory, swap, network, …)
• tracking of Grails events (requests, …)
• visualization of these two and more components over time to detect points of interest

I’m open to all kinds feature requests but please bear in mind that I’m only doing this in my spare time, after work, school whatsoever ;-)

At last let me point you to a link (for the non-Apple-developers) to how Apple’s Instruments-app looks like. I think it has some other cool possibilities.

At the moment I’m also waiting for some response from the guys from Hyperic. I really want to license the plugin under the terms of the Apache License (the license Grails uses and many other plugins) – SIGAR is GPL-only at the moment.

Do you think this could be a useful plugin for the Grails infrastructure? What’s your opinion.

The default method for creating thumbnails is the following

def imageTool = new ImageTool()

imageTool.load("/path/to/image.jpg")

imageTool.thumbnail(640)

This will load the file, create a new ImageTool instance and make a thumbnail out of it, where the max width/height will be 640px. However the resizing quality is very bad because by default it uses nearest neighbor for interpolation. You will encounter this especially when you’re rendering small thumbnails out of large images.

A possible solution

Because there is no detailed documentation about the ImageTools plugin (actually I didn’t find any) one will have to inspect the source code to find something very cool: ImageTools also provides a method called “thumbnailSpecial” which can use other interpolation and rendering algorithms and you can use it instead of the basic “thumbail()”. It’s signature is as follows

thumbnailSpecial(float maxWidth,
                 float maxHeight,
                 int interPolationType,
                 int renderingType)

maxWidth and maxWidth are pretty self-explaining, just set them to the same values (for example 640) to achieve the same dimensions as above.

interPolationType is one of

1: “bilinear interpolation” – which is at least better than nearest
2: “bicubic interpolation” – which is even better and
3: “bicubic2 interpolation” – which may even better

these are JAI names for them, don’t ask me about the details

renderingType is one of
1: “scale” – the default function JAI uses
2: “SubsampleAverage” – which will provide better results

Please keep in mind that all the “better results” require a lot more rendering time and are likely to consume more memory – but the results will be way smoother then.

So that may help some people out there which have experienced poor quality at this end.

For me there are only two further caveats when using ImageTools/JAI.

First: performance is very bad when using pure Java mode (which will be the default unless you provide a “native accelerator” for JAI). On my Mac this is no problem – as far as I know Apple provides a native accelerator in its own JDK on Mac OS X (and I never got messages like detailed on the plugin page) – yeah, using CoreGraphics/CoreImage, whatever… correct me if I’m wrong ;-) but I never got it to run on my production system (Ubuntu Server).

Second: JAI seems to have some other strange problems with JPEGs on Linux (I tried both OpenJDK and Sun’s JDK, always JAI in pure Java mode): some JPEGs will have inverted colors in the resulting thumbnail – couldn’t find out any details but I think something about the JPEG implementation is broken. This also affects saving JPEGs for me. It got strange exceptions when saving images as JPEGs causing that to fail – so I fell back to using PNGs – which is everything but cool in many cases :-(

As said these two problems only occurred on Linux (Ubuntu 9.04) – not on Mac OS X (couldn’t test on Windows) – but since most web apps I’m writing will be running on Linux some day this is a very annoying problem – and unfortunately I don’t see any progress on the ImageTools plugin or JAI. Anybody else got this kind of problem?

So I’m very tended to give something like good old ImageMagick a try – maybe using im4java or JMagick – or something else.