May 052012
 

First of all, let me put this in perspective. I’ve been using Linux on workstations and laptops since Red Hat Linux 6.2. I stayed with Red Hat all the way through Red Hat Linux 9. When Red Hat split the distribution into Red Hat Enterprise Linux and Fedora Core in 2003, I switched to Debian. I ran Debian for about six months, then switched to Gentoo Linux. In the summer of 2008, I switched to openSUSE Linux and I’ve been on openSUSE since then.

Every time one of the major community Linux distributions ships a new stable release, I try it out. So far, none of the Debian, Fedora, Ubuntu or Mint releases has come out significantly better than openSUSE, so I’ve stuck with it. And that remains true for Ubuntu 12.04 LTS “Precise Pangolin”. If that were the end of the story, I could close this blog post now. But it’s not.

If you’ve been following this blog and my Twitter stream and Github account, you’ll know that I’ve been collecting tools for computational journalism and packaging them as appliances. And I’m moving on towards a Platform as a Service. One of the requirements I’ve put on that is that the tools should be distribution-agnostic as much as possible. Up to now, everything has been on openSUSE because of the SUSE Studio appliance construction tools and to a lesser extent the openSUSE Build Service package repositories. But I’ve come to the point where I need to make things work on Fedora and Ubuntu.

So I’ve quad-booted my laptop (Windows, openSUSE, Fedora 16 and Ubuntu 12.04). And I’m trying to triple-boot my workstation with openSUSE, Fedora and Ubuntu. Which brings us to the first problem – openSUSE and Fedora installed cleanly on the workstation, but Ubuntu 12.04 didn’t. In particular, the Ubuntu desktop doesn’t even come up on a 1024×768 monitor!

I can understand Linux not coming up on a wireless card that’s relatively new. I can understand Linux having trouble with a touchpad or with audio. After all, the hardware makers design for Windows and Apple, not Linux desktops / laptops. But a 1024×768 monitor that’s run everything from Gentoo / WindowMaker to KDE 3.5 to KDE 4 to GNOME 2 and GNOME 3 and LXDE and Cinnamon on openSUSE? A 1024×768 monitor that runs Fedora 16 without any problems? That’s just plain wrong!

I did get the Ubuntu desktop working on the laptop, which is a much newer configuration. I’m not going to spend a great deal of time on how ugly the desktop actually is when it works. That’s been covered in numerous places and desktops are

  1. A matter of personal taste, and
  2. Customized to the user’s workflow.

But for someone who, like me, is used to the GNOME 2 desktop as delivered in previous versions of Ubuntu and Fedora, the openSUSE customization of GNOME 2 and the current clean implementations of GNOME 3 on openSUSE and Fedora, Ubuntu’s Unity desktop is jarring. And it’s really hard to figure out how to do things, where stuff is, and so on.

Moreover, the whole distribution is “pushy” – it’s hawking subscriptions to Ubuntu One cloud music, for example. The software installer has favorite apps, and so on. It’s like having a Kindle Fire or an iPad or visiting the Chrome Web Store or Google Play – the Ubuntu desktop is trying to sell you something every time you move your mouse. Ubuntu has turned the Linux desktop into just another media consumption device!

That’s two strikes – annoyances but not deal-breakers. But what I want to do with Fedora and Ubuntu is use them as hosts for virtual appliances, just like I use openSUSE and Windows / VirtualBox now.. In openSUSE and Fedora, I can go into the software installer and select a “pattern” and get everything I need to do that. If Ubuntu has that, it’s well hidden under the games and the productivity suites and the media apps. Sure, I can go find how to do that on Ubuntu on the web, but it seems to be going against the grain of the distribution. It only took me two minutes to find it on Fedora after almost four years of working daily on openSUSE!

I’m sure “Precise Pangolin” is a fine distribution “under the hood.” The previous long-term support version, 10.04, is an acknowledged workhorse in servers along with Debian, RHEL/CentOS/Scientific Linux and SLES. I have to test on it, and I’ll figure out how to be productive at it. But if Canonical can’t come up with a desktop built for Linux professionals like me, they’re going to lose us.

May 022012
 

As I’ve noted here, the Computational Journalism Server “wants to be a Platform-as-a-Service (PaaS) when it grows up.” In plotting the way forward to that goal, I’ve looked at three options:

  1. Remain on openSUSE / SUSE Studio and collect other open source tools to provide the additional services that would make the current server into a true PaaS.
  2. Start with the Cloud Foundry or one of its derivatives and add the computational journalism tools.
  3. Start with the Red Hat OpenShift Origin PaaS and add the computational journalism tools.

Remain on openSUSE

If I remain on openSUSE, as noted above, I’d need to collect more tools to provide additional services. Many of these deal with the underlying infrastructure. My target infrastructure is OpenStack Essex. That’s still the target. Moreover, the overall goal is still to provide an R-language tool set for dealing with large-scale computational journalism problems using library packages in the Comprehensive R Archive Network that implement parallel, cluster and grid computing.

When I started the original Data Journalism Developer Studio project, the Platform as a Service concept was in its infancy. It has matured rapidly, though. Cloud Foundry and Red Hat OpenShift are both about a year old, and several derivatives of Cloud Foundry have already appeared.

Cloud Foundry runs on Ubuntu Linux and OpenShift on Red Hat / Fedora. There isn’t an equivalent packaged solution for openSUSE. I’d have to build that for the Computational Journalism Server to be a true PaaS. And that seems to me a diversion from the mission.

Cloud Foundry

Cloud Foundry is an open source project from VMware. Derivative projects include AppFog for PHP projects, PaaS.io for Haskell, Stackato for Perl and Python and a community fork called CloudFreeStyle. Of these, neither AppFog nor PaaS.io are relevant, since there’s little PHP or Haskell in the current or planned use cases for the Computational Journalism Server. So the options are Cloud Foundry itself, Stackato or CloudFreeStyle.

I’ve ruled out the base Cloud Foundry for two main reasons.

  1. Most of the frameworks, services and tools provided by Cloud Foundry are totally unfamiliar to me. I know enough Ruby to do simple scripting and enough Java to call class libraries, but I know virtually nothing about Ruby on Rails, and I know absolutely nothing about Spring, Scala, RabbitMQ or Eclipse. I’d have a steep learning curve on tools that aren’t relevant to the core of the Computational Journalism Server – R, SQL and NoSQL databases, Hadoop and to a lesser extent Perl and Python.
  2. For an open source project with a year’s history under its belt, the documentation is, in a word, abysmal. In particular, the tasks I need to accomplish to make Computational Journalism Server into a PaaS – primarily adding R parallel programming capabilities and application packages – are totally undocumented.

I’m a member of the CloudFreeStyle project. I joined because I wanted to learn how to do what’s in Cloud Foundry and how to enhance it. Because it’s a source-level fork of Cloud Foundry, it would be easy to add functionality and ignore the components I don’t need, at least in the beginning. The “glue logic” to talk to applications and to the cloud infrastructure is already there and should “just work”. But, like its parent, there’s little documentation and I’d have to figure things out from the source.

Finally, there’s Stackato. Stackato is a very impressive product and ActiveState’s documentation, support and tool set is world-class. I’ve been a happy ActiveState Perl Development Kit user for years. If the Computational Journalism Server was a commercial product / business venture rather than an open source project, I’d go with Stackato. But the Computational Journalism Server isn’t there yet and may never be.

OpenShift Origin

OpenShift Origin, released on April 30, 2012, is an open source PaaS construction platform from Red Hat. I’ve spent about a day and a half browsing the documentation and I’m blown away by how comprehensive it is, especially for someone like me who wants to build a tool set from the ground up. The OpenShift Origin documentation is every bit as awesome as Cloud Foundry’s is abysmal.

The demos include a number of “LAMP stack oldies but goodies” – MediaWiki, WordPress, and Drupal. There’s also an OpenShift Origin LiveCD, based on Fedora 16, that turns any 64-bit Intel / AMD workstation, laptop or virtual machine into an OpenShift PaaS. With a few additional steps you can install OpenShift Origin permanently on a real or virtual machine.

The Way Forward

At the moment, I’m keeping three options open:

  1. Remain on openSUSE,
  2. CloudFreeStyle, and
  3. OpenShift Origin.

But I suspect the strength of the documentation will pull the project towards OpenShift Origin sooner rather than later. “Watch this space,” as the saying goes.

Apr 272012
 

Computational Journalism Server: SUSE Gallery Download Page

Computational Journalism Server: Github Project

Data Journalism Developer Studio Users Google Group


Computational Journalism Studio 1.0.0 is now available in the SUSE Studio Gallery at http://j.mp.compjournoserver. It’s not exactly a full Platform-as-a-Service yet, which is the eventual goal, but just about all the components I want in the appliance are there. I’ve put fairly detailed instructions on getting started on the SUSE Gallery download page and they’re duplicated on the Github project page. If you run into difficulties, please feel free to comment here, send me a tweet, post an issue on Github, make a comment on the appliance download page, or rant on the Google Group. No carrier pigeons, please – I’ll just cook them for dinner.

I’ve got fairly big plans for the evolution of this tool set. In rough calendar order but with no firm dates yet, they are:

  • Get the major use cases described in Parallel R working,
  • Get the R integration with the ATLAS libraries working by default,
  • Get the appliance working as an lxc Linux Containers guest,
  • Package the appliance so it can function as an OpenStack Essex compute node,
  • Port the workstation development / testing environment to Ubuntu 12.04 and Fedora 17, and
  • Port the server to CloudFreeStyle and OpenShift Platform-as-a-Service environments.