Today I decided to chronicle my switch to Ruby and attempt document any revelations along the way. Hopefully this can be helpful in the future to others who would like to switch. First I will summarize all my thoughts leading to this point as I can recall them.
Background:
- I have been coding in Java since 1996 when there was only 1 Java book available.
- I consider myself a user interface designer/programmer.
- I have done a lot of webapp work over the past 5 years: JSP/Struts etc.
- I found long ago that designing a good UI was hard, but implementing it was much harder.
Personal Goals:
- Make great User Interfaces.
- Empower Domain Experts (who are not programmers).
- Side Quest: I'd love to be able to build a note taking app for myself in 5 minutes. I think it's possible.
Requirement MetaData
Early on I found that the kind of information I wanted to use in my interfaces was not typically available in the backend others were developing. The short example is TextFields. I would usually have a method "String getName()" that I needed to create a UI for, so I knew I needed a textfield. I could assume that the label on the screen would say 'Name'. However I did not know, nor could I assume, the max/min length of the text field since that was dictated by the database. There may also be some EJB or other code somewhere else that limits the size further, especially if there was a requirements change at some point. Continue this to things like 'first name', 'last name', 'middle initial' and so forth, and you have a lot of guesswork about info that is usually only in some Word requirements doc somewhere, or more likely spread over 2 years worth of emails and meeting minutes. Of course there
should be an up to date requirements doc, but this was quite often not the case. So I eventually became a 'backend programmer' in order to give my UI the information it needed to be better.
Summary: I need more "MetaData" than most backend programmers are used to giving me, and I don't want to spend all my time hunting down these requirements since the backend folks have already done this work once.Domain Experts and the "MetaDatabase"
As a UI guy I generally deal with Domain experts. People who really know their data, but don't know about Databases and certainly not code (Java or otherwise). All of my data comes from these Domain Experts: the domain model, the UI 'screens', the validation logic. Everything. Same as a requirements doc.
My solution to this over the years was to create a "MetaDatabase" which contained all of this domain data. This is a central location where I store all the data I have about the system I'm building: Objects, attributes, associations, sql types, java types, field lengths, etc. The idea is to put everything the Domain Expert tells me into this Meta database, and generate as much code as possible (preferably all) it. At the very least all of my requirements data is in one place. The idea is not new. It is practically stolen from the Pragmatic Programmers book (see footnote). You could also think of this as a requirements machine. To that end I did a lot of work with various persistence layers, some homegrown, and finally landed on Hibernate. More on that later.
Summary: I created a Meta database full of requirements data with the goal of generating all of my backend code and supplying my UI with the data it needed to be better.Model Driven Design
I find Model Driven Design quite interesting. The golden idea being that businesses could have define all business objects, rules, user interfaces, etc for an entire system in a language neutral formal (UML). Then vendors would build MDD Servers or somesuch that would run these applications. Freeing businesses from technology lock-in. In say 5 years when your server is yesterdays news, you just take your business app data and drop it into the newest leading edge MDD server and your app works. This MDD server could be written in anything, so whether it's Java, Ruby, ADA, Fortran, no one cares. I think this is interesting, and it coincides with the meta database idea. It's pretty much the same idea.
What the MDD guys offer is UML. Using Poseidon for UML I have been able to train domain experts to "model" their own systems using UML Class Diagrams. This works on a number of levels:
- It relieves me from the requirements gathering duties to a large extent.
- the domain experts learn a (relatively) simple tool and in the process learn a few basic coding ideas (like structure).
- they can see their system and play with it immediately.
From these UML diagrams I can generate their Persistence layer. That's a lot of work I don't have to do once the generation tools are written.
Summary: I can train domain experts to use UML to design their system, and effectively build their own persistence layer.Technologies
I currently use Hibernate and Struts for my webapps, and Maven (based on ANT) as my build system.
I love Hibernate. Before Hibernate I used Torque, which is similar but not as configurable. Hibernate (as many rubyists have pointed out) can be extremely complex to configure, but I have a giant book "Hibernate in Action" to help me out. The general belief is that its complexity is "necessary" given the complexity of the problem. With Hibernate I can create a set of configuration files (XML), and from that generate my entire persistence layer.
Hibernate XML → generates → Java classes, Sql Scripts.
The trick is this: Since all of my domain objects are defined in UML (XMI) I can also generate my Hibernate XML files. So...
Domain Expert → Visual UML Tool → XMI → Meta Database → Hibernate XML → Java Classes and Sql Scripts.
I now have my entire persistence layer generated from the Domain Expert's Visual tool. SO, Hibernate's complexity is largely irrelevant if I use code generation.
The only thing that actually deals with Hibernate are tools I wrote once.
Hand coding:I still have to hand code specialized queries, computed fields on Domain Objects, and other things like that.
This seems like a necessary step now, since that logic has to be written anyway.
Practically everyone I talk to now hates Struts. Too many config files, Form classes etc. Lots of duplication. I understand where you're coming from on this, however code generation solves a lot of my duplication woes so it doesn't bother me currently. I happen to like having a config file that lets me wire my web actions to various web classes. Granted, I do have a set of Struts utilities that I've developed over the last 3 years or so so perhaps I've masked the problems. Again, the point is that I can generate practically everything I need from my meta database so the complexity is handled for me.
Hand coding:I still have to hand code the business logic that the Struts actions perform.
Not Implemented Yet:I actually am not currently generating the Struts code either. It's a TODO item now.
I freaking love Maven. Ant is cool, but Maven is freaking sweet. In maven I define all my project info in a config file. My project name, cvs repo, dependencies and all that are in the config. This plays nicely into my central meta database idea. Maven also manages a central repository of versioned jars for you, which is a huge win since jar version management is a royal pain. More importantly, Maven generates tons of incredibly useful documentation for you for free. I'll enumerate the stuff later, but the 'project comprehension tool' aspect of Maven is where it's at.
Also, maven plugins allow me to build very flexible reusable tools for my projects. If I have an XML file from Poseidon in my project (and a few config properties) I can generate my persistence layer with one command: maven metadb:i.
That command generates my Java model classes, creates a Cloudscape database and configures hibernate for me, all in 1 step. Very sweet.
So Maven is really the glue for all of my various tools. More than that, it is what I use to massage any differences in the tools I use. The plugins I write define how tools fit into my overall architecture. From that perspective, I use Maven to build tool adapters for my framework. When I create a new project, I don't use Hibernate directly, I use maven plugins.
*So my development is in terms of my framework's plugins, not the tools I use.
Hand coding:I still have to hand code any specific build tasks that don't exist as plugins. I also have to occasionally duplicate tool settings when different tools need similar settings. (properties)
Caveat:Maven itself does have a reasonably steep learning curve. I think it is mostly because it assumes a certain project structure and each of the many plugins have their own goals and properties. Writing plugins is a tad ridiculous. You really have to know what's going on and it's no trivial task. Integrating Cloudscape and Hibernate as a Maven plugin to work on any arbitrary project with sufficient customizability was a royal pain. Debugging plugins sucks because they have to be built and deployed, no quick debugging. It takes a while.
Standing on the shoulders...
One big plus of all of this is that other people wrote the tools. The configuration files are very rich. Some say that's too complex, but I was able to learn a ton from the hibernate, struts and maven configs. These configs showed me all the meta data that these kinds of tools needed to operate, much like any API would. My task was simplified because I could look at their configs and map it or add to my Meta database. Over time my understanding of the metadata I needed to define my projects was augmented by many other projects as well.
So with one central meta database I can generate all of the configs to wire together the various tools. I love the flexibility. I have all the info I need, and I didn't have to write Hibernate, Struts or Maven.
I currently think this mapping will always be necessary so long as tools are developed by different people with different goals. I think this is what tools like Spring and Copland are supposed to address with AOP, but I have not tried them yet.
Caveat:I think the biggest problem currently is that my system is fairly complex. I think it's elegant, and it does a lot of work for me, but you have to know when to regenerate your persistence layer, and rebuild your database, etc. I have simplified this greatly with maven plugins that do everything for you. Running the cloudscape database is pretty sweet too since it means people can get started without having to configure a database, which is a big win since my goal is to Empower Domain Experts (who aren't programmers).
Minor Thoughts:tools are coupled to a philosophy. The tool builder had something in mind. It may or may not coincide with your philosophy.
Main Thoughts: (attempting to be language neutral)
- A centralized Meta database allows me to store business system meta data separate from implementation complexities.
- Learning all the rules you have to conform to for any tool is hard. Trying to conform to the rules many tools can be error prone and extremely difficult.
- Using existing tools is crucial so I can focus on my application/tool.
- Wiring together tools with custom adapters allows me to define my framework's semantics instead of trying to merge multiple tool's philosophies.
- A single way of thinking is easier to teach a newcomer.
- No one needs to know what tools the framework uses. (I believe this is just plain old OO design)
Questions I need to Answer:Understanding the concepts to begin with is a learning curve. Mapping terminology to the concepts is a learning curve. Is learning a new set of rules harder than adapting any moderate number of ideas to work together?
Things Missing:
- I currently can only generate a persistence layer from the metadatabase.
- I have not yet integrated Struts into my toolset with generation and plugins.
- I cannot capture business rules in metadata, they must be coded in Java in either Domain Object classes or Struts Actions.
- I still don't have the rich UI metadata that started me down this road years ago.
And that wraps up my current state of mind, before everything gets torn asunder at RubyConf 2004. Should be interesting.
Footnotes:
The Pragmatic ProgrammerThis book changed a lot of things for me. The book crystalized ideas I already had, and many I didn't. Most notably I think, it let me know I was not crazy, there was a better way. Nowadays, simply knowing a person liked that book gives me confidence in that person, because that means they want to be a better programmer.
