New gem: rails_framework_diff
It started on twitter:
It’s amazing how hard it is to find problems when caused by developers incorrectly modifying framework-generated files. - Chad Fowler
i’ve always wanted to write a rake task to diff framework-generated files against their generated default for that very reason. - Brian Doll
I would install that gem/plugin if you released it. - Chad Fowler
I had been looking forward to writing another gem ever since reading the Building a gem with BDD article in the initial issue of The Rubyist. In that article, Jamie van Dyke mentioned a gem called Mr. Bones for creating gem skeletons. It differs from other similar solutions in that it’s just a generator and does not insert itself as a dependency on your gems.
The problem is simple enough. The rails command builds a project skeleton which includes many key files. If those files are modified incorrectly, lots of badness can happen. Since they are automatically included in all rails projects though, they can look innocuous when auditing or reviewing a rails project. When they are the culpret, you’ve likely spent hours and hours reviewing eveything else first.
Now, with rails_framework_diff you can quickly see a diff against the rails framework-generated defaults and those corresponding files in your project. Nice and easy, nice and quick.
I should also note that without github, not only would the process of releasing a new gem into the world take a whole lot longer, it certainly wouldn’t be as fun. This gem was conceived, developed and released on a major gem server in under two hours. Awesome.
Deep Send: How to wrangle with large object graphs and come out alive
In a perfect world, all of our objects are small, orderly and well defined. They encapsulate data that is semantically obvious and easy to work with. And then… in an obvious homage to Jack Handey, we attack that world with large, complex and ugly object graphs, mostly the result of monolithic integration technologies like SOAP.
Sure, with gems like soap4r, the pain of dealing with SOAP is somewhat minimized, but at the end of the day, you’re still going to have to look deep into the eyes of a huge object graph, and get at the delicate properties you so covet.
What if we were able to reference deeply nested properties from within the bowels of these large object graphs, as easily as we could with neat and orderly objects? How nice would it be not to have to repeat yourself while asking the object graph to cough up two properties buried thirty chained-methods deep?
Deep Send
This is so simple it feels like cheating. The deep_send method could be attached somewhere high up in the SOAP hierarchy if you wish, but I like having it available everywhere.
As you can see, deep_send works like the regular send method, but takes chained method calls as arguments. Let’s see how this might work in a sample SOAP client:
In our example, we have a method appropriately called get_nasty_graph. Let’s assume this invokes a SOAP service method and returns all of it’s enormous goodness. We could write a million methods in this class to pull out a few related elements to make a reasonable API. The problem with this is that you’re always wrong. There is always a reason to pull out three of these properties and two of those, and your API just won’t accommodate. If an object needs to use two different API methods to get all that data, it may result in two SOAP calls, which is surely no good. Using that magic get method now allows us to construct our own method to pull out exactly what we want. Nothing more, nothing less.
Encapsulate, you must!
If we stopped here, we’d end up painting ourselves in a corner. Requiring the users of your SOAP client to know about the deep recesses of your nasty object graph is just bad manners. It’s also a leaky abstraction, so stop it!
Now that we’ve defined a few constants, the clients of our SOAP client can use our magic get method, without knowing how horrific it is to get at those name properties.
Gotcha! Order thy calls!
You may have noticed that rescue NoMethodError bit in our magic get method. Choosing to implement that method in this way is not without it’s faults (ha!). If you passed in a method chain that resulted in a NoMethodError being raised, you’ll need to decide what to do. In this example, I’ve chosen to swallow it and return what I got so far. This means that the order of your method chains is significant, if there is a higher likelihood that some of them will occasionally go missing. Isn’t SOAP marshaling awesome? You may decide, of course, to rescue the error, call the authorities and raise the threat level to red instead. Good luck with that.
Things I learned at RailsConf 2008
There were about 2k people at RailsConf 2008, and like most of them, I thought of posting my review of each of the presentations I saw and heard about. Thankfully, the masses have spoken, and there are lots of reviews for you to read elsewhere. Also, if you want to check out some of the presentations themselves, O’Reilly is hosting many of them online.
What I’d like to do instead is to document the things I learned at RailsConf, that stretch across presentations. The first thing that I noticed were the themes that seemed to find their way into almost every session, lighting talk and hallway conversation. There was a lot of focus on testing, deployment, automation, performance benchmarking, design (anti-)patterns and the social aspects of distributed source control management. These are real topics, discussed in great detail by people with real experience to an impassioned audience. It was impossible not to get caught up in the energy and passion, two things that may seem at odds with the typical software conference. Thankfully, RailsConf is anything but typical.
JRuby
WIth all this talk of the various Ruby implementations, my colleagues and myself have been interested to learn more about JRuby. I attended a few JRuby events (the “hackfest” on Thursday was great!) and got a much better understanding of JRuby and what sort of costs/benefits there may be in using it.
The Good:
- I was able to get a small Rails app running on JRuby in a few minutes. It just works.
- The availability of Java types from Ruby is astoundingly simple
- Simple .war and .ear deployments can make for easy integration into a Java environment
- Using JDBC drivers and JNDI is pretty sweet
- For some applications, switching between Ruby and JRuby can be seamless. Thoughtworks does dual-development on some projects (for reasons not explained)
The “bad”:
- Exception handling in Java/JRuby projects is tricky. Exceptions wrap either a Java or JRuby exception and you must unwrap and inspect upon rescuing them.
- It is not uncommon to get Java stack traces in JRuby projects. It would be nice if these were entirely hidden regardless of root cause.
- Windows paths for file URIs is currently broken (I reported this bug during the hackfest, JRUBY-2591)
- The ease of integration for Java libraries can turn an otherwise clean and simple Rails project into a horrific beast of amalgamation. While this is an implementation detail and not a problem of the project, I can see a Java team not wanting to follow “The Rails Way”, and taking away any and all advantages of using Ruby and Rails by overusing Java.
- JRuby is essentially on par with Ruby 1.9 in most performance benchmarks. The JRuby team will be focusing on making Rails perform better on JRuby, but for the time being it does not bring significant performance improvements.
Rails Hosting
With Engine Yard as a major sponsor of the conference, and many startups opting for hosted solutions, this was a well worn topic, with a few key insights.
- The golden ratio is 3 to 4 mongrels (or instances of your app) per CPU core. Don’t debate this, it’s written in copper.
- Cache, cache, cache. Use the same file system across your app instances so they can share page cache. Engine Yard and others spoke highly of Red Hat’s GFS system.
- (Almost) nobody needs database sharding. It is a special use case, with lots of considerations. If you don’t understand it, you probably don’t need it.
Building Software with a dynamic language
Neil Ford had a “Design Patterns” in Ruby presentation, and Obie Fernandez had an interesting talk that showcased awful Rails code, which started to identify some anti-patterns.
- The Gang of Four design patterns are all structural, in that they use objects to construct them. In Ruby, many of these patterns are solved within the standard library, and in an elegant and simple manner.
- Dynamic languages allow you to do things that you can’t conceive of when working in static languages. The mixin/unmix portion of his presentation was good on this point.
- An interesting code smell for refactoring: If code looks visually similar, and differs only in whitespace, it’s likely that you should refactor.
- An obvious notion that should go without saying: Know your tools. Github’s rss feeds on projects (like Rails) makes for an easy time keeping up with the direction of the project, on each commit.
- OOP paradigms are still critical in Ruby and Rails projects. A programmer without OOP experience will likely make for a terrible Rails programmer. Thanks Obie for reiterating this point. Don’t get caught up in the “blog in 20 minutes” vision of what Rails is.
- Beware of Helpers that try to do too much. They can feel anti-OO and are usually the first to break down.
Scalability and Performance Monitoring
- Scalability, on large and high-volume systems, is a never-ending battle of finding and reducing bottlenecks. There is no single solution or architecture. Understand how your app performs, and rely only on data, not hunches.
- Someone else’s benchmarks are as relevant to your application as pickles to ice cream. Sure there is an edge case where they relate, but that’s just gross.
- It’s essentially impossible to benchmark a framework. However, it may be relevant to get lots of specific performance data from lots of Rails applications to determine some relationships. I asked Koz about this during his presentation and he has already talked to New Relic about allowing their customers to share the framework-relevant performance data with the Rails core team.
- Rely solely on real-world performance data in production when making scalability or performance-related adjustments to your system. All else will be either time poorly spent, or will end up hurting you.
- Trending is nice, but the unexpected hockey stick graphs can still happen. Sometimes you just hit a wall (cache sizes, max IO nodes, etc.) that you never saw comming.
- Understand the ratio of capacity to application instances. If you are not getting close to linear growth, find out why.
- New Relic offers some great real-time performance monitoring. Their monitor points are extensible, so you can wrap your own stuff (SOAP calls, etc.) and have it monitored automatically
Testing
There were several talks on testing, and the topic came up in many others. It’s nice to see a community so focused on testing from both TDD and BDD perspectives.
- Josh Susser’s obscure quote that summed up his presentation well: “De gustibus non est disputandum” or, There is no accounting for taste. Josh Susser wrote test suites in all the top testing frameworks for Rails and noted that personal preference and taste may be the only core difference.
- Some testing frameworks (Rspec) influence code design to some degree, although all were capable of helping you evolve your API during the test cycle.
- Each framework had similar levels of test code to test the same app. None stood out as particularly terse or verbose in comparison to one another.
- It is certainly possible and in some cases ideal to mix-and-match testing frameworks. Using test/unit for unit tests and shoulda for functional (behavioral) testing would work well.
- unit_record gem allows you to run unit tests without touching the database. Huge speed improvement, but means you need to plan accordingly when dealing with models.
- fixtures, when used for test data are evil and should be avoided. Instead rely on a Factory pattern or Object Mother pattern which assumes reasonable defaults.
- unit_controller gem allows you to better separate controller tests from the view
- deep_test gem for running tests in parallel. Will sometimes cause problems, decent for individual developers, not for CI
During the conference, I also got a better understanding of Phusion’s Passenger, a.k.a mod_rails. While I didn’t attend either of their sessions, I met with one of their developers at breakfast. They seem to have a great story for reducing the complexity of rails deployment, they leverage some of the great performance of Apache and can handle up to 20 instances of your app per Apache (+mod_rails) instance.
Both David and Kent Beck’s keynote presentations were of an inspirational nature. While David played role of “life coach”, Kent Beck talked a lot about what the best perspective might be on “agile” development, and the various value propositions that make sense when selling ideas to the masses. While I took several lessons away from these great presentations, it’s pretty difficult to sum them up in writing. I’m sure (better quality) videos of them both will be online soon, so you can see them for yourself.
Overall, the conference far exceeded my expectations. It was nice to meet and hang out with a lot of the Rails community, see where our collective heads are at and learn a lot along the way.
Connecting to Oracle from Ruby on Rails
When Rails 2 was initially announced, we heard that commercial database adapters would be maintained outside of rails core.
The commercial database adapters now live in gems that all follow the same naming convention: activerecord-XYZ-adapter. So if you gem install activerecord-oracle-adapter, you’ll instantly have Oracle available as an adapter choice in all the Rails applications on that machine. You won’t have to change a single line in your applications to take use of it.
That also means it’ll be easier for new database adapters to gain traction in the Rails world. As long as you package your adapter according to the published conventions, users just have to install the gem and they’re ready to roll.
While getting ActiveRecord to speak Oracle is available by way of just one gem, connecting to Oracle from rails can still be a touch difficult. Here are some notes from my recent venture into Rails/Oracle land, where I came out victorious, if left feeling just a little dirty.
The Goal
I’ll assume that the desired goal is to have an “application server”, which hosts your Rails app, connect to a remote database server, running some recent version of Oracle (8i+).
Application Server
- First, make sure you’ve got everything you need to run your Rails app. Ruby, rubygems, Rails, necessary gems (vendor’d or otherwise).
- Install the Oracle Client (or instant client) software for your hardware architecture.
Note: On Linux, the client requires over 1Gb of swap space to install. (insert snide remark about how “fat” the Oracle database is here) Here are some tips on adding swap space to a UNIX system. - Create a tnsnames.ora file on your app server with the connection information to your trusty, hardworking oracle database.
(/etc/tnsnames.ora is a nice (and default) spot on UNIX systems.) - Install the ruby-oci8 gem.
On UNIX systems, you’ll need to build the gem, which means you’ll need a fairly complete development stack including make, (g)cc, headers, etc. I’d love to find out if there is an easier way to build this gem so it could be deployed in app/vendor/gems! - Install the activerecord-oracle-adapter gem:
> gem install activerecord-oracle-adapter –source http://gems.rubyonrails.org - Configure your rails environment(s) to use the oracle database adapter:
production:
adapter: oracle
database: [your tns name]
username: [your user name]
password: [your password]
Gotcha!
Rails migrations specify modifications to your schema in a ruby syntax. The available column types defined in a rails migration will be translated into a database-specific type during the migration. The table below maps the ruby column types to Oracle data types:
| Rails migration notation | Oracle data type |
| :integer | NUMBER(38,0) |
| :float | NUMBER |
| :datetime | DATE |
| :date | DATE |
| :timestamp | DATE |
| :time | DATE |
| :text | CLOB |
| :string | VARCHAR2(255) |
| :binary | BLOB |
| :boolean | NUMBER(1,0) |
Fixtures
If you’re using fixtures with an Oracle database, make sure the values in your yaml file can be translated approrpiately into the corresponding Oracle data types.
One example may be the translation of a bare Fixnum (eg. 100) into a column set to :text. Since Oracle translates :text columns into CLOBs, the Fixnum in your fixture data will need to be coerced into a String. FAIL! In this case, quote the values in your fixtures (eg. “100″).
Further Reading
Obie Fernandez’s article on Connecting to Oracle from Ruby on Rails (A little dated, thus this post)
Oracle Instant Client
Ruby on Rails with Oracle FAQ
Tips for optimizing Rails with Oracle
Supporting unique headers per SOAP request in soap4r
soap4r is a SOAP implementation in Ruby. If you’re writing a SOAP client in Ruby (with or without Rails), soap4r is great. Until it isn’t. Since this is a rather specific use case that I’m solving for, I’ll assume that you have some experience using soap4r.
Headers
Some SOAP services will utilize a header, which is part of the SOAP spec. A header may include information that is used to process the request, but is not part of the SOAP request message body. It is typical for some services to use headers to send API keys or other authentication mechanisms.
Here is a simple example that ships with soap4r
Headers are a property of the Driver?
Yes. As you can tell, the SOAP driver has a property called headerhandler, which stores the header(s) that will be sent along with each service call. The call looks like this:
Since << is an alias to add, we’re just adding a header to the driver here.
But what if the service requires certain unique properties in the headers of each service request? Let’s say you have a service that within the header of the SOAP request, requires that you specify the IP address of the remote host, if the service is initiated via a web browser. In this use case, we need a unique header per SOAP request.
Open up the headerhandler
We can see that the headerhandler property of the SOAP driver holds a SOAP::Header::HandlerSet object. Let’s open ‘er up. We’ll add two new methods that will allow us to set a single header on the driver, regardless of its current state.
Set and send
Now we can do this, to set the header before we make our call:
Gotta synchronize!
Since the creation of a SOAP driver is expensive (for those of us who create them on-the-fly by reading in the WSDL), it is common practice to cache the driver and reuse it each time. (Remember that this is how we got into this mess in the first place, since the headers live on the driver, and we reuse the same driver for every request.)
It is possible, then, to have two threads interacting with your service client class, which in turn is relying on this same driver for those two requests. The code above is not thread safe, in that the driver’s headerhandler may have been set by thread1, but thread2 makes the first getFoo service call, resulting in the sending of the wrong header. DOH!
To fix this, we’ll need to wrap that call in a synchronize block:
Sure, it is ugly. Since the design of soap4r is such that the headers belong to the driver, this sort of solution may be our best bet. I look forward to any alternate suggestions.
Note: This work was in response to my original question to the soap4r forum.
-
Brian Doll
brian@emphaticsolutions.com briandoll @ github
Software/Systems Architect
San Francisco Bay Area, California
Subscribe in a reader