Category: Flagship Geo

Open Source Mapping Framework

RCOS Meeting 2010-08-13

Prof. Moorthy was unable to attend the RCOS meeting last Friday (2010-08-13) so I’d like to provide a few updates on the projects and groups that presented during the meeting.

Matthew O’Brien started the meeting off with some updates on Project Community Connected.  Matt has finished the main application for his community bulletin board, and is in the process of cleaning up some of the style elements and fixing a few bugs.  He’s hoping to test out the program sometime this fall and was going to be reaching out to a few communities over the next few weeks.

Anthony Loven and Brittany Jason have been developing IntuiTask, a lightweight agenda and calendaring application for the Android platform.  They’ve completed the basic application and are busy at work polishing the user interface and are going to be testing the application on different hardware devices over the next few weeks.  During the Fall semester (or may the Spring) they plan to add GPS support to their program so you can be notified of tasks you can complete around you, like picking up prescriptions in a pharmacy when you approach one.  Anthony mentioned they would be holding off on releasing their application on the Android Marketplace until they could finish some more testing to avoid any bad reviews an early beta edition might receive.

Sean Austin, Diana Mazzola and Griffin Milsap presented their final updates on droidViz.  They have been very busy refactoring their application to serve as a visualization framework which can be used by other applications.  They’ve named this new tool ‘LucidEngine’ and were able to show off a neat demo application they built using it.

Jacob Katz presented via Skype on his chess program, OpenGambit.  Over the past few weeks he has been fixing a few board-evaluation bugs and is going to be starting development work on the GUI over the next week or two.

Finally, Graylin Kim presented on all of the open source work he has been doing this summer.  He has been working with the New York State Senate on  a bunch of projects to help open up their various data feeds.  Graylin presented a few demos, showing off the Senbook, demonstrating all the work he has done aggregating different data sources from the Senate’s various data sources.

Posted on by Brian Michalski 0

Polyline Editing

I spent the better part of a week or so adding support for polylines into Flagship Geo.  The hard part wasn’t the data structure development, it was the ability to easily edit the lines in a way that made sense.

Google Maps works as a great User Interface control for maps, but its just a control.  You have to tell it what to do when someone drags a marker around or clicks on the map, there is no automatic way to tell it that the person is trying to edit the line.  All the objects people can interact with need event handlers written for them, which reminds me a lot of regular desktop application development.  I usually take for granted the fact that a tag like <a href=”#”></a> will automatically know what to do when clicked.  Tools like jQuery make it easily to build on those actions, sending that a href into a AJAX library or something.  Building the polyline editing interface was much more like building a desktop application in C, Python, or alike where you have to bind everything yourself.

Looking back on the past week, the hardest part was the ability to delete a point from your line.  This kind of reminded me of my days in Data Structures or Algorithms at RPI when you’re manipulating nodes in a tree or a list (a polyline is just an array of latitudes & longitudes) except unlike C, JavaScript doesn’t have any notion of pointers.

I got stuck trying to do have my markers (the icons you drag to manipulate the map) both update the visual line you see AND update the hidden HTML fields that store the latitudes and longitude for the point.  I found that replacing the event listeners with code that contained the update line point weren’t working correctly, so I ended up building my own pointer-esque data structure that contains id numbers to match up markers, polyline points, and html points.  While I try and minimize my use of global arrays, I couldn’t find any decent way around JavaScript’s (and Google Maps event listener) limitations.

You can take a look at the commit where most of this code was sent along here: http://github.com/bamnet/flagship_geo/commit/f2e8e113bcaef2a78573d8f5569e10f18bc8de4a

Posted on by Brian Michalski 0

Development Environment [offline]

One of my biggest complaints about Ruby on Rails relates to the environment needed to develop an application.  Last Wednesday I took-off for a few days of holiday up in Vermont.  Internet access is limited to my mobile phone while I’m in Vermont, so I have to take development as “offline” as possible while I’m up there.  Doing this with my Rails applications is a mess.

If I was doing PHP development, things were pretty straightforward for me to setup my laptop as a mobile development environment.  I could download XAMPP, or another all-in-one Windows-AMP stack, to a USB drive and copy all the source code I needed.  I guess I would also need to export a copy of a MySQL database as well, but that’s a pretty quick step.  After running the XAMP installer, I could drop all my source code into the folder and presto! things would be working good as new.

Ruby on Rails doesn’t afford me those kind of luxuries.  My laptop runs Windows 7, so I can’t just `gem install rails` and call it a day.  I’d never setup a local development environment on Windows (I always use a Linux server), so my plan was to download a few different packages and see what I could make work.  Since I was in a rush to depart, I downloaded as much as I could under the premise that I would be able to install things later.  Hopefully I would have enough pieces.

I found two common approaches out there to quickly setup a development environment on Windows.  The first involved virtual machines, where you would run your own Linux server within Windows.  I tried downloading 2 (or maybe 3) different server images, none of them would boot correctly in VirtualBox.  My second plan was to install a XAMP-like environment but with Ruby on Rails instead of PHP.  I was able to get this to install, but it seemed the stack I downloaded included some strange version of Ruby that wasn’t compatible with my application.

Nevertheless, I realized that regardless of what I did while “offline”, I wouldn’t have been able to get my applications up and running without internet access to install the different gems and plugins I require for my applications.  I had probably downloaded enough pieces to patch together a framework/Rails-stack, but that wouldn’t include the externally referenced modules in my code.  Gems and plugins (and by plugins I mean git submodules) are great, but you need to plan for them first when you’re going offline.

In a deployed, production-level application, gems and plugins might be frozen or bundled with your application, but that doesn’t fit well with the development of an application.  It would be really handy if the “download” button on Github automagically included all the git submodules you would need for an application, and I’m optimistic bundler will help solve some of the gem-deployment issues Rails applications face.

Now that I’m back “online,” I’m going to stick with my trusty development servers.  They might not be easy to carry around with me, but at least I don’t have to worry about keeping their configuration up to date or taking them offline.

Posted on by Brian Michalski 0

Application Testing Tools

Yikes, last week I was too busy working/in-meetings to squeeze in a blog post, so I’ll post a quick one to start of this week.

Since I’ve taken the plunge into developing a test suite for all of my new rails applications (most notable Concerto 2), I’ve struggled with the testing work flow.  Its very natural for me to write a piece of code, refresh my browser, verify the results, and repeat the cycle until I get it right.  The more official testing strategy requires me to write a test + the actual code and make sure the two of them mesh up before I even bother to refresh my browser.

Starting this process was a bit hard for me, probably because my test-writing skills weren’t so hot.  Many times my `rake test` results were failing not because there was an actual bug in my application code, but there was a bug in my testing code.  Unlike the application code which can be viewed in a browser, there really isn’t any way to know if your test is running as expected, so you really have to keep things as simple and straightforward as possible.

If you’re going to be developing in Ruby on Rails and using tests for your application, I’ve found a few tools that help out a lot:

  • autotest – Autotest is a tool that automatically runs the relevant portion of your test suite as you save files.  Essentially, when you save the file foo.rb autotest will automatically run the tests it thinks are related to foo.rb and nothing more.  In rails land, this saves a lot of time waiting for tests to run that are unaffected by a small or incremental change.  You can install some cool plugins to make the output red or green or use growl notifiers if you want… I just use the red/green plugin to spice up my terminals
  • CI Joe – CI Joe is a really simple continuous integration server.  A continuous integration server (automatic testing server) doesn’t make a whole lot of sense for single developer projects, but as more people are contributing and committing code it helps to have an automated system that is making sure the latest commit doesn’t break any tests.  I set this up for Concerto 2 development so everyone on the development team can quickly see if the HEAD is broken or not without having to ssh somewhere, git pull, and rake test on their own.

My one cautionary word about testing: don’t dive in too deeply.  It makes sense to develop really well formed test cases for critical portions of your application or code that is constantly being changed, but I’m not sure there is value in developing 12 tests to validate every possible form input and exact error message wording… at some point your time is better spent developing the application instead of supporting tests.

Posted on by Brian Michalski 0

Builder Files

Typically when I go to generate some XML-style documents in Ruby on Rails I manually code the XML syntax and manually escape and substitute the strings where I want them. This technique is pretty sloppy in to toss in a view, and relies on your ability to generate well-formed XML off the top of your head. (Its usually the escaping that becomes an issue.)

Thinking back, this the probably the quick and easy technique I picked up from my PHP development projects.  I could hand-code and debug some XML faster than I could find a suitable library, install it, and figure out how to work it.  Sometimes it’s just easier to do things the hard way.

As a rule of thumb, I think the hard way in PHP never translates well into Ruby on Rails.

Yesterday I was struggling to cleanly generate XML in Rails 3 because of the new default sanitation.   Using Builder was easy enough to generate the right XML structure for my KML file, but getting it to output was a challenge.  All of my &lt; tags were getting replaced with &lt; and such, and the raw parameter (&lt;%=raw foo %>) wasn’t cooperating.

I ended up discovering that I could rename my file from show.kml.erb to show.kml.builder, and I wouldn’t have to mess around with any escaping or erb syntax at all. You can check out the code I used in this commit.  It might be just me, but I always struggle to find the appropriate documentation for these little nuances in Rails.  There is tons of code showing how to use Builder to build XML documents, but not one of them mentioned what to name your file.

This technique definitely took me longer than a quick pass through manually plugging in the string would have, but its a lot cleaner.  I don’t have to worry about escaping or generating valid XML, and if performance is an issue I can install a new XML builder to speed things up.

Posted on by Brian Michalski 0

Bumping to Beta 4

I just bumped Flagship Geo to use Rails 3 Beta 4 (commit), luckily everything seems to still be working when I run my test suite.   Installing the new version of Rails is pretty easy, you just have to run `gem install rails –pre`.  You might want to sudo if you keep your gems system wide.

I use Passenger to serve most of my rails app on my development server and I had to change around the config a bit to get things working with Beta 4.  Specifically, I had to switch Passenger to treat my application like a Rack application.  I don’t know exactly what this means from an application-architecture standpoint, but I believe its related to the initialization procedures used to boot the application.  To make the switch, I had to edit my public/.htaccess file.  You might need to edit your apache virtual host config if that is where you store your Passenger config info.  The switch is pretty easy, switch every instance of “Rails” to “Rack”:

RackEnv development
RackBaseURI /geo

Also, bundler has been getting on my nerves.  When I upgraded rails, bundler was updated to 0.9.26 which is very confused about its gem locations.  From the command line, everything works great.  I can rake test, ./script/rails console, and do all of that great stuff but when I load my application up in the browser half of my gems can’t be found.  I needed to sudo gem install them to manually make them available system-wide, doing just `bundle install` would install them locally which was good enough for CLI work but didn’t cut it for Passenger.  I believe this might be fixed in bundler 0.10… due out ASAP?

Otherwise, everything is working great.  I look forward to the Rails 3 RC upgrade later in the week… hopefully the upgrade won’t be much harder.

Posted on by Brian Michalski 0

Validating HTML Colors

I’m working on adhering to the HTML 5 standard for things whenever possible in Flagship Geo.  While I haven’t gone all-out just yet and completely standardized things, I recently spent a few minutes working on code to help validate colors.

If you had asked me over the weekend if I care a lot about web-colors I would have said no, but then I discovered a bunch of stuff in the HTML 5 about colors.  There is a special “color” input box (not yet implemented anywhere) which is suppose to help people select colors and there is a pretty thorough section on how to validate what belongs in an official color box.

Since validators in Rails 3 are much more straightforward to implement, I took a few minutes to toss one together that validates simple color strings, i.e those hexadecimal strings that start with a pound sign.  If you’re looking to do something like this in your application, here’s how:

lib/hex_color_validator.rb

# Validates a hexadecimal color string as defined in the HTML 5 spec.
# This validator only works for the simple case and does not support
# any legacy formats. See http://dev.w3.org/html5/spec/Overview.html#valid-simple-color
# for the format spec.
class HexColorValidator < ActiveModel::EachValidator
  # Verifies a color string is 7 characters long and contains
  # only hex values after the pound sign.
  def validate_each(object, attribute, value)
    unless value =~ /^#[0-9a-fA-F]{6}$/
      object.errors[attribute] << (options[:message] || " is not a properly formatted color string")
    end
  end
end

… paired with …
app/model/layer.rb

class Layer < ActiveRecord::Base

 # Validations
 validates :color, :hex_color => true
end

In my example, I’m validating the color attribute that exists on the Layer model. You can view the commit on GitHub if you’d like.

Posted on by Brian Michalski 0

Starting Flagship Geo

These days everyone is making maps, which is great!  As more an more geographic data is made available on the internet everyone is trying to display it on a map (how it should be displayed).  Unfortunately, most of the time when people/organizations start to display this data they start from scratch, writing their own custom mapping plugin leveraging something like the Google Maps API.  Other times people install Drupal or MediaWiki plugins to hack in map support, but none of these solutions solve the underlying problem: people want an easy way to embed more than just a Google Map in a webpage, they need some sort of framework to interact with the underlaying map data.

My goal for Flagship Geo is to develop a framework for geo-data that can easily be plugged in to existing applications or stand on its own, allowing users to generate maps with points, polygons, and paths.  You’re right to recognize Google has implemented a lot of this functionality with their Google Map Maker tool but unfortunately their tool is limited to Google Maps (i.e no Bing, Yahoo, Open Street Map, or alike) and the data completely lives in Google.  You can’t add a line of code or two to your existing model to instantly plot it on a map AND keep things synced up.

From a more technical angle, I’m going to be writing most of this in Ruby on Rails, probably starting with the latest beta3 release.  I’m not quite sure how I’m going to handle the multi-map-vendor display bit, but I’m interested in diving into JavaScript to see if I can whip up a single library that wraps multiple map providers APIs.

Posted on by Brian Michalski 0