Dec
24
2010

Order Tara's Bicycle Touring Cookbook Today!

Our Process: Google Maps & GPS Tracks

by Tyler

This entry is part of an ongoing series about how we've documented our adventure.


Throughout our trip, I have saved our GPS tracks nightly. My plan to maintain one unbroken line, showing every road we've ridden for the entirety of our adventure has been mostly successful. Though some part of me liked the idea of this, I didn't originally spend too much time thinking about what I'd actually do with the result.

For the first month, as our track crept its way through England at a snail's pace, and we wondered if we'd ever get out of Northumberland, the result of my efforts was stuck on our laptop, visible only through the software that communicates with our GPS (MapSource).

(Speaking of which, I just had a good laugh at myself reading the entry Fare Thee Well Northumberland. I can only shake my head at the fool who was so wrapped up with concern about how far and how fast we rode.)

Eventually, it dawned on me that we ought to be able to put the tracks on our website; in this way, we could see our progress next to our pictures and writing! It took a few hours effort, but sure enough, I was able to do it.

Warning: technical content ahead.


Over the last two years, I've fielded numerous emails about how our map works. In this entry, I will attempt to explain enough of it to reproduce something similar. Jumping right into the code, here is a minimalistic example of how to display a polyline overlay using the Google Maps v3 API.

A heavily commented sample of the code below can be downloaded here.

<!DOCTYPE html>
<html>
<head>
  <meta charset="utf-8"/>
  <title>Google Maps: Garmin Tracklog Polyline</title>
  <script src="http://maps.google.com/maps/api/js?sensor=true"></script>
</head>
<body>
  <div id="gmap" style="height:500px"></div>
<script">
var map = new google.maps.Map(document.getElementById('gmap'),
{
  zoom: 7,
  scrollwheel: false,
  mapTypeId: google.maps.MapTypeId.ROADMAP,
  navigationControl: true,
  navigationControlOptions: {style: google.maps.NavigationControlStyle.SMALL},
  scaleControl: true,
});
function coord(lat,lng) { return new google.maps.LatLng(lat,lng); };
function polyline(m,coords,color,opacity,weight)
{
  if(!color) color = '#ff0000';
  if(!opacity) opacity = 0.5;
  if(!weight) weight = 6;
  return new google.maps.Polyline(
  {
    path: coords,
    strokeColor: color,
    strokeOpacity: opacity,
    strokeWeight: weight
  }).setMap(m);
};
polyline(map,
[
  coord(46.6609,9.5765),
  coord(46.6589,9.5828),
  coord(46.6126,9.5921),
  coord(46.5671,9.6215),
  coord(46.5163,9.6304),
  coord(46.4758,9.6465),
  coord(46.4648,9.6993),
  coord(46.4707,9.7542),
  coord(46.4562,9.7932)
]);
map.setCenter(coord(46.5163,9.6304));
</script>
</body>
</html>

The end result should be something like this:



With the hurdle of showing a map and a line overlay cleared, the next step in actually using this stuff is generating a series of real coordinates to show. Most likely, this would be dynamically created by some server-side code that reads points from a tracklog/database/whatever.

The method I use is saving a track in MapSource as a text file, and then parsing the results into a database for later retrieval. To create a compatible document for testing, open MapSource and retrieve a track log from a GPS unit (or some other saved location).

Once the map has a single track in it, go to File > Save As and choose text for the Save as type:

Track Saving

The only data the example below retrieves is the coordinates for each trackpoint (the file also contains information about altitude, speed etc). There is one tricky bit to making this work, and that is converting the coordinates from decimal minutes to the plain decimal format required for google maps. MapSource defaults to hddd°mm.mmm, so that is what the conversion code is expecting.

Here are a set of PHP functions for converting coordinates, parsing a track file into an array, and turning the resulting data into a javascript call just like the one from the first example:

<?php

// convert decimal minute coordinates to decimal format for google
function convert_coord($coord)
{
  // match N46 39.559 to [N][46][39.559]
  preg_match('/^([N|E|S|W])+([0-9]+) (.*)$/',$coord,$match);

  // if matches not found, return passed token
  if(!count($match)) return $coord;
  
  // assign results
  list($coord,$hem,$deg,$min) = $match;

  // calculate decimal format, flip negative for W/S points
  $coord = ($deg+($min/60))*($hem=="W"||$hem=="S"?-1:1);
  
  // round by precision of 4 and return
  return round($coord,4);
}

// convert a mapsource text file to usable php array
function parse_tracklog($file)
{
  // fail gracefully if file is not read
  if(!$track = file_get_contents($file))
  {
    print "Unable to read tracklog [$file].";
    return false;
  }

   // explode by newline and remove header lines
  $lines = array_slice(explode("\n",trim($track)),9);

  // determine number of entries
  $count = count($lines);

  // initialize array to save track data
  $data = array();

  // loop over track data
  for($i=0;$i<$count;$i++)
  {
    // get data for current line
    $coord = explode("\t",$lines[$i]);

    // skip lines that don't have the right amount of columns
    if(count($coord) != 10) continue;

    // store converted lat/lng (could also get altitude, speed etc)
    $row = array
    (
      "lat" => convert_coord(substr($coord[1],0,10)),
      "lng" => convert_coord(substr($coord[1],11))
    );

    // append parsed row to data array
    $data[] = $row;
  }

  // return the array of points
  return $data;
}

// parse tracklog and print javascript call to draw polyline
function display_track($file)
{
  // parse data
  if($points = parse_tracklog($file))
  {
    // get middle point for centering on line
    $midpoint = $points[(count($points)/2)];

    // loop over coordinates and build javascript calls
    $coords = array();
    foreach($points as $point) $coords[] = "coord($point[lat],$point[lng])";
    $coords = implode(",",$coords);
    
    // display polyline
    print "polyline(map,[$coords]);";

    // center map on middle point
    print "map.setCenter(coord({$midpoint['lat']},{$midpoint['lng']}));";
  }
}

This methodology works equally well with other file formats like GPX or KML. Basically, any structured data that contains a list of coordinates can be used. The text file format does have the advantage of avoiding XML parsing, though.

A zipped package of the code from this entry can be downloaded here.


I use a slightly modified version of the parse_tracklog function shown above to store our converted points in a PostgreSQL database. Here is a simplified sample of my schema, the implementation of which is an exercise left to the interested reader.

CREATE TABLE day
(
  id          serial PRIMARY KEY,
  datestamp   date NOT NULL DEFAULT now(),
  lat         numeric(8,4),
  lng         numeric(8,4),
  dist        numeric(6,2) NOT NULL DEFAULT 0,
  alt         int,
  ascent      int NOT NULL DEFAULT 0,
  descent     int NOT NULL DEFAULT 0
);

CREATE TABLE day_trackpoint
(
  id       serial PRIMARY KEY,
  day_id   int NOT NULL REFERENCES day(id),
  lat      numeric(8,4) NOT NULL,
  lng      numeric(8,4) NOT NULL,
  alt      int NOT NULL DEFAULT 0,
  dist     numeric(6,2) NOT NULL DEFAULT 0.0,
  speed    numeric(6,2) NOT NULL DEFAULT 0.0
);
CREATE INDEX day_trackpoint_day_id ON day_trackpoint(day_id);

Storing tracklogs in a database has many advantages over maintaining (possibly hundreds of) text files. Primarily, it opens up all kinds of interesting possibilities for viewing the information. Here are just a few examples:

  • Draw a line or show an elevation profile for the entire trip, or any chosen section.

  • Write an function to export any portion of the tracklogs in any format, for any GPS.

  • Do a search for any point on the Earth to see if we've been near it.

  • Do a search for any day and time in the period we traveled to see where we were.

Eventually, I plan to implement several of those ideas (and many others). Most of this will have to wait until we return home though, lest I get stuck procrastinating by programming, and we never finish our journal!

A zipped package of the code from this entry can be downloaded here.


Previous Our Process Entry
Our Process: Automation & the GSDB
In Years Past and Future
2009 - To Tataouine
G
Topics:

5 comments

You so got in there, lol
Posted by Amanda on March 2nd, 2011 at 10:22 PM
Why don't you try to put together a package of your codes so that the less-geekified members of society who would like to duplicate some of the things you've done with the combination of your blog and trail? I know that when I'm setting up a group's trail and trying to get people enthused enough to get out there and go hike/bike/walk...I'd be ecstatic to have a way to set up something, instead of using my present laborious method of MS Streets & Trips with hand entered routes or trying to create a route on Google Maps. (MS Streets & Trips doesn't even offer a bike route feature or even show hiking or biking trails or trail heads.) I've been so disgusted with GPS use in the car that I didn't even entertain the idea of using one for creating route maps because of the awkwardness of trying to use the data in terms of posting it to the web. I love the way you've managed to integrate it all together so seamlessly!
Posted by Gia Scott on March 3rd, 2011 at 3:58 AM
It's good to see a programmer workin' that magic from the road, a real inspiration to the land-locked office workers among us. Keep up the good work. I find putting the data in the database and using the Google API to overlay vectors on map graphics a very powerful presentation option. Sounds like you are developing an application here, nice.

What I do is take my .gpx track files from my Garmin Dakota GPS and convert them to .kml files which is the format that my GoogleEarth likes. I convert them using GPSBabel inside a Python script via the Python-GPSBabel bindings, but the website http://www.gpsvisualizer.com/map?form=googleearth also works well.

Then once in GoogleEarth I am able to use the ever-improving interface to organize my data into a logical structure, which unfortunately is limited to hierarchical folders, i.e. no tagged objects. But the maps are drawn real-time which is super.

Then once everything is in the right format I save off my latest data to a subset .kmz datafile which I upload to my website and reference for that day's blog entry with the great Wordpress blog plugin called XML Google Maps by Patrick Matusz which does the heavy lifting: requesting the map and overlaying all the vectors and icons.

At the very end I save off all my data from GoogleEarth, "My Places", to a .kmz file which is a compressed version of the plain text XML .kml format. This serves as my data store which, at 50 KB, has averaged only about 100 bytes per kilometer so far.
Posted by Steven on March 4th, 2011 at 2:18 AM
Amanda--

Totally! :D

Gia--

I've entertained the idea of doing something like that on several occasions. It is quite a bit of work though! Writing a tool for another programmer is a relatively simple undertaking, but coding something with a sleek, easy-to-use interface for a non-technical user is at least a few orders of magnitude more work.

I might yet build some things for the cycle touring/travel world when we get back to the states though. Happily, there are a lot of great tools out there already, if you know what to look for.

Have you seen the cycling maps from Cloudmade or the tool Bike Route Toaster? Also, GPS Babel is pretty handy too (Steven mentions it above).
Posted by Tyler on March 6th, 2011 at 12:05 PM
Steven--

Thanks for sharing your process man! I think you're definitely on the right track with the KML/Google Earth stuff.

For people who don't use Wordpress: displaying a KML or GeoRSS file is super simple with the v3 API (I'm guessing that is what the plugin uses).

I've been meaning to try it with all of our tracks visible to see how badly it slows down the map. Basically, I'm curious if there are any optimizations that take place when you use that API vs manually drawing all the polylines. More tinkering for another day...
Posted by Tyler on March 6th, 2011 at 12:20 PM
...and sign up for our newsletter!
Post a Comment
receive email for new comments
check this box to prove you are human

HTML allowed:<a><strong><b><i><u><em><strike>