Dec
23
2010

Order Tara's Bicycle Touring Cookbook Today!

Our Process: Automation & the GSDB

by Tyler

This entry is part of an ongoing series about how we've documented our adventure.


When we left on this trip, neither of us had an inkling that we'd be documenting our adventure in such exhaustive detail. The original idea was simple: a flickr account, and a nice template at Blogger for our journal—just a little writing and a few pictures sprinkled here and there. I didn't know how to use commas, and neither of us had a clue what an aperture was.

Fast forward a month and things were slowly getting out of hand. Sitting at a coffee shop in England, it dawned on me that the GPS tracks we'd been saving for who-knows-what could be used to show our progress on a Google map! Then, somewhere in Tunisia, we signed up for our dreaded Twitter account, and shortly after, we started taking field recordings too.

By this point, we were generating quite a lot of "stuff" every day: journals, photos, audio, tracklogs, expenses, tweets and more. As the data piled up, and our Excel files got messier, it became pretty obvious that we either needed to give it all up, or start building systems.

I don't want data entry to be a part of cycling around the world!

As luck would have it, the software platform I've been developing for the last few years lends itself very well to this sort of thing. Warning: technical content ahead.


I develop a generalized database management tool which can be highly customized to administer data for things like insurance policies and claims, union employee scheduling and payroll, trucking company compliance, event registration and billing, corporate asset management, customer service systems, employee training, project management, and much, much more.

Basically, I write systems to manage all kinds of super-boring data. It is actually a lot of fun, mainly due to the fact that I like organizing things, and also because my clients rule. In order to support them (and now, us), my laptop is running a full web development environment. This is a big part of what makes working on the road possible for me, as I can program without being connected to the internet.

I develop using Python / PHP (and Node.JS for the next generation of my platform), store our data in PostgreSQL, and host our sites with Apache. I manage my source code and deployment with SVN (I can't wait to migrate to Git when the free time presents itself).

We work on the GSDB (journaling) simultaneously by connecting our laptops with an ad-hoc wireless connection, a network cable with a crossover adapter, or just by being on the same wireless network. Tara edits her hosts file to point the "fake" domain gsdb.local to whatever my laptop's IP address currently is.


In October of 2009, after many hours of coding, and way too much time spent trying to make heads or tails of the mess we'd created, our days, tracks, expenses, pictures and recordings had been entered into my software. (Long after this, we eventually migrated our journal as well; more on that in another our process entry.)

When the dust settled and the kinks were ironed out, we launched what would become the engine room of our website, and what is now the hub of all of our projects together, the behind the scenes section of our website, which we've dubbed the GSDB (Going Slowly Database).

Here is one of the many things we use it for:

Converting Tracks to Travel Data

When we make camp or arrive at a hotel, the first thing I do is fire up my laptop and connect our GPS to it. Then, using Garmin's MapSource, I download the tracklog. This is, in very simple terms, the result of our GPS saving information about its current location to a file every second or so. The points in a tracklog form a long connect-the-dots which shows everywhere we've been.

GPS Tracks

With our tracklog at the ready, one of us creates a new day in the GSDB, and attaches the file to it. When we hit save, my system automatically uses the track data to enter our current location / altitude, distance traveled, total ascent / descent etc.

From turning the laptop on to completion, this entire process takes less than five minutes.

GSDB: Days

The track data we store in our database powers our google map, creating the markers that show each place we've slept, the tracks and elevation profiles between them, and the Show Map & Statistics link for our journal (which displays where we were on the date of the entry).

Next up: some example code for our google map, more about our custom journaling platform, and lots of details about how I'm working to automate everything in our documentation process that isn't writing or editing photos.

Previous Entry
-
Next Entry
-
G
Topics:

43 comments

As someone who spent the last three years as part of the MySQL documentation team I found this article absolutely fascinating! :)

I see you are using Postgres - I thought you were originally using MySQL - maybe a wrong assumption on my part.

I've used SVN for the last 4 or 5 years, and am also moving on from it. I'll probably be making the jump to Mercurial though (need to research more). I've used Bzr a bit and didn't really like it.

How do you find Python compared to PHP? I've used PHP for a while (and wrote some of the documentation for ext/mysqli, ext/mysqlnd). Which parts of the system do you use Python for? I will probably stay with what I know, but was curious on your use of Python.

Looking forward to further details.
Posted by Tony on March 2nd, 2011 at 9:59 AM
OK, so I'm only mildly geeky and I enjoyed your article. It brought back memories - and wishes that we had two computers while we were one the road. We used our GPS tracks to tag all our pictures - which has been very handy. Scott just did a post-process - ensuring that the time stamps on the camera matches the GPS, once both files were uploaded to the computer the GPS points could be automatically added to the photos (don't ask me the details, Scott did it). I love the database idea for storying all the source files - could definitely use that for my research projects!
Posted by Becky on March 2nd, 2011 at 5:02 PM
I stumbled upon your journal about a year ago while searching the net for inspirations for my 9th Swiss bike tour, and I'm following it ever since. I almost wrote a comment on several occasions, but somehow I was always too lazy to do that, shame on me. I could have written about how all my Romanian colleagues immediately agreed to your "you know you're in Romania when you are relieved to ride on a dirt road riddled with potholes" statement, or the fun I had reading about your troubles with Ingrid and Yves' german-speaking oven, because I have the same model and it does know English (and some twenty-odd other languages, too!), or how disappointed I was when there was noone at the top of Julierpass after I climbed it in an afternoon after a very late start from Tiefencastel, or how I followed your path to Madonna del Ghisallo on my tour's last day, except that I missed a turn and ended up taking a longer way up there with an _additional_ 200m climb, or that I spent so much time reading your journals that when I type "go" into Firefox' address bar, then it offers goingslowly.com before google.com... But all this was not enough, you had to mention that you're planning to migrate to Git to make me actually write a comment (;

Go ahead, it has fundamentally changed how I approach a new programming task, how I think about the evolution of source code, and my view about the purpose of revision control in general. How could I have ever lived without it?!

And I'm also glad that you will run my code, even when it's only a couple of lines (;
Posted by sz on March 3rd, 2011 at 12:22 AM
Tony--

Even though I don't use MySQL, thanks a million for contributing to that project; good documentation is so incredibly important for development! I made the switch to PostgreSQL many years ago, back in the 4x days when it was missing all kinds of important features like foreign keys, views, subselects etc.

On Python/PHP: I started web development using PHP when I was a really inexperienced programmer. As my knowledge and skills have improved over the years, so too has my disdain for the language. Some things that realy stick in my craw are the joke of an OO model and the hacked in namespacing implementation. There are lots of other things too; UTF8 handling is a nightmare, the memory footprint is huge, the standard library naming conventions are ridiculous, I could go on and on.

All of that said, there is no denying it is easy to get a lot of stuff done quickly using PHP. If that is your only definition of a good language, PHP is great. I've outgrown PHP in the sense that I've become a good enough programmer to see all the rough edges and be driven nuts by them.

Though I am migrating my business to Python, but I don't have a hybrid model in place. The next version of my platform will be Python (using Pyramid).
Posted by Tyler on March 6th, 2011 at 12:42 PM
sz--

Thank you so much for finally saying hello! I'm so pleased you've enjoyed reading along with us for these last years.

Tara and I now joke about how Romanian roads were like a US interstate compared to Mongolia :) Phew, climbing Madonna del Ghisallo on the last day of a tour? That is dedication. I can't believe you found another 200m of climbing on that hill!

Your vote of confidence on git is music to my ears, as is the siren song of rebasing. I can't wait.
Posted by Tyler on March 6th, 2011 at 12:52 PM
Becky--

What a great idea! What did you do if your tracks didn't have a perfectly matching timestamp? Fudge it by choosing the closest? Leave the photo without a geocode?
Posted by Tyler on March 7th, 2011 at 2:06 AM
> What a great idea! What did you do if your tracks didn't have a perfectly matching timestamp? Fudge it by choosing the closest? Leave the photo without a geocode?

If the track didn't have a perfectly matching timestamp, but there were points before and after, do a linear interpolation using the closest two points. That got about 95% of our photos, since we generally had our GPS turned on whenever we were moving.
In cases where we were stopped for the night (or first thing in the morning), we used the last/first point of the day. On the odd occasion where our GPS tracks didn't cover our location (i.e. Syria, since GPS was illegal there at the time), I used Google Maps to place the photos.
We used Geosetter http://www.geosetter.de/en/ for most geotagging, and the free version of GPS Trackmaker http://www.gpstm.com/dwlpage.php to simplify the tracks (remove extraneous points).
One of these days I really should write up our workflow like you've done... very nice!
Posted by Scott on March 7th, 2011 at 3:22 AM
In case you need to bypass Google Account you can get Bypass APK
Posted by Roger Federer on April 25th, 2018 at 1:23 AM
Get Quick Shortcut Maker APK download if you wish to make shortcuts
Posted by Brad Ron on April 25th, 2018 at 1:29 AM
In case you need to bypass Google Account you can get FRP Bypass Tool
Posted by John on April 28th, 2018 at 2:18 PM
Do checkout these case fans for gaming pc here.
Posted by James Pal on May 16th, 2018 at 12:50 AM
Getapk Market
Posted by Ben on May 23rd, 2018 at 1:22 PM
Find the best pictures related to
Happy Independence Day 2018
Posted by Welrop on May 26th, 2018 at 6:19 AM
best kodi builds
Posted by Sam on June 1st, 2018 at 10:58 PM
Addressing the nation on the 71st Independence Day from the ramparts of the historic Red Fort Independence Day Drawings and Ideas
Posted by alex on June 17th, 2018 at 9:43 PM
sony a7 iii is a brilliant full-frame mirror-less camera created to date. Loaded with a range of fantastic features this camera can do everything, which others high-end cameras can do

Posted by a7 iii on June 21st, 2018 at 12:18 AM
Happy Independence Day Images 2018
Posted by ICC World Cup 2019 Schedule on June 26th, 2018 at 9:50 AM
Let's celebrate with us by sharing wishes to our dear and near ones.
happy independence day 2018
Posted by happy independence day 2018 on June 26th, 2018 at 3:43 PM
Nowadays everyone wants to watch online videos and movies. No one has much time to download the video and watch here
Posted by thevideo.me/pair on June 28th, 2018 at 12:57 AM
we are celebrating 72nd independence day of India 2018.
Happy Independence Day 2018 Sms & Message

Posted by Deepak Chauhan on July 2nd, 2018 at 3:07 AM
extratorrent proxy 2018
spacemov.in
sportsarefree.xyz apk download
Posted by Jake on July 11th, 2018 at 9:10 AM
Find here best independence day images 2018 download
Posted by Images on July 17th, 2018 at 11:34 AM
It is beautiful collection of
Posted by Imtiyaz Mahetar on July 18th, 2018 at 9:20 AM
Its great. Read More about sushma buildtech
Posted by flatooz on July 19th, 2018 at 4:06 AM
Check out!!!
Posted by Bigg Boss on July 21st, 2018 at 9:02 AM
Hi check out my website pls. the best indain independence day website
Posted by gautam on July 25th, 2018 at 4:58 AM
Check out these happy independence day to India
Posted by Jagdish Kumar on July 25th, 2018 at 7:20 AM
A good site with exciting content that is what I need.
Thank you for making this web site, and I will be visiting again. It is really resourceful website with great content.
It’s my first visit to your website and i have found it so useful and informative specially this article. . Thanks for this article. Keep up the good work!!

Bangla Sad sms-bangla valobashar koster sms

17 best Valobashar kobita bangla love poem

Happy Valentines Day Bangla Sms 2019 For Boyfriend Girlfriend

Happy Valentines Day Bangla Sms 2019 For Boyfriend Girlfriend

Bangla caption for facebook-status for facebook bangla

Bangla birthday sms for lover

Bangla romantic sms for girlfriend

Facebook status bangla
Posted by Bangla love sms on July 27th, 2018 at 1:06 PM
Bangla status

Bangla sad sms quotes 2018

great work thank you so much ..
goog job keep it up....
Posted by Azma falla on July 27th, 2018 at 1:08 PM
Thanks admin for sharing this great content here
Posted by indian independence day wishes messages on August 2nd, 2018 at 12:08 AM
HI Thanks for sharing these infornative posts also check these articals
<----------------------->
Independence Day 14 August Sms in urdu and hindi 2018
<----------------------->
14 August Quotes on Independence Day 2018
<-----------------------
Quotes on independence day in hindi on 15 August 2018
<----------------------->
14 August wishes, Pakistan Independence day greetings 2018
<-----------------------
14 August Quotes Quaid-e-Azam & Allama Iqbal 2018
<----------------------->
50+ Best Pakistan Independence Day Quotes in English [2018]
<----------------------->
10-best-pakistan-independence-day-songs 2018
Posted by Azhar Habib on August 2nd, 2018 at 8:59 AM
HI Thanks for sharing these infornative posts also check these articals
<----------------------->
Independence Day 14 August Sms in urdu and hindi 2018
<----------------------->
14 August Quotes on Independence Day 2018
<-----------------------
Quotes on independence day in hindi on 15 August 2018
<----------------------->
14 August wishes, Pakistan Independence day greetings 2018
<-----------------------
14 August Quotes Quaid-e-Azam & Allama Iqbal 2018
<----------------------->
50+ Best Pakistan Independence Day Quotes in English [2018]
<----------------------->
10-best-pakistan-independence-day-songs 2018
Posted by Azhar Habib on August 2nd, 2018 at 8:59 AM
slogan on independence day
Posted by slogan on independence day on August 4th, 2018 at 10:13 AM
happy independence day 2018 images
Posted by jh on August 6th, 2018 at 2:41 AM
jhdjhjdhjdh jdhjdhjhdjh dhjdh jdhjdh jdhjhdhjdj
Posted by happy independence day 2018 on August 6th, 2018 at 2:52 AM
raksha bandhan wishes for brother
Posted by raksha bandhan wishes for brother on August 6th, 2018 at 6:38 AM
raksha bandhan wishes for sister
Posted by raksha bandhan wishes for sister on August 6th, 2018 at 6:39 AM
Great article thanks
Posted by Click here on August 12th, 2018 at 3:36 AM
Thanks for sharing this great content here I love this post very much.
Posted by Bigg Boss Tamil Vote on August 12th, 2018 at 8:22 AM
quotes on independence day in hindi 2018
happy independence day shayari hindi 2018
happy independence day in hindi 2018
happy independence day wishes in english 2018
independence day Shayari in hindi 2018
happy independence day quotes 2018
independence day quotes and sayings 2018
short thoughts on independence day 2018
Posted by sanjana Singh on August 14th, 2018 at 4:01 AM
Thanks for sharing this nice post check my this post about
child support calculator florida
Posted by 99blogging on August 15th, 2018 at 11:44 PM
मर्दाना कमजोरी इलाज


Posted by Ansaar on August 19th, 2018 at 1:39 AM
live nettv
live net tv app
live tv app
net tv live
livenettv download
That's all there is to it. It's super simple to set up live TV on the Shield and makes the overall experience just that little bit better.
Posted by live nettv on August 23rd, 2018 at 9:15 AM