Friday, October 20, 2006

My Holy Grail of Synchronization

the-holy-grail-of-synchronization

I came across this blog post (the link above) the other day claiming to have the “Holy Grail of Synchronization”. I started to get really excited as I clicked on the link, hoping for some relief to an issue that’s bugged me for a long time. Alas, it was only a guide to hack your way to synching your Outlook, Google Calendar, Gmail, iPod, and Mobile Phone together. It didn’t solve any of my problems, the best I could hope for it would be to raise awareness of this issue. So I decided to write my own thoughts down, with the hope of furthering that awareness.

My example compares my experiences with Palm and Yahoo in the realm of contacts and calendar syncing, but I could equally apply this to any device that contains address and calendar information, including Outlook and Gmail. In this article I mostly mention the address book, but the same can be applied to the calendar as well.

If there was ever an easy one sentence statement that would sum up what I want everyone to get from this article, it is this: Let me easily access my data so that I can share it with myself!

My Setup

Yahoo (Past and Present)
As for my email setup, I use Yahoo Mail. Yes I have a Gmail account, yes I really like it, but it lacks some features of Yahoo that I simply cannot ignore. So much so that I even have a Yahoo Plus account, which I happily pay $25/year for the use of.

It should be noted that Yahoo used to provide a tool for syncing with the Palm Pilot. It used to sync during the Palm’s HotSync process, when the Palm was trying to sync with the host computer. Yahoo’s tool would make a (often slow) connection to Yahoo to make updates to your web account. Often time the process would time out, or didn’t work at all.

I often felt the application would have worked much better if it would forgo trying to update Yahoo during the HotSync process and just update directly with Palm Desktop. I even wrote a few emails to Yahoo about this, but as usual, they never seem to respond to any user requests. I noticed the other day that Yahoo doesn’t even offer the application anymore (at least for Yahoo Mail beta users).

The only current way to get contacts into Yahoo is through their import procedures, which just require a specially formatted text file (CSV, vCard, etc) to import your contacts. There are no syncing abilities available.

Palm
I have been a Palm user for 8 years now. I started with the Palm III, and I used it to my hearts content until the device was on its last legs. Then I waited with that device a few extra months to get the Palm Treo 650.

If there was ever a complaint that stuck with me since my very first Palm, it was that I couldn’t share my contacts between devices. That’s why I got the Palm Treo – I was tired of trying to get my contacts on my Palm to be in sync with my cell phone. Finally that issue has been solved with the Treo, but I still have one remaining: I can’t sync my contacts on my Palm to my email address book.

Palm gives you the choice of 2 different applications to use when syncing your Palm device with your computer: Outlook and Palm Desktop. It should be noted that there are some utilities that will sync with Outlook, but I want no business using Outlook. I use web-based email for a reason – I have multiple computers and want access to my email from anywhere I am. I have been a Palm Desktop man ever since owning my first Palm. Anytime I have tried using Outlook, it just hasn’t worked out too well and I end up back in Palm Desktop land.

What I’d like to see

An application that can sync with everything; it would have a common database that would contain my address book. It would have plugins able to sync with all my devices/services that require it. It could access a Yahoo API to update users in my Yahoo Mail’s Contact list. It could contact directly with the Palm Desktop on my computer (and leave my Palm alone). Heck, it could even sync with Outlook, Gmail, Thunderbird, etc. Any company that wanted access to that data could provide a plugin for the user to install. It could be so simple.

I would like to see the application be a free open source tool. I would understand if a company was behind this effort and wanted to charge for plugins, but the base application would be free and open source. This way the application could avoid political issues (or at least attempt to). It could be ported to run on any OS, and if someone wanted they could write a plugin for it on that platform.

The application could set itself up by requesting which plugin it should use to import the users, then it could go from there. Conflicts? You can easily map out what fields are available for each service and provide an easy way for the user to decide.

What’s available now

These are the present possibilities as I see them. There might be other solutions, and if there are I’ll be glad to take a look at them. I’ll start off with my current setup, then move on to other services.

Here is my present setup for syncing my Palm and Yahoo (as per the Yahoo instructions):
1. Export my Palm address book via the Palm Desktop as a vCard
2. Go into Yahoo and delete all my current address book entries
3. Import the vCard with all my contacts in it


Plaxo
Honestly, these guys seem to be the closest to achieving the goal. My disclaimer is that I have never used them. I’m a little afraid of giving out all my contacts to a service that wants me to email and invite them to Plaxo. Stuff like that makes your contacts question if you are trying to sell their information.

Plaxo has a tool that claims it can sync with Yahoo and Outlook, but it will only touch a Palm if it HotSyncs with Outlook. To use this tool I would have to convert my address book over to Outlook, and I don’t use Outlook. Their solution is to export my contacts from Palm Desktop and import them into Plaxo. Why use them at all then? I can cut out the middle man and import those same contacts directly into Yahoo.

Goodlink
Goodlink is an Exchange (Outlook) synchronization tool that allows corporate users to read their email and keep their contacts and calendar in sync via the phone’s internet service. It has applications available for both the Palm and Windows CE devices. It basically makes these Smartphone devices act like a Blackberry.

My work bought me a Treo setup with this service (so now I have two Treo’s). I work from home, and I like that I can access my work email and work contacts while I’m not in front of my computer. It works out great when I’m making breakfast and I get an important email that I wouldn’t have gotten unless I was constantly hovering over my computer.

However, I despise the tool’s interface. When installed onto your Palm, it replaces the Palm’s address book and calendar applications. You can’t access any of your old contacts or calendar events, and you have to import them all into Outlook one way or another. Its interface is sorely lacking in features and the navigation system is not intuitive.

Coincidently, I ran into an article on how to break this take-over mentality in Goodlink. By making this change it allows you to manage the exchange contacts/calendar separately from the Palm contacts/calendar. Thank goodness I found this link; it completely made my day when I found it.

Chad Dickerson's blog

Conclusion

I hate to say it, but it seems like Palm is the thing holding me back the most. No one seems to interface with Palm Desktop, and nothing else will interface with the Palm except for Outlook. It’s hard to believe, since it seems like the Palm Treos are more popular than ever. They have been around for so long it boggles my mind that there isn’t some sort of solution for this problem.

Palm Desktop stores all of its data in a binary file. I’ve looked into reverse engineering this file to figure out if I can grab my contact’s information out it. There are a few sites that have tried something similar and posted their notes, but they always give up at some point. It just shouldn’t be that hard, I think Palm should have realized by now that their application is being worked around, they should open it more to inspire more development. Give us an API please!

As far as Yahoo goes, I know that there are tools in place for the type of application I am proposing. Plaxo already has tools where they demonstrate how they can access your contacts on Yahoo, Hotmail, Gmail, etc. There is even a Yahoo Widget that can access your Yahoo contacts. You can also buy 3rd party APIs that access these services. So why hasn’t an official API been released from Yahoo? Or even Google (they already have a calendar API)? My explanation is either these tools are hacking into the service to get the data, or they are paying some money to get exclusive access.

So in the end, I’ll leave you with my original request: Let me easily access my data so that I can share it with myself!

Monday, September 18, 2006

pghCAST

pghCAST was a ltttle project I started a while back. It was a podcast about topics going on in and around Pittsburgh. It all started after a series of events and a good conversation with friends. I started it up with one goal: at least get a free beer out of it. Well, I didn't, so I thought I might as well write about the experience.

The beginning

I've always had a good relationship with Pittsburgh. It's where I was born, and even though my family had to move away, we always keep our roots in Pittsburgh. It's also the place where I went to college – the University of Pittsburgh. After every move I have made away from Pittsburgh, I somehow ended up back in Pittsburgh. My friends call it the "rubber band theory". My most recent move away from Pittsburgh was to Seattle, where I stayed almost 3 years before the rubber band brought me back. The best way I've heard to explain my relationship with Pittsburgh is this: Pittsburgh is my mother and Seattle is my wife. I choose Seattle, but no matter what happens or changes in my life, Pittsburgh will always be there in my heart.

I would say the first seed for the show was laid when I moved back to Pittsburgh from Seattle. I've grown into a big fan of public radio, and Seattle has an excellent NPR station, KUOW (kuow.org). The station has 2-3 daily talk shows that discuss current events, usually with an expert guest and people calling in with their questions. It is a great format, and listening to their morning show made me a big fan of Steve Scher. He's really good at handling callers' different opinions, and always seems to find an elegant way of being fair to all. I've never heard another radio station (though I'm sure they exist) that has the mayor on every month to answer callers' questions, or feature all the candidates for an upcoming election.

Pittsburgh's NPR station has no such thing. Their main station, DUQ, has a lot of jazz, which I like to listen to while I'm cooking dinner, but no talk shows where I can listen to keep up with current events. I really wanted to feature something like this, and podcasting was the cheapest way I could think of to try. After discussing this with some friends in a pub, everyone thought it was a good idea. My friend Kristin even volunteered to be on the show.

You might be wondering why we started this when it seems that everyone and their brother has a podcast. Well, I did consider that angle. It seemed to me that every new podcast seemed to be targeting a national or global audience. I thought it might work to aim for a local audience - a niche that if caught on, could be a hit in an area like Pittsburgh.

The setup

The cost was extremely cheap considering my funds. The entire cost of microphones and mixers was around $300 dollars when it was all said and done. We could have done it a lot cheaper, but I wanted to try to do things right straight out of the box. My wife's opinion was that it would be worth the cost if it kept me busy for a month or two. Besides, I could always sell the equipment afterwards if things didn't work out.

While the cost of hardware was around $300 dollars (not including the computer I already had), we did had some other expenses. Mostly it was the $10 dollars per month for hosting and bandwidth from libsyn.com. They had a deal where the cost included all the bandwidth charges, which I really couldn't beat at the time.
I knew nothing about sound recording. In fact, I'd say I still know little about sound recording. I asked a friend for his input on this project; he used to be in a band and loved the production aspect of things. We went to Guitar Center and purchased 3 oval microphones with desk stands, and a 5-port mixer. I read that condenser microphones were better, but we heard that we could make these microphones sound good and the price was cheaper. Looking back, I would have spent more time up front with microphones. Some things are just easier if you spend a little more money, therefore you don't have to do as much work. The microphones were (and still are) a thorn in my side - they never sounded right to me and we couldn't figure out how to make them sound better.

For software, we used an old version of Cakewalk, which sucked. It was easy enough to pick up, but for some reason it would just stop recording in the middle of a recording session, probably because it was an old version of the software. We wouldn't find out until 15-20 minutes into recording, and let me tell you nothing takes the enthusiasm out of the room like telling someone we had to talk about the same topics again because all was lost. After a while I picked up a refurbished iRiver mp3 player/recorder (iRiver iFP-899) that had a line-in connection. It limited our editing abilities, but it didn't matter too much.

Podcasting involves putting up rather sizable mp3 files on your server and letting a potential large audience eat up your bandwidth, causing big charges from your web host. We didn't think we were going to become the number one podcast overnight, but we didn't want to be stuck with a 12 thousand dollar bandwidth bill if a particular show hit it big. For hosting the files, we chose libsyn. They were so cheap and made the entire process so easy that we could forgive some of their growing pains and hiccups. For $10 dollars a month we got hosting and unlimited bandwidth. Originally I wanted to host only the files on their site and run a separate server for the content; we were hoping to draw people to our website and have some areas where they could interact. But that plan was a little too far out, so we started with the default blog software that was supplied by libsyn. I would have loved to do more on the web side, but we just didn't have the time and resources.

It's hard to tell if a customized website would have mattered. Almost all our downloads were from iTunes, which meant that most people had the content delivered to them through a subscription. However, our website was comprised of mostly show notes, something that the user could see via iTunes anyway. We had a few links to an online store and photo-gallery, but the links were not laid out very nicely and were underused. Some of this had to do with the blog-style template system used by libsyn, but probably had the most to do with our lack of using them.

I bought pghcast.net and pghcast.com because of a Yahoo sale. That particular day I went to buy the domain names, I searched around for the best deal and ran into this sale at Yahoo for $1.99 domains. So I figured I could buy 5 years for about the same price as 1 year at other sites. Now that the show is over, I have no idea what to do with the domain names. If you're interested, let me know.

The Show

Originally, I thought I wanted to have 2 people host the show. I bought the 3rd microphone for guests, but when my friend (who helped with the microphones) expressed interested in joining me and Kristin, I figured the more the merrier. Internally this caused some problems, because everyone had a different view of what the show should do: some of our friends wanted more arguing-style debate, others didn't want that at all. If I had to do it again, I would be very cautious of how my friends participated.

At first we tried recording on Sundays, but after a while we recorded mid-week. It was much easier for us to get people over after the order of work than the disorder of a weekend. I guess we could have used Skpe, but we decided to all meet around the table and eat/drink afterwards. I definitely wanted to be on a schedule; the group and the audience would benefit from a schedule. There are a lot of choices out there on the web, and it's hard to keep someone interested for more than a few minutes at best. By setting a schedule we at least made it easy for fans of the show to know when to expect to see a new episode.

The show started on September 4th, 2005. The first couple shows basically relied on the news of the week, and to be honest, they were kind of weak. I would keep a list of topics I saw in the news during the week, and then form some sort of show from those notes. This clearly wasn't our best format - we were basically mimicking the local news. After a while we tried to make the show more personal, with a lot of commentary about local topics and events, and at the end we would interview some random person. At first we used some friends, but then we started lining up friends of friends, and eventually we started getting requests from other people. We started noticing that the format could really work, and I think if the show would have continued we would have strengthened with this format. It's amazing the stories that any random person will tell when you give them the floor.

I submitted the site to a couple of websites, and to my surprise our first pilot show had around 30 downloads. I wasn't complaining - we went from 0 downloads to 30 in the first week. The next couple weeks we were slowly rising in the ratings. It was kind of hard to get our numbers with libsyn. They showed their stats by who requested them: iTunes, odeo, direct download, etc. They also tried to filter out unique visitors by IP address, which didn't always work too well. I noticed after a while that there were "podcatching" networks that would stream our show off their site, and if multiple people listened to the show through that site, it would only register 1 listener, because of the same IP address being used over and over.

After a while we were slowly growing to almost 60 downloads per show, then 100. An amazing thing about podcasting is that most shows, whether you plan to or not, are timeless. People would discover our show around the 10th episode and then go and download all the past episodes. It got so bad that we were discussing removing the pilot episode from the list, because we felt it was so much worse than the others! The best show had to be the show immediately before the Super Bowl. Our downloads were in the 400-500 range, and even a couple months after the Super Bowl it was still getting some downloads. If we would have had some merchandise attached to the show, we probably would have made some money those few weeks.

The beginning of the end

Eventually it had to come to an end. The absolute nail in the coffin was when I found out I was moving back to Seattle. Truth be told, I was well aware of the possibility of moving before I even started the show, but I wanted to try it anyway. I had some pie-in-the-sky dreams that maybe someone would be interested in taking it over, but no one really seemed interested. This wasn't our only problem. We always had problems with enthusiasm, maybe someone was tired from working that day, or just being lazy. It was worse yet if the person came to the show but was growing bored of it. Voice recording has no pictures, but it was amazing how the recording exposed all these enthusiasm issues.

Finally, the thing that probably did us in more than the others was lack of time. One of the things I overlooked was that we were talking about things going on in the Pittsburgh area, and we really like to do those things. It didn't leave much time to organize and get together a group and talk about it. A lot of us would go for weeks at a time doing something in the city every night. The plays, concerts, the sport games all took their toll. If we missed recording the show one week, it seemed all the more likely that we would miss the next week.

Eventually I had to be done with it. I recorded our last episode, alone, about a month after the previous show. I made no promises, but I said basically the show was over.

Lessons learned

Some of the biggest things I learned from this experience:

* If you are serious about it, it's a lot of work.

I thought it would be a simple thing to gather a few things to talk about, get some friends together, and record the show. It was simple, but I had no idea how long it would take to do all those things. I often found myself not as prepared as I thought I was before a show. Then there's the actual recording of the show: sound levels, microphone placement, EDITING, compressing, uploading, creating show notes, etc. After all this, I barely had any time to think about designing the website.

* It's hard to do by yourself, but if you work with others you need to know where you and they stand.

I was very happy to have my friends involved. In some cases I was willing to let someone slack in one area because they were so good in another. However, I think we all underestimated the amount of work. I didn't set their expectations correctly because I didn't know what to expect. After a couple of weeks, I was asking them to help find stories on their own and do more than they thought they were signing up for. If I had to do it again, I would definitely try to set that expectation right away. Then again, I don't think you can ever get expectations correct - there's always something in any project that doesn't go as planned, and you need a team that is flexible and truly understands how to react.

* Spend some time (but not too much).

I would have definitely spent more time on show development up front. That's not to say that we didn't do any - we actually had a few meetings where we brainstormed some ideas for topics for slow weeks, etc. However, I think we probably could have used a little more time on the technical side. I don't blame us, we were eager to get started and I didn't want it to get old before even starting. I will also contradict myself in the same vein and say that you should also not plan too long - it's a amateur podcast for God's sake, it's not going to be perfect.

* Watch your shortcuts.

Be careful where you make shortcuts. Sometimes spending a little more time or money can provide a huge benefit down the line.

* I'm glad I did it!

Even though I had a good idea that I would be moving, I went ahead and started the show anyway. I wasn't sure what to think of it, but we got consistently 50-60 downloads a week, and around the Super Bowl we were in the 400-500 range. All these downloads just to hear our silly show.

The emails really made me feel like someone was listening and getting a benefit. We got a few emails from people who didn't even live in Pittsburgh anymore, and just wanted to write to say that listening to our show reminded them of their old home. Others would write to respond to particular topics in the show, or how much they laughed out loud (in public) at some story we told while they were listening on their headphones. Those stories always made me feel like we were contributing.


Conclusion

Would I do it again? If the conditions were right, I definitely would. I've learned by this point in my life that I'm not the type to sit around idling by and not get involved with something. As soon as some free time shows up, I have a way of eventually finding something to fill it up. Whether it's recreational, volunteering for public causes, or helping friends with their projects. There's a million things I'd love to have the time to learn, but I have to keep a cap on things or I'd never get anything done.

Tuesday, May 30, 2006

Project blog

Well I guess it’s come to this. I figured that with all the “blog knowledge” going on in the world today, that the world really didn’t have any room for me. They say that most blogs are written by semi-pro/semi-amateur writers looking for an outlet. I thought about that for a while, because I’ve always kind of liked the semi-amateur writing style I have. So I figured I would finally answer all my fans’ requests to start writing a blog. All zero of you should be pretty happy about now.


No really, the real reason I decided to do this was after reading a post on a mailing list I like to read about Linux users in Pittsburgh (WPLUG). The person (he whom must not be named because I’ve become too lazy to switch between windows) mentioned started writing a blog about the projects he was part of, their problems, solutions, etc. It had two goals: provide some insight into their project for others to reference, and (I’d say mostly) to provide a sort of real-world resume for things he has worked on.

I immediately liked the idea - I had just read an article somewhere about more and more companies Googling job applicants to see what they have in their past. If this article sounds familiar - we probably didn’t read the same one. I think I might have read about 5-8 articles with this subject over the past couple years. Anyway, the article was obviously spelling out doom for perspective job-seekers. “Oh my god, look at this - he dressed up as a bunny on Halloween, he might cry if we pressure him too much!” Anyway - that’s not the point. My point is that there are two sides to every story - and my take on this story is that if a company cares so much to read past my resume, that they might actually find out more about my human side and find it to be a better fit.

It’s like my little photo-album on my website. I obviously take WAY more pictures, but I put a few up that I really like. I’m not a professional photographer - I’m obviously a computer geek who takes a million pictures and only shows the one that happened to look like I’m a professional. You should know the type - there are a million of us out there.

Anyway, back to the photo-album. I had a job interview that went pretty well. At the end I met the CEO - who only seemed to look at my website, not much else. He sat there and made guesses as to where each picture was taken. If you haven’t seen my album, it’s brilliantly simple (actually I was lazy, but I’ll take brilliantly simple if you’d let me) . Back to the point - the album did exactly what I wanted; it showed a side of me and instilled conversation.

So it's been a while and I’m thinking of trying the same thing on a sort of journal. Only now these journals are called blogs. Hope you read some more, let me know if you have any comments. Though no promises - I’m not sure if I want to use blog software, or just write a little code.