Off the Top: E-mail Entries
After seven to eight years with the same web host a move is underway again. This will be slower and it may hit reverse for parts. Having read Phil Gyford’s recounting his host move I think I am prepared for the scope of what this move entails.
In my time with this host I put up dev projects that then moved to their own servers, I experimented with a lot of different tools and services that I continue to use (or would love to get back to using), and I have a boat load of mail accounts set up for contextual use and family.
The are a few domains, a lot of subdomains, many databases, and many applications (my hand built and third party creations). I’m expecting to have to tweek some code and scripting to handle different versions of PHP and Python than I currently run.
Two Host Mambo
The move is splitting things to two different hosts (one a VPS and the other a modern hosted web host). The split is to give some of the more resource heavy applications like Fever and ThinkUp room to flex and grow, these are applications that only I use and are really helpful for a variety of reasons. Other applications I have run locally on a Mac Mini will move there as will old services I shuttered due to lack of web resources availalbe.
The bulk of what is here at vanderwal.net and publicly available will move to a web host with good management consoles and mail (for now). While I am comfortable living and working at command line, I like a good console to make things easier. Also having someone else doing security updates, patching, and upgrades is nice as well so I can focus on the things I really want to do.
Mail is something that I really want to improve. Having to pester to have somebody give the mailserver a kick to get mail functioning again is something that shouldn’t need to happen. I have run my own mail server in the past (10 years back) and I know not to do that again as it was a pain and that pain is much worse with dealing with SPAM as well as getting on the wrong side of a SPAM blacklist (difficult to get off of a blacklist or greylist for domain or IP block when you are fewer than a few thousand accounts). The days of small email services died 5 to 10 years ago and that gets worse with each passing year.
Mail for the moment is going with my web hosting. This is less than optimal, I know that well. But, a good sized web host has the bulk to deal with black / grey listing to some degree. I know my optimal choice is FastMail, but making that move will take more decisions to narrow down mail accounts and sorting the size I am willing to limit some accounts to.
This only gets real when the domains get repointed and propigate. That is not today.
Communicating is simple. Well it is simple at its core of one person talking with another person face-to-face. When we communicate and add technology into the mix (phone, video-chat, text message, etc.) it becomes more difficult. Technology becomes noise in the pure flow of communication.
Now With More Complexity
But, what we have today is even more complex and difficult as we are often holding conversation across many of these technologies. The communication streams (the back and forth communication between two or more people) are now often not contained in on communication channel (channel is the flavor or medium used to communicate, such as AIM, SMS, Twitter, e-mail, mobile phone, etc.).
We are seeing our communications move across channels, which can be good as this is fluid and keeping with our digital presence. More often than not we are seeing our communication streams fracture across channels. This fracturing becomes really apparent when we are trying to reconstruct our communication stream. I am finding this fracturing and attempting to stitch the stream back together becoming more and more common as for those who are moving into and across many applications and devices with their own messaging systems.
The communication streams fracture as we pick-up an idea or need from Twitter, then direct respond in Twitter that moves it to SMS, the SMS text message is responded back to in regular SMS outside of Twitter, a few volleys back and forth in SMS text, then one person leaves a voicemail, it is responded to in an e-mail, there are two responses back and forth in e-mail, an hour later both people are on Skype and chat there, in Skype chat they decide to meet in person.
Why Do We Want to Stitch the Communication Stream Together?
When they meet there is a little confusion over there being no written overview and guide. Both parties are sure they talked about it, but have different understandings of what was agreed upon. Having the communication fractured across channels makes reconstruction of the conversation problematic today. The conversation needs to be stitched back together using time stamps to reconstruct everything [the misunderstanding revolved around recommendations as one person understands that to mean a written document and the other it does not mean that].
Increasingly the reality of our personal and professional lives is this cross channel communication stream. Some want to limit the problem by keeping to just one channel through the process. While this is well intentioned it does not meet reality of today. Increasingly, the informal networking leads to meaningful conversations, but the conversations drifts across channels and mediums. Pushing a natural flow, as it currently stands, does not seem to be the best solution in the long run.
Why Does Conversation Drift Across Channels?
There are a few reasons conversations drift across channels and mediums. One reason is presence as when two people notice proximity on a channel they will use that channel to communicate. When a person is seen as present, by availability or recently posting a message in the service, it can be a prompt to communicate. Many times when the conversation starts in a presence channel it will move to another channel or medium. This shift can be driven by personal preference or putting the conversation in a medium or channel that is more conducive for the conversation style between people involved. Some people have a preferred medium for all their conversations, such as text messaging (SMS), e-mail, voice on phone, video chat, IM, etc.. While other people have a preferred medium for certain types of conversation, like quick and short questions on SMS, long single responses in e-mail, and extended conversations in IM. Some people prefer to keep their short messages in the channel where they begin, such as conversations that start in Facebook may stay there. While other people do not pay attention to message or conversation length and prefer conversations in one channel over others.
Solving the Fractured Communication Across Channels
Since there are more than a few reasons for the fractured communications to occur it is something that needs resolution. One solution is making all conversations open and use public APIs for the tools to pull the conversations together. This may be the quickest means to get to capturing and stitching the conversation thread back together today. While viable there are many conversations in our lives that we do not want public for one reason or many.
Another solution is to try to keep your conversations in channels that we can capture for our own use (optimally this should be easily sharable with the person we had the conversation with, while still remaining private). This may be where we should be heading in the near future. Tools like Twitter have become a bridge between web and SMS, which allows us to capture SMS conversations in an interface that can be easily pointed to and stitched back together with other parts of a conversation. E-mail is relatively easy to thread, if done in a web interface and/or with some tagging to pull pieces in from across different e-mail addresses. Skype chat also allows for SMS interactions and allows for them to be captured, searched, and pulled back together. IM conversations can easily be saved out and often each item is time stamped for easy stitching. VoIP conversations are often easily recorded (we are asking permission first, right?) and can be transcribed by hand accurately or be transcribed relatively accurately via speech-to-text tools. Voice-mail can now be captured and threaded using speech-to-text services or even is pushed as an attachment into e-mail in services as (and similar to) JConnect.
Who Will Make This Effortless?
There are three types of service that are or should be building this stitching together the fractured communications across channels into one threaded stream. I see tools that are already stitching out public (or partially public) lifestreams into one flow as one player in this pre-emergent market (Facebook, Jaiku, etc.). The other public player would be telecoms (or network provider) companies providing this as a service as they currently are providing some of these services, but as their markets get lost to VoIP, e-mail, on-line community messaging, Second Life, etc., they need to provide a service that keeps them viable (regulation is not a viable solution in the long run). Lastly, for those that do not trust or want their conversation streams in others hands the personally controlled application will become a solutions, it seems that Skype could be on its way to providing this.
Is There Demand Yet?
I am regularly fielding questions along these lines from enterprise as they are trying to deal with these issues for employees who have lost or can not put their hands on vital customer conversations or essential bits of information that can make the difference in delivering what their customers expect from them. Many have been using Cisco networking solutions that have some of these capabilities, but still not providing a catch all. I am getting queries from various telecom companies as they see reflections of where they would like to be providing tools in a Come to Me Web or facilitating bits of the Personal InfoCloud. I am getting requests from many professionals that want this type of solution for their lives. I am also getting queries from many who are considering building these tools, or pieces of them.
Some of us need these solutions now. Nearly all of us will need these solutions in the very near future.
If you want to send an e-mail that gets my attention, please use an address other than my Gmail address. I mostly use that address for listserves. The ability to search, parse, and scan e-mail in Gmail just does not work for me and things I really want to follow-up with only get addressed if I forward them to myself at an other address. [Granted the amount of e-mail I am getting and daily communication is more then I can normally keep-up with at the moment. I deeply apologize if I owe you a response. I need to better embrace the DTD model as my
Filters, Labels, and Tags
The Gmail interface does not work well for me personally to highlight, track, and respond to the mail. I had a lot of hope for Gmail and its ability to tag (or in Google terms, "label"), but its interface is really poor for doing this with anything more than 10 or 15 labels. When I want to manually applying more than one label the interface is really poor (at best).
I have looked at the GTDGMail mail as a solution, as its interface is much much better than what Google has churned out. While the GTDGmail is a vast improvement the remainder of Gmail for personal or work mail does not scale to meet my needs on that front. If you are unpleased with the Gmail labelling, as most I know are, you owe it to yourself to look at GTDGmail.
I get to spend my Halloween morning clearing out 28,445 junk/spam e-mail from my junk folder on my server. It made up 580MB of garbage. I did not pay attention to the junk folder for 45 days and this is what I get.
I was thinking of moving all of the listserves I track to that server as the Google Gmail interface is horrible for the volume of mail that comes in to those (largely because I can not unbold, or scan easily) and are filtered into their respective spots (for the filters that are not broken). The volume of listserve mail is getting to be beyond usable. But, I mostly search those archives when I am stumped (rarely is the answer there and I turn to the experts - most are no longer paying attention to the listserves any longer). I have come close in the past few weeks from pulling the plug on most of the listserve I subscribe to as most questions in them have answers in the first 5 or 10 Google or Yahoo search results.
I thought this would be the year I was going to ETech, but with a few other things going on it was not the year. I have many friends that go each year and I see them very rarely.
But, I think I would have been very frustrated by ETech this year. It is still about the web. Achingly, still about the web. The problem is digital information and media is increasingly living beyond the web. The web is but one platform to distribute information, but thinking people live their lives in and on the web is silly. Want the information that is on the web, but need the information in their lives, in their devices they have with them, and in context to the rest of their life.
The panel that triggered this reaction is one by friends, Jesse and Jeff "Designing the Next Generation of Web Apps". In Tom Coates review the binary approach (web for reading and web for apps) sounds so short sited and really caused the trigger. Is Emerging Tech just rehashing the current and the past? Or can it move forward? I am not seeing much of that forward movement this year.
People live their lives attracting information and focussing on the Come to Me Web and Personal InfoCloud we know people need the information to better mesh into their personal digital information workflow, which involves very little of the web. People find the information that they want and need and work very hard to keep it attracted to themself for easy refindability. Other than social bookmarking tools and a few others web based tools, much of this is done with tools that are beyond the web. Some people tuck all of their needed information and links into e-mail, others calendar, to do lists, PIMs, text files, syndication, e-mail, SMS, MMS, documents, mobile syndication, mobile documents, outlines, wiki on a stick, etc. There are many tools and many ways of working around lack of web access when people need the information most.
Many people, unlike those of us that build web-based tools (I am in that category), don't live on the web and their digital information needs to live beyond the web as well. That is the future of the web, it is a platform for just one state of information. That state that the web represents is the state of information transience. The information is in the process of moving from the creator to the person needing that information for their own use or for their reuse. This use will most likely not be on the web, but the reuse of information may be on the web.
The web as it exists now is a tool for publishing and aggregating. Some will use the web for use and reuse, but we need far more options that the web for real people to adopt their future and our now. We, as developers of tools, information, and resources must pay attention to real people. We must pay attention to their lives beyond the web and the large box in front of them. We need to understand their problems that they really have, which revolve around refindability and information reuse in their environment and context.
Now please go back to paying careful attention to the great things that friends and other alpha geeks are presenting at ETech and other conferences and un-conferences as that information is needed, but remember we are moving beyond, far beyond this current state of the web.
[update] Um, well Ray Ozzie just made me wish I was at ETech. He just showed what is emergent and what is the future. It could answer many of the items I just listed above. You go Ray!
Most of the e-mail coming in is now coming through the new hosting company. I am getting feedback from people that they can see this new site. The change of the domain to the new hosting started resolving correctly in 30 minutes for me, which was very fast.
The choice in hosting came down to two strongly recommended companies, Pair.com and Segment Publishing (SegPub). Both had nothing, but positive recommendations and each had more than four recommendation (Pair actually had over 10 recommendations). Both offered the services I wanted and needed (PHP, Python, secure mail, MySQL, solid up-time records, and Rails (Rails was more of a nice than a need)).
I chose SebPub based on two things that separated it from Pair. One was price, SegPub was a little less and offered a good introduction and trial period. It is an Australian company (servers in the US) and payments are in Australian dollars. Secondly, the first person to recommend SebPub put me in contact with the guy who runs SegPub in a chat. I was able to asks a lot of questions specific to my needs and he was able to easily point me to the needed information or just provide it in the chat. That personal touch was an incredible sell. Just knowing there is that access helped. I had really grown tired of waiting a few hours between responses to trouble tickets and e-mail exchanges and not getting resolution and just getting the ticket closed.
Getting things prepared and moved over took the usual amount of time, but I had far fewer bumps than I thought I would have had. I needed to better optimize MySQL to perform better in MySQL4. I was to the point I needed to make the hosting change and the time disruption was not welcome, but I knew I could focus when the transition was over and done.
So far I have been incredibly happy. I also know that Pair would also be a very good company to go to as well. This option gives me great peace of mind.
Super spam build-up. Thursday night I did my weekly pull of my stored "junk" e-mail, which was more like 10 days of build-up. I had found some legitimate bounced e-mail during my past round when I scanned through the mess, so I was saving up until I had a little time to visually parse through the pile. This time I had 31,000 items in the junk mail bin. I did not even look in the output bin this time, I just did a straight push to delete and dumped the trash.
If you sent something that you did not get a response on that you believe should get a response lets try this again. Send it. I have around 15 e-mails I have been working on longer responses to, but am going to be sending, "I got it" responses then put it in my longer response queue.
One of the things that bubbled up while using a phone other than my usual phone on my trip to Amsterdam was my normal use patterns. I use my mobile phone (Palm Treo) not only to communicate with others, but also to communicate with myself.
I often send myself e-mails with ideas for projects or articles. I also send e-mails to myself on things to spend time thinking about later or to research later. I had one of these moments in talking with Peter Bogaards about Paul Otlet, whom I have problems remembering. I started reaching for my tool to send a note and had a different phone, which took a little more effort, but got the job done.
I also SMS myself with the cross streets of where I parked my car, or what garage level and other metadata. This solves an ongoing problem, particularly in Georgetown, where my normal parking patterns are ousted, I am parking in a completely new location, or parking in long term at an airport. All part of my mind occasionally not being on the task at hand or I am just deep in thought and did not make a conscious effort to remember where I parked.
One other usage I have is taking a photo of my radio display in my car or the song name and performer. This mostly happens when listening to XM Radio (satellite) as the options are broad and deep, which works wonders for coming across new music I like. The radio in my car does not have a means to save the song name, or even better would be to e-mail me the song info, a link to listen to a snippet, and buying information. I resort to the tools at hand. Yes, I only (well usually) take photos while stopped. I usually have 5 to 10 photos of my radio on my mobile at any one time. It makes it handy when I go into a music store or wait until I can put it in a wishlist online. I use my Amazon Wishlist to find items of interest in a store or to add to my Wishlist and having that live access has made things far better for me as a consumer (my only desire is to have a better mobile interface to one's own Wishlist on a mobile it would seem to have immense benefits for Amazon).
I also browse the web and read articles that I save down to my device to I have access in no-signal zones or zones that I do not have MMS or web access and just phone (DC METRO while underground). I often save down Boxes and Arrows articles for such trips. I also push articles and presentations I am working on to my mobile as back up as well as for review in down periods.
How do you use your mobile?
Since I have been playing with Gmail I have been greatly enjoying the greatly improved means of labeling and archiving of e-mail as opposed to throwing them in folders. Many e-mails are hard to singularly classify with one label that folders force us to use. The ability to drive the sorting of e-mail by label that allows the e-mail to sit accessibly under a filter named with the label make things much easier. An e-mail discussing CSS, XHTML, and IA for two different projects now can be easily accessed under a filter for each of these five attributes.
Dan Brown has written a wonderful article The Information Architecture of Email that dig a little deeper. Dan ponders if users will adopt the changed interface. Hearing many user frustrations with e-mail buried in their Outlook or other e-mail application, I think the improved interface may draw quite a bit of interest. As Apple is going this way for its file structure in Tiger (the next OS upgrade) with Spotlight it seems Gmail is a peak at the future and a good means to start thinking about easier to find information that the use can actually manage.
I picked up a Gmail account over this long weekend. What , how did I get it? I bought it. Yes, I know they are free and I know it is still in beta. Yes, I got it from an auction site. No I do not think I am crazy.
- I subscribe to a lot of lists and I also get an incredible amount of e-mail to my personal address
- I do not have external SMPT access during the work day to post queries to lists or to quickly respond to mail.
- I did not have a Web mail account that allowed me to search or organize the e-mail as I wanted (searching through months if not years of list services is very helpful)
- I wanted to make sure I got a specific name
- I can use my mobile account to deal with personal e-mail and forward all other e-mail to Gmail
- I don't mind the advertising and having targeted ads is better than the garbage I don't care about
So far I am quite impressed with the interface. There are some things with the application that I was not expecting, such as spelling. I was also not expecting the labels for e-mails rather than silly folders. The labels allow for more than one category for each e-mail and the mail is not buried in a folder somewhere.
Outbound SMTP e-mail has been blocked at work, so I will most likely respond using my mobile account for personal e-mail that needs addressing during business hours. This restriction is understandable, but also limits the ability to shoot a quick e-mail to say I will be meeting for dinner this evening at place X. I loathe the telephone and find e-mail more usable as it captures the information for reuse and later referral, where as the phone does not permit this ease of reuse, or ease of accurate reuse as may be more appropriate.
There is another e-mail redesign in the work, IBM's Remail is a research project that seems to be on the right track. There is serious need for outlook replacements so that we can have a mail, calendar, and other communication tool that actually works as one needs it to.
There are two articles that are direct hits on managing information for the individual and allowing the individual to use the information when they needed it and share it as needed. Yes, this is in line with the Personal Information Cloud.
The first article, The inter-personal information manager (iPim) by Mark Sigal about the problem with users finding information and how the can or should be able to then manage that information. There are many problems with applications (as well as the information format itself) that inhibit users reuse of information. In the comments of the article there is a link to products that are moving forward with information clients, which also fit into the Personal Information Cloud or iPIM concept. (The Personal Information Cloud tools should be easily portable or mobile device enabled or have the ability to be retrieved from anywhere sent to any device.
The second article is from the MIT Technology Review (registration required) titled Trash Your Desktop about Mitch Kapor (of founding Lotus Development fame) and his Open Source project to build Chandler. Chandler is not only a personal information manager (PIM), but the tool is a general information manager that is contextually aware. The article not only focusses on Mitch and the product (due late 2004), but the open and honest development practices of those that are building Chandler at the Open Source Application Foundation for Windows, Mac, Linux, etc. distribution.
Ray Ozzie discusses the death of e-mail as a work process tool. Of course Ray has an interest in this as his Groove application provides encrypted shared workspaces for workflow and sharing. If you rely on e-mail for document sharing or an Intranet, Groove is a large step above and beyond these technologies. E-mail was not designed, many many years ago, for the type of tasks and volumes that are required of it today. As every work environment struggles with privacy and security most e-mail solutions do not provide a sufficient level of support, particularly with e-mail storage limitations.
E-mail also does not often provide portability and tracking across various work environments. Groove however does do this. I was testing and using Groove in a beta mode and the free version a year or two ago. Groove had these capabilities then.
Yes, secure e-mail is available on many e-mail platforms, but the portability and retention of state of work does not work as easily on other applications. The one downside of Groove for me is it does not yet have an OS X version,
Amit has launched Spamotomy, a site focussed on the complete removal of SPAM from our lives. The site keeps up on news as well as pointers to nearly 100 tools for SPAM removal and so much more.
Important! This system is beta and not production-ready. You may experience errors and other problems!It seems to remember that there are other problems than errors. (?)
I have been looking for a solid e-mail client for quite some time. I seem to have found one that I really like, MS Entouragge for Mac OS X. Entourage is much farther ahead of Outlook in that you can not only flag an e-mail, but set a reminder in your calendar to come back to that e-mail at a later time. The ability to set categories in addition to rules and other elements is a great help. I find it easy to use and work with.
The competition has left me cold to some degree. Netscape 6(plus) is light years ahead of version 4 (did not allow pulling from multiple e-mail sources), begins to really choke with a couple thousand e-mails in a folder (the downside of being on some wonderful listserves that are full of great information). Outlook Express does not permit archiving. Outlook on Windows has a muddled interface that works well with Exchange and that is not completely a plus.
In all I am very impressed with MS Office on Mac OS X. That is my "productivity tool" of choice. It loads faster and the application does not get in the way of doing what you want it to do, although "Clippy's" cousin is alive and still annoying.