April 13, 2014

Site Shift has Started

After seven to eight years with the same web host a move is underway again. This will be slower and it may hit reverse for parts. Having read Phil Gyford’s recounting his host move I think I am prepared for the scope of what this move entails.

In my time with this host I put up dev projects that then moved to their own servers, I experimented with a lot of different tools and services that I continue to use (or would love to get back to using), and I have a boat load of mail accounts set up for contextual use and family.

The are a few domains, a lot of subdomains, many databases, and many applications (my hand built and third party creations). I’m expecting to have to tweek some code and scripting to handle different versions of PHP and Python than I currently run.

Two Host Mambo

The move is splitting things to two different hosts (one a VPS and the other a modern hosted web host). The split is to give some of the more resource heavy applications like Fever and ThinkUp room to flex and grow, these are applications that only I use and are really helpful for a variety of reasons. Other applications I have run locally on a Mac Mini will move there as will old services I shuttered due to lack of web resources availalbe.

The bulk of what is here at vanderwal.net and publicly available will move to a web host with good management consoles and mail (for now). While I am comfortable living and working at command line, I like a good console to make things easier. Also having someone else doing security updates, patching, and upgrades is nice as well so I can focus on the things I really want to do.

Mail

Mail is something that I really want to improve. Having to pester to have somebody give the mailserver a kick to get mail functioning again is something that shouldn’t need to happen. I have run my own mail server in the past (10 years back) and I know not to do that again as it was a pain and that pain is much worse with dealing with SPAM as well as getting on the wrong side of a SPAM blacklist (difficult to get off of a blacklist or greylist for domain or IP block when you are fewer than a few thousand accounts). The days of small email services died 5 to 10 years ago and that gets worse with each passing year.

Mail for the moment is going with my web hosting. This is less than optimal, I know that well. But, a good sized web host has the bulk to deal with black / grey listing to some degree. I know my optimal choice is FastMail, but making that move will take more decisions to narrow down mail accounts and sorting the size I am willing to limit some accounts to.

Getting Real

This only gets real when the domains get repointed and propigate. That is not today.

April 6, 2014

Changing Hosts and Server Locations

I’ve been in the midst of thinking through a web host / server move for vanderwal.net for a while. I started running a personal site in 1995 and was running it under vanderwal.net since 1997. During this time it has gone through six of 7 different hosts. The blog has been on three different hosts and on the same host since January 2006.

I’ve been wanting better email hosting, I want SSH access back, more current updates to: OS; scripting for PHP, Ruby, and Python; MySQL; and other smaller elements. A lot has changed in the last two to three years in web and server hosting.

The current shift is the 4th generation that started with simple web page hosting with limited scripting options, but often had some SSH and command line access to run cron jobs. The second was usually had a few scripting options and database to run light CMS or other dynamic pages, but the hosting didn’t give you access to anything below the web directory (problematic when trying to set your credentials for login out of the web directory, running more than one version of a site (dev, production, etc.), and essential includes that for security are best left out of the web directory). The second generation we often lost SSH and command line as those coming in lacked skills to work at the command line and could cripple a server with ease with a minor accident. The third has been more robust hosting with proper web directory set up and access to sub directories, having multiple scripting resources, having SSH and command line back (usually after proven competence), having control of setting up your own databases at will, setting up your own subdomains at will, and more. The third generation was often still hosting many sites on one server and a run away script or site getting hammered with traffic impacted the whole server. These hosts also often didn’t have the RAM to run current generations of tools (such as Drupal which can be a resource hog if not using command line tools like drush that thankfully made Drupal easier to configure in tight constraints from 2006 forward).

Today’s Options

Today we have a fourth generation of web host that replicates upgraded services like your own private server or virtual private server, but at lighter web hosting prices. I’ve been watching Digital Ocean for a few months and a couple months back I figured for $5 per month it was worth giving it a shot for some experiments and quick modeling of ideas. Digital Ocean starts with 512 MB or RAM, 20GB of SSD space (yes, your read that right, SSD hard drive), and 1TB of transfer. The setup is essentially a virtual private server, which makes experimentation easier and safer (if you mess up you only kill your own work not the work of others - to fix it wipe and rebuild quickly if it is that bad). Digital Ocean also lets you setup your server as you wish in about a minute of creation time with OS, scripting, and database options there for your choosing.

In recently Marco Arment has written up the lay of the land for hosting options from his perspective, which is a great overview. I’ve also been following Phil Gyford’s change of web hosting and like Phil I am dealing with a few domains and projects. I began looking at WebFaction and am liking what is there too. WebFaction adds in email into the equation and 100GB of storage on RAID 10 storage. Like Digital Ocean it has full shell scripting and a wide array of tools to select from to add to your server. This likely would be a good replacement for my core web existence here at vanderwal.net and its related services. WebFaction provides some good management interfaces and smoothing some of the rough edges.

There are two big considerations in all of this: 1) Email; 2) Server location.

Email

Email is a huge pain point for me. It should be relatively bullet proof (as it was years ago). To get bullet proof email the options boil down to going to a dedicated mail service like exchange or something like FastMail, a hosted Exchange server, or Google Apps. Having to pester the mail host to kick a server isn’t really acceptable and that has been a big reason I am considering moving my hosting. Also sitting on servers that get their IP address in blocks of blacklisted email servers (or potentially blacklisted) makes things really painful as well. I have ruled out Exchange as an option due to cost, many open scripts I rely on don’t play well with Exchange, and the price related to having someone maintain it.

Google Apps is an option, but my needs for all the other pieces that Google Apps offers aren’t requirements. I am looking at about 10 email addresses with one massive account in that set along with 2 or 3 other domains with one or two email accounts that are left open to catch the stray emails that drift in to those (often highly important). The cost of Google for this adds up quickly, even with using of aliases. I think having one of my light traffic domains on Google Apps would be good, the price of that and access to Google Apps to have access to for experimentation (Google Apps always arise in business conversations as a reference).

FastMail pricing is yearly and I know a lot of people who have been using it for years and rave about it. Having my one heavy traffic email there, as well as tucking the smaller accounts with lower traffic hosted there would be a great setup. Keeping email separate from hosting give uptime as well. FastMail is also testing calendar hosting with CalDAV, which is really interesting as well (I ran a CalDAV server for a while and it was really helpful and rather easy to manage, but like all things calendar it comes with goofy headaches, often related timezone and that bloody day light savings time, that I prefer others to deal with).

Last option is bundled email with web hosting. This has long been my experience. This is mostly a good solution, but rarely great. Dealing with many domains and multitudes of accounts email bundled with web hosting is a decent option. Mail hosting is rarely a deep strength of a web hosting company and often it is these providers that you have to pester to kick the mail server to get your mail flowing again (not only my experience, but darned near everybody I know has this problem and it should never work this way). I am wondering with the benefits of relatively inexpensive mail hosting bundled into web hosting is worth the pain.

I am likely to split my mail hosting across different solutions (the multiple web hosts and email hosts would still be less than my relatively low all in one web hosting I currently have).

Server Location

I have had web hosting in the US, UK, and now Australia and at a high level, I really don’t care where the the servers are located as the internet is mostly fast and self healing, so location and performance is a negligible distance for me (working with live shell scripting to a point that is nearly at the opposite side of the globe is rather mind blowing in how instantaneous this internet is).

My considerations related to where in the globe the servers are hosted comes down to local law (or lack of laws that are enforced). Sites sitting on European hosts require cookie notifications. The pull down / take down laws in countries are rather different. As a person with USA citizenship paperwork and hosting elsewhere, the laws that apply and how get goofy. The revelations of USA spying on its own people and servers has me not so keen to host in the US again, not that I ever have had anything that has come close to running afoul of laws or could ever be misconstrued as something that should draw attention. I have no idea what the laws are in Australia, which has been a bit of a concern for a while, but the host also has had servers in the US as well.

My options seem to be US, Singapore, UK, Netherlands, and Nordic based hosting. Nearly all the hosting options for web, applications, and mail provide options for location (the non-US options have grown like wildfire in the post Edward Snowden era). Location isn’t a deciding point, but it is something I will think through. I chose Australia as the host had great highly recommended hosting that has lived up to that for that generation of hosting options. It didn’t matter where the server was hosted eight years ago as the laws and implications were rather flat. Today the laws and implications are far less flat, so it will require some thinking through.

Non-UNIX Timestamping has Me Stamping in Frustration

In the slow process of updating things here on this site I have nearly finished with the restructuring the HTML (exception is the about page, which everytime I start on it the changes I start to make lead quickly to a bigger redesign).

Taking a break from the last HTML page restructuring I was looking at finally getting to correctly timestamping and listing the post times based on blog posting location. Everything after the 1783rd blog post currently picks up the server time stamp and that server is sitting in Eastern Australia, so everything (other than blog posts from Sydney) are not correctly timestamped. Most things are 14 to 15 hours ahead of when they were posted - yes, posted from the future.

Looking at my MySQL tables I didn’t use a Unix timestamp, but a SQL datetime as the core date stored and then split the date and time into separate varables created with parsing timestamp in PHP. This leaves 3 columns to convert. It is a few scripts to write, but not bad, but just a bit of a pain. Also in this change is setting up time conversions that are built into the post location, but shifts in time for day light savings starts adding pain that I don’t want to introduce. I’ve been considering posting in GMT / UTC and on client side showing the posting time with relative user local time with a little JavaScript.

I would like to do this before a server / host move I’ve been considering. At this point I may set up a move and just keep track of the first post on the new server, then at some point correct the time for the roughly 500 posts while the server was in Eastern Australia.

March 16, 2014

Mind the Construction Dust

I’m in the midst of a structuring here across all the pieces of vanderwal.net. It started in January with another project, a meetp-up hack to dive into Zurb Foundation. Within a couple weeks of starting down that path I decided it would be fine time to rebuild and redesign vanderwal.net using Foundation. Before I started down the road leaving the horses behind I desided I was going to update the structure of the HTML of these pages and bring them into modern times with HTML5 and CSS3.

This thinking and tinkering has been finally fixing some of the underlying details that bugged me, but it also allowed to set a much better and more object focussed semantics. This shift will also enable the content objects to flow better and be better foundations for a redesign as well.

While I have no idea what the redesign will be and not even thinking of that, I did find the original photo that I modified to be used for the header image and I put that to the pages I have touched. The new image now is much wider to allow for a fluid page and the “vanderwal.net” text is now out of the image and I a truly proper H1, that has alluded me for a long time (and bugged me to no end). The menu of the updated pages has brought back the selected portion of the site with a bleed to page, which was there at the beginning, but some shift in CSS caused it to go away.

I may, possibly likely, shift hosting at some point in the near future, but that may wait until I have some of the underpinnings of the blog tool updated a little. Some of those changes will wait a little, but have been brewing a long time. I don’t think I am bringing comments back, but will likely bring in web mentions (Jeremy Keith has a great explaination). There is a lot going on in the IndieWeb that has been inspiring and may trigger some more changed that I have longed for to finally get put in place.

BTW, this is the short version of this. Two prior attempts at writing up something short both ended up over 2,500 words.

January 16, 2014

OmniOutliner Counts to Four

One of my favorite applications that a lot of my work and workflow lives in and through, OmniOutliner, updated today. OmniOutliner 4 finally was released today. Its interface becomes a little easier to use for more advanced functions, but if you use the iPad version the new Mac version now looks and works a little more like the iPad version (I think this is a good thing for consistency and ease of use).

I have been using OmniOutliner since version one. I learned to think and organize in outlines and I loved in the old days of WordPerfect the start a document in an outline and then start fleshing it out allowed me to work in the same manner I learned in the fourth grade in Mrs. Norman’s class at Raleigh Park Elementary. This seemed natural to prepare writing this way and once WordPerfect went missing from my workflow other writing tools faked outlines and I looked for good outlining tools to be that foundation. OmniOutliner filled this void. But, once I found OmniOutliner I found other fans who had scripted it to do really helpful tasks, like capture web site maps and dump them into OmniOutliner to annotate and arrange them, then use a script to push into OmniGraffle to visualize. Doing this in 2003 (or so) was pure joy. Not only was was OmniOutliner easy to use, it was really powerful because it was well structured and scriptable.

OmniOutliner is Where I Think

About 2003 I was asked by friend Jesse James Garret, “What tool to you think in?” At that point my answer was OmniOutliner. OmniOutliner was my capture tool that allowed for easy structuring and arranging of order. In years to come with OPML becoming the glue to connect many things in my workflow, I would would move my outlines from OmniOutliner to a mind mapping tool and back and forth. This moving the outline into a mind map allowed me to see it and see relationships spatially and to identify order, modify structure, and make connections between nodes in different branches of the mind map. From the mind map I could take all the modifications and move them back into the outline and tweak a little more. From this point it was moving into writing or into a Keynote presentation (also with a script that would take the OmniOutline and convert it to a presentation to flesh out visually).

The Initial Foundation of What Became OmniFocus

With OmniOutliner I went through the early productivity layer for it that later turned into OmniFocus. My old business started and was kept on schedule in that precursor to OmniFocus that Ethan Schoonover cobbled into and on top of OmniOutliner that was called Kinkless GTD (or KGTD for short).

I still think in OmniOutliner. I have all of the (now) 54 elements of the social lenses tucked in there with their hundreds of sub-nodes. This outline is what became the initial foundation for the four days of walk through of them with Dave Gray for what would turn into the Connected Company book. The collection of similar outlines are all within easy reach. I have a saved Spotlight search in the Finder sidebar that aggregate all my OmniOutliner files for one easy view across everything.

OmniOutliner 4 Offers Even More Potential

I really look forward to how OmniOutliner 4 becomes a new part of my world and workflows. The AppleScript looks robust (I didn’t try it in the many months of beta, but look forward to it now). With scripting and the structure there is a whole lot that is possible.

Brett Terpstra Focusses on His Work Full-time

I have been a fan of Brett Terpstra for some time. I found his site through a few buds who focus on productivity and personal workflows (including scripting). I have followed his Systematic podcast since the first episode and have found it is the one podcast I listen to when my weekly listening dwindles to just one podcast. His nvALT became an app that is always running and where a lot of writing snippets get stored on my Mac (that content I also reach on my iOS devices to edit and extend). His Marked2 app not only is my Markedown viewer, but a rather good writing analysis tool.

Not only all of this but Brett is a tagger and not only tags, belives tagging is helpful for personal filtering and workflow, but has built tools to greater extend tagging in and around Mac and iOS. If you talk folksonomy and pull the third leg of it, person / identity back into just your own perspective and keep the tag and objects in place you have the realm that Brett focusses and pushes farther for our own personal benefit.

Brett Steps Out to Focus on His Work

This week I have been incredibly happy to learn that Brett moved out of his daytime job to and his new job is to focus on his products, podcast, writing, and new projects and products. This is great news for all of us. Brett took this step before, but we can help him keep these tools and services flowing by supporting him.

If you have benefitted from Brett’s free products of want to ensure the great services and tools that Bret has created you paid for keep improving go help him out.

January 1, 2014

Tipping into the 14th Year of This Blog

Pwhoooo… Pwhoooo… Just blowing a little dust off this blog.

Here at vanderwal.net this blog started in the final hours of 2000. It started on Blogger and within a year moved off onto my own held build platform, which occasionally still gets tweaks. The volume and frequency have gone through vast swings over the years, lately things are a bit more sparse, as words floated into various services, like Twitter and Facebook, and more professional posts over to my other blog that started in 2004 or 2005 , Personal InfoCloud.

This blog has remind my personal space on the web. It tipped over 2000 posts a year or two back even in the relatively sparse posting of the past few years. But, I still attend to this blog and post things. Between here and my Personal InfoCloud blog there is a lot of content brewing to be finished and posted.

Blogging Still Matters

One of the best summaries about blogging was posted this week by Euan Semple, on why blogging still matters is brilliant and nails it. Blogging when done successfully is not about proclaiming brilliance and answers (as those never meet their goals as the world is too large and there are millions of people with more experience and understanding who can and often will point out the flaws). Blogging is about sharing perspective and experience, often not fully formed, but are a part of a collective of humanity in trying to think through things in out in the open. The best business and professional blogs (best meaning they get passed around like mints at the exit gate of a garlic festival) follow this model. They are open and honest. They are full of observation, good thinking through things, explaining perspective, and honest. Honest as in humanity is embraced, in the we are all in this together honesty.

One of the best quotes I stumbled into in recent years, nails the blogging perspective (from the from The Book of Tea):

Those who cannot feel the littleness of great things in themselves are apt to overlook the greatness of little things in others.

The humility and finding great things in others and the understanding of the little things that make a difference is key.

Your Blog is Your Own Home

Along these lines I really liked Frank Chimero’s Homesteading 2014 post sentiments of bringing things that were placed elsewhere and posting them on one’s own site. Along these lines my KM World articles, which I have rights to do what I want with I will be posting fully into Personal InfoCloud.

As well, there are some vanderwal.net site wide changes to the CSS I have been wanting put in place for quite some time. The code for the whole of the site was last widely tweaked in the 2001 to 2003 timeframe (a mobile version of the blog was made then along with a mobile blogging interface, which logged the impending entrance of the kid to the world from my HipTop / Sidekick as I walked from parking to the hospital maternity ward). It may be time to up date it a bit more. There is a long list of mods to the blog categories (160 or so) and discovery for the blog, including in site search, that I may tackle. Also considering a full rebuild and redesign using Zurb’s Foundation. Who knows what or when it will happen. I think writing may take some precedence, but I am badly missing building things and the long latent hacker mind is itching to build, hack, and test.

Here is to a Great Blogging Year

Here’s to a great 2014 to all. Maybe it is time to think about blowing the dust off your blog and let things fly again.

[Oh, by the way, this is posted on United States Eastern Time and the blog date and time pick up those from the server, which is in Eastern Australia (I fixed this on the development site and then over wrote the adjustment before moving it to the production side of the house - features)]

May 13, 2013

Driving Out of the Valley

The stretches of time after my mom passed and I was back in California’s Northern Central Valley I found a rather deep appreciation for it that I had lacked much of my life (moved there before 8th grade after living in Los Angeles, Portland, Seattle, Spokane, and the Bay Area - by comparison in my limited life span there was nothing there).

What I found was beauty in the breezes, the long sky, the long drives next to farm land, and the fog and clouds coming over the Coastal Range to cover and fill the Valley as they headed to the Sierras. The lighting was bright with tinges of amber, yellow, blue, and green hues bouncing around. There was an old school charm that settled back to past centuries. Lunch in the bar with friends and the bill settled with dice cups, with the bar covered with small dents from many years of the dice’s resolution to bills and a few bills passed between hands under cover of the bottom side of the bar.

The drives to the Bay Area were regular and helped keep sanity and connected to a larger world that spanned beyond the hills and mountains to the East and West and the rich soil running North and South.

The drive down I–5 became wonderful as the turn West beyond though Tracy pointed to the Altamont and the wondrous golden brown hills. That part of the drive was the most boring, but driving and having the right music turned it into a dream space of dancing bathing light with clouds moving in time lapse pacing in real time driving at speed. The Altamont turns into graceful enwrapping curves you drive to and into them. The occasional clouds hanging in the hills give the sun a halo of golden light or the occasional spherical rainbow that only can happen in a land of magic and light.

April 29, 2013

Broken Decade Precedes It Works Decade

I had long forgotten this Carl Steadman response to Michael Sippy’s “Just One Question - What do you want for Christmas”, but the response from 1997 is fantastic and frames the 1990s as the broken decade. (I’ll wait for you to go read it)

I’m not so sure that Carl’s broken decade got better in the first half of the 2000 decade, but it really started to. We are much farther along now. Our consumer world started to improve quite a bit and slowly business systems and services are slowly improving. The initial part of Carl’s rant focusses on the number of steps to get something going. Once it is working the steps are still clunky.

Carl gets in a great rant about time and how broken it was in the 90s within technology (calendaring and syncing is still a beast and likely to for a bit longer - you understand the problem sets and pain points if you have ever tried to build syncing). With calendaring and its related activities we now have Tempo, which is freakishly close to the next step scenario I used in many of the Come to Me Web presentations and Personal InfoCloud presentations from 2003 through 2007 (I’ve been getting requests to represent them as this is what more and more developers and designers are dealing with today and need to have a better foundation to think through them). There was an internal Yahoo presentation (and follow on day of deep discussions and conversations) with a version of the Personal InfoCloud and Come to me Web flow that is nearly identical to the Tempo app video scenario and ones spelled out in Robert Scoble’s interview with Tempo CEO, which is utterly awesome that it is getting built out some 10 years later (we had the technology and tools to do this in 2004 and beyond).

Carl’s rant gets worn away over time though consumer devices, services, and applications. The refocus on ease of use and particularly the use through mobile, which requires a very different way of thinking and considering things. It thinking through design, the dependancies, and real user needs (all while keeping in mind the attention issues, screen size, networking, and device limitations). The past couple years mobile finally caught on with mainstream users and people doing real work on the mobile and tablets - Box 40% mobile access of files stored there over the last couple years. Many other business vendors have had mobile use rates of their services from mobile over the past two years. When talking to users they opt for mobile solutions over their full enterprise tools as they are much easier to use, which quickly translates into getting more work done. As Bernd Christiansen of Citrix stated in an onstage interview the employee’s most productive part of the day is often the walk from their car to the front door of the office working on their mobile devices.

This world is not fully better and fully easy to use from the days of Carl’s rant, but it is getting better. We still have quite a ways to go.

January 16, 2013

A World without Aaron Swartz

It was a weekend not focussing on technology or internet much, but I saw some usual patterns that are the usual signs that something is not well, but I had other things that took priority of focus. Sunday I gave a quick look and let out that human deep gasp and my kid looked up and asked what was wrong as my head slumped. I was late to the news that Aaron Swartz had taken his life. I don’t know what site I read it but all my screens of services and people I follow closely where sharing the news and their remembrances.

Aaron’s passing was beyond the “he is too young” and “what a shame”, he was not only someone special he was changing the world and had been for quite a while with his sorting through the battles of standardizing RSS, working with the Creative Commons to create a modern equivalent that could be relatively easily used attach to published and shared content allowing much better and more open access to it, he helped contribute to Reddit as it was hatching and stayed with it through to it being purchased. He has done so much more since. But, he died at 26 years old. At 14 when he was partaking in the RSS discussions on listserves nobody knew his age. Nobody had a clue and it wasn’t known until they asked him to come to a gathering to discuss things face to face. This story of his age and the wonderful story about how people found out was spread on listserves I participated in and at around post conference drinks in the early 00s. At the age of 14 Aaron had lore. He had earned the respect and right to be a peer with the early grey beards of the Web that were battling to understand it all, help it work better, and make the world better because of it. Aarron fit right in.

I was at a few events and gatherings that Aaron was at in the early 00s but I never had the chance to work or interact with him. But, I have worked and interacted with many who did have that fortune and even with the lore Aaron had they were impressed by his approach, capabilities, and what he could accomplish. In the tech community it is a meritocracy and you earn credibility by doing. The world has always been changed by those who do, but also by those who are curious. but are guided by an understanding of an optimal right (correct way).

What we lost as a society was not only a young man who earned his place and credibility, and earned it early, but he gave to others openly. All the efforts he put his heart and mind to had a greater benefit to all. The credo in the tech community is to give back more than you take. Aaron did that in spades, which made thinking of a future with whatever he was working on a bit more bright and promising. Aaron’s blog was at the top of my feed reader and was more than worth the time to read. He was a blogger, an open sharer of thoughts and insights, questions as well as the pursuit of the answer to the questions. This is not the human norm, he was a broken one in all the understandings that brings as being outside the mainstream norms, but much like all those in the Apple “Think Different” ad campaign he made a difference by thinking different and being different from those norms.

I love David Weinberger’s Aaron Swartz was not a hacker. He was a builder. as well as his Why we mourn. From the rough edges of hearing friends talk about their working with Aaron and following along with what he shared, we as a society were in for a special future. Doc Searls’ Aaron Swartz and Freecom lays out wonderfully the core of Aaron’s soul as a native to the Net in the virtues of NEA (Nobody owns it; Everybody can use it; Anybody con improve it). Doc also has a great collection of links on his memorial posting, many from those who worked with Aaron or knew him well on his Losing Aaron Swartz page.

Dang it, we are one down. We are down a great one. But, this net, this future, and this society that fills this little planet needs the future we could have had, but now it is ours to work together to build and make great.

How to we get there? Aaron’s first piece of advice from Aaron Swarts: How to get my job is, “Be curious. Read widely. Try new things. I think a lot of what people call intelligence just boils down to curiosity.”

Peace!

Previous Month

This work is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike License.