vanderwal.net

Rearranging Automations and Aggregations

This week and the prior had a spattering of updates and reworking of some of my regular automations and aggregations.

I’ve posted about my automation patterns with scripts (mostly python), some AI (sometimes Claude), many web scripts and web services (IFTTT and a few others), many local scripts and triggers that one action for an intended purpose also pushes things to additional places (for aggregation, organization, and later retrieval), other patterns. I’ve written about some of this recently: My Automations Over the Ages and Today’s View and Adjusting Daily Aggregation for Less Friction.

I’ve long played with data to sort out what is in it, the shape of it, and possibilities in it (going back to before grad school – it was a large part of my second job out of undergrad and grad school expanded and honed it). This continues today with different tools and services. The ability to quickly build a proof of concept for data analytics and munge the data to get a helpful data visualization is something I’ve liked from AI services like Claude. I can test a workflow, set up a quick process for making use of the data and information I normally tuck away. Most often I’ll try something and if it works well I will modify it and move it out of Claude and into a python script (or set of python-based processes) to get a usable output I can gain insight from and/or work with. On occasion the data is not steady enough, so I will keep things in Claude and run scheduled automation to do the grunt work so that I have my aggregations in my Daily Dump markdown note I keep in my notes folder Obsidian looks over, and it is ready early in the morning for me to sort out what I continue with, what is fine, and what I need to process in some way.

Nearly all of my morning aggregations and automations are dealing with local (on my device(s)) data, information, and objects. I don’t want this going into anybody else’s service or cloud as it is foolish and pointless to do this. This is my Personal InfoCloud (see Overview of the InfoClouds – Personal InfoCloud) and it remains there not drifts out.

Claude Cowork Shift Local Scheduled Automations

A little less than a third of my morning automations and aggregations have run through a local Claude Cowork Scheduler. Anthropic announced it was not going to allow the creation of new Cowork Scheduled tasks and new ones (possibly modifications too) would need to move to the cloud. Uh… no! The options were to move fully to Python and/or other scripted workflows, move off of Claude, or move the scheduled tasks out of Cowork to Claude Code. I also had a few morning automation and aggregation tasks I kick off manually as they are based on me finishing my personal tasks.

Manually Mapping Out Flows in Obsidian Canvas Helps

A little more than a year ago I mapped out my morning and weekly workflow automations and aggregations with the services, scripts, tasks, time triggers, schedulers, and regular manually triggered tasks, automations, and aggregations. I mapped this rather quickly in Obsidian’s Canvas Obsidian Canvas that is helpful for seeing groupings, related processes through links, and flows. I went through my grouped tasks that I automated and marked the ones I needed to deal with using a yellow dot emoji. There were a few nested ones I hadn’t considered. (A great thing with Canvas is having a link to my notes in the automation node so I can quickly jump to see details.)

I have long used Claude Code for debugging code and suggesting optimizations to code (sometimes the suggestions are good and quite helpful, I moved to it after deep frustration with ChatGPT and GitHub Copilot poor output). Moving things to Code was relatively straight forward (it didn’t have the battles with time that were prevalent in Cowork, which was really frustrating until I sorted out a workaround so Cowork could understand “today”, “yesterday”, “past 24 hours”, etc – working with time can be problematic, but it mangled things that are relatively straight forward).

Move from Claude to Python Driven App, Triggered by Apple Shortcuts

I went to convert one of my manually triggered automations and realized it didn’t need to be in Claude at all, and was a good candidate to move python, but I would need to build a Slack app / bot to pull from my personal Slack instance. About 5 to 10 years ago building a Slack app was rather a common and straightforward endeavour. That changed a bit and the setup screens did too. Slack is now AI bot-friendly first and I’m not building an AI bot, but going in the opposite direction (I had used the MCP from Slack to run the aggregation prior). Rather than 15 to 30 minutes creating the Slack app it took around 90 minutes to deal with the changes in Slack. But, I also wanted an easier way to trigger the Slack app to run and capture the links to videos I had found interesting, and put them in a list in a markdown file, so I could take the next steps. I ended up using Apple Shortcut to trigger the Python script. The automation now is done in under a second rather than just under two minutes and no files uploaded anywhere.

I have two other automations I manually trigger that I can likely move to Python in the same manner using secure APIs and triggering them from Apple Shortcuts. I haven’t converted them yet.

The Shift to Code Started a Few Months Ago

In the past few months I moved a few of my Claude tasks I run for clean up of transcripts from Cowork to Code and found a large reduction in time, tokens, and Claude usage in that move. The quality of the shift has been better quality, output is more consistent, and also calls out potential issues on transcription clean-up (I pull podcast and YouTube transcripts which are riddled with errors and poorly structured, I don’t change the content but have proper sentence and paragraph structures, but also included some sub-headers). Some of the data conversion tasks and modifying script tasks that I would run or plan out in Chat or Cowork all moved to Code and the quality improved and token / usage all noticeably improved. The usage and quality improved when I started using Opus 5.5, with Sonar I had a lot of rework and Claude off and running changing things that were reviews not to change things. My token and usage decrementing is about a third less or better with Code and Opus 5.5, which doesn’t fully make sense.

The last push was moving my Daily Dump related aggregations and automations and the Weekly round-up out of Cowork as everything done is local with those tasks. The move from Cowork to Code was rather straightforward. I asked Claude Code to guide setting up the needed resource access it would need for its own processing and assistance setting up the Routines and its timing (Code’s routine timers aren’t precise and I wanted them running at the exact open times with locked windows I regularly run things within). That timing took had pushback from Claude but got it close to the window and then it took me understanding Code Routines have some odd buffers of a few minutes.

How it is Going

This took a chunk of time early in the week, but things are running fine. All but a couple of manually triggered tasks are now green check emoji and not yellow dots. Tomorrow morning when my Weekly roundup runs will be a big question and test.

The biggest friction points are the pieces I’ve moved to new homes and remembering where they get triggered. The upside with using Apple Shortcuts is that it can be organized rather well to keep things related together. I moved a lot of my local web servers that enable data visualization web apps to Apple Shortcut triggers while setting up other Shortcuts and that has helped the efficiency of those processes.