Repo Roundup #2

This week’s collection of repositories I dug into are all about artificial intelligence, LLMs, and using them as part of a software engineering process.

Repositories

Claude-Code-Usage-Monitor

I’ve been trying out Claude Code lately using the Claude Pro plan. One thing I’m curious about is the actual value of using the Claude Pro plan versus using the Claude Code CLI with a normal Claude API token: you can use either method. Claude Code itself has a /cost command which will print out token cost information in-app if you are using a Claude API key but it doesn’t tell you any cost data if you’re using the app with a Claude Pro subscription.

It turns out that in either case Claude Code logs your whole session’s messages to disk in jsonl files that this CLI app can read and give you token usage/cost information anyway. I’m curious if my real usage will be more than $20 worth of token usage in a month thus making the Claude Pro subscription a better value than just using a Claude API key.

exo

This project lets you combine multiple random computers varying CPU/GPU and RAM/VRAM configurations into a cluster that can run larger AI models than any individual device could on its own.

I have a pretty capable hardware setup at home for running local models but this project has made me look at some unused older Apple devices I have laying around my house a little differently.

kokoro-web

I was excited to see this repository because I was trying out Kokoro in my homelab a few weeks ago as an API service with FastAPI. I really liked the experience and was able to get it working as the TTS voice of the OpenWebUI instance I also run locally.

I especially liked that I was able to generate arbitrary strings of text at speeds consistently far below realtime without using third-party compute: no Internet necessary and importantly no third-parties logging everything I use it for (which as an Autistic person with ADHD is a lot of things).

This repository takes that even further: this can run Kokoro within a browser and using GPU acceleration if available. Although most of my ideas for speech synthesis make best sense outside of a browser environment, it’s definitely exciting to see this kind of thing is possible.

whisper-web

This repository is from the same author as kokoro-web above about but instead it is for transcription using OpenAI’s open-sourced Whisper model in a browser with GPU acceleration. As usual, impressive work.

I have been using Whisper extensively in my workflow through MacWhisper.