dmesg --follow
[ 68698440.000 ] posts.x: Full talk on building and deploying MCP servers and agents with Spin and Akamai Functions, including the live cold start numbers:  |   [ 68698440.000 ] posts.x: Cold starts are the tax every serverless AI agent pays, and @ThorstenHans spends "Zero Cold Starts: Serverless AI Agents on Akamai Functions" trying…  |   [ 68692020.000 ] posts.x: Full talk on building a structured query engine for agent context, from schema discovery to self-correcting queries:  |   [ 68692020.000 ] posts.x: When Claude starts a task with no context attached, it behaves like a brand new employee: grepping through the codebase, guessing at what matters…  |   [ 68673720.000 ] posts.x: @davidsenra you will love this!  |   [ 68673600.000 ] posts.x: https://t.co/vhJifvV8y9  |   [ 68673600.000 ] posts.x: This Jay-Z documentary, In 8, is exceptional! Nearly 6 hours of intimate conversation between @sc and the brilliant @RickRubin. It’s rare to get such…  |   [ 68662260.000 ] posts.x: Full talk on turning re-prompted fixes into linters, including the OpenAPI drift fix and the skill-writing whack-a-mole:  |   [ 68662260.000 ] posts.x: Re-prompting an agent to fix the same sloppy pattern over and over isn't a workflow, it's a sign the rule should live in code, not in your head…  |   [ 68650020.000 ] posts.x: ↻ @willcb: me: kinda fucked up how many people are offloading their whole lives to muse or instinct or grokbot or whatever claude: yeah  |   [ 68650020.000 ] posts.x: ↻ @iamdevloper: signs you've become the senior dev: you're the only one who understands the deploy script you're the only one who remembe…  |   [ 68649720.000 ] posts.x: Gooooooooood morning! Brazil Oberon Cerrado. Roasted 110 seconds past first crack. Milk chocolate and almond.  |   [ 68647800.000 ] posts.x: @Saanvi_dhillon “Git expert” is a weird flex, though, isn’t it?  |   [ 68610720.000 ] posts.x: @sujee_dev @dkundel @aiDotEngineer Thanks mate!  |  
corey@gallon.me:~/til$

Copying Files in Linux With a Progress Bar

FIGURE 1 ⋅ Copying Files in Linux With a Progress Bar

I’ve long used rsync for high fidelity, detailed copies of large swathes of files in Linux. For example:

rsync -av --progress /source/directory /destination/directory

This gives you really nice, file-by-file progress indications — which is especially useful when moving large files across the network between machines or even locally across physical devices.

Today I was moving data from old hard drives and just wanted overall progress on the total job to be done. rsync still FTW!

rsync -a --info=progress2 /source/directory /destination/directory/

NB — do not use a trailling / in the source directory if you want the copied files to be retained inside of that directory name in the destination!

This provides lovely summary output like this:

ubuntu@ubuntu:/media/ubuntu$ rsync -a --info=progress2 /source/directory /destination/directory
  2,675,525,167  71%   13.49MB/s    0:03:09 (xfr#1281, to-chk=0/1712)  

A Better, More Robust Way to Do This

Running this command gets some extra good stuffs.

rsync -a --info=progress2 --checksum --partial /source/directory /destination/directory/

Explanation of the Additions:

  1. -checksum:
    • Ensures that files are compared using checksums rather than just timestamps and file sizes.
    • Guarantees that files are transferred only if their content differs, even if metadata like modification times are identical.
  2. -partial:
    • Retains partially transferred files if the transfer is interrupted, allowing resumption without starting over.
corey@gallon.me:~$ tail -f /writing Attach to the stream. An email when I have something worth sending. Replies encouraged!
corey@gallon.me:~$ ls -lt /til ↑2024-12-16 Linking to Posts in Python Pelican
▸2024-11-28 Copying Files in Linux With a Progress Bar ⋅ you are here
↓2024-11-28 Why An 8TB Drive Isn’t 8 Usable TB