asmi
One binary. Every node. Full remote control. The control plane for Apple Silicon clusters.
Architecture
What asmi monitors
CPU
P/E-core cluster residency, per-core frequency, thermal pressure readings.
GPU
Metal utilization percentage, active command queues, shader core occupancy.
RAM
App/Cached/Available breakdown with per-model memory segments for running servers.
Power
CPU and GPU wattage in milliwatt precision via IOReport sampling.
Processes
Running model servers, PIDs, RSS/virtual memory, uptime for each process.
RDMA
InfiniBand device health, ARP peer discovery, link-local state per TB5 cable.
What asmi controls
Model Serving
Load, stop, reload, and hot-swap models via HTTP. No SSH, no scripts.
Distributed Inference
JACCL tensor parallelism across nodes. Hostfile generation, rank coordination.
Model Sharing
Transfer models between nodes at 5+ GB/s over Thunderbolt 5 RDMA fabric.
Topology Discovery
Auto-detect TB5 cables, enumerate Tailscale peers, build cluster graph.
Watchdog
Self-healing daemon with GPU lock detection, peer heartbeats, auto-restart.
Process Management
Kill hung processes, view process trees, clean up orphaned model servers.
Experimental
--experimental-aneNeural Engine Compute
Experimental Neural Engine compute endpoints, available only in asmi builds compiled with ANE support.
--experimental-egpueGPU / TinyGPU Detection
Detects external GPUs and TinyGPU DriverKit extensions attached over Thunderbolt. Detection only.