asmi
One binary. Every node. Full remote control. The control plane for Apple Silicon clusters.
Architecture
What asmi monitors
CPU
P/E-core cluster residency, per-core frequency, thermal pressure readings.
GPU
Metal utilization percentage, active command queues, shader core occupancy.
RAM
App/Cached/Available breakdown with per-model memory segments for running servers.
Power
CPU and GPU wattage in milliwatt precision via IOReport sampling.
Processes
Running model servers, PIDs, RSS/virtual memory, uptime for each process.
RDMA
InfiniBand device health, ARP peer discovery, link-local state per TB5 cable.
What asmi controls
Model Serving
Load, stop, reload, and hot-swap models via HTTP. No SSH, no scripts.
Distributed Inference
JACCL tensor parallelism across nodes. Hostfile generation, rank coordination.
Model Sharing
Transfer models between nodes at 5+ GB/s over Thunderbolt 5 RDMA fabric.
Topology Discovery
Auto-detect TB5 cables, enumerate Tailscale peers, build cluster graph.
Watchdog
Self-healing daemon with GPU lock detection, peer heartbeats, auto-restart.
Process Management
Kill hung processes, view process trees, clean up orphaned model servers.
Experimental
--experimental-aneNeural Engine Compute
Direct Apple Neural Engine inference via private IOKit APIs. Offload quantized layers to ANE while GPU handles attention — hybrid compute across both accelerators.
--experimental-egpueGPU / TinyGPU Detection
Enumerate external GPUs connected via Thunderbolt. TinyGPU DriverKit support for NVIDIA/AMD eGPUs on macOS without SIP bypass.
37
HTTP Endpoints
9090
Default Port
Rust
Written in
~4 MB
Binary Size
SSE
Streaming