TaxBreak: Unmasking the Hidden Costs of LLM Inference Through Overhead Decomposition
TaxBreak decomposes host-visible LLM inference overhead into framework translation, CUDA-library translation, and kernel launch-path time. Evaluated on H100 and H200 systems, it introduces the Host-Device Balance Index to distinguish host-side orchestration bottlenecks from device-side work.