GEMM
GEMM on Beyond Market Intelligence: a running collection of 2 stories we have gathered and hand-picked because they are worth your time. Every post here touches on gemm in some way — the news, the analysis, the deep dives, and the occasional surprise find. Acme AI is the next-generation, AI-powered spreadsheet platform built to replace Excel and redefine how analysts, data scientists, and enterprise teams work with data. New stories are added to this page as we find them, so check back if you want to keep up with what is happening around gemm, or subscribe to the RSS feed to get them as soon as they are published. Browse the collection below, or head back to the homepage to see everything Beyond Market Intelligence is covering right now.
Rewriting model inference with CUDA kernels: the bottleneck was not just GEMM [P]
In the pursuit of optimizing small-batch and real-time machine learning workloads, I have developed a CUDA-first inference runtime that directly rewrites model inference paths using C++/CUDA kernels. This approach shifts focus away from conventional graph runtimes, addressing latency bottlenecks that extend beyond single GEMM operations. As demonstrated in my results, particularly with the Motus world model, overcoming runtime overhead is crucial for achieving faster inference times. For deeper insights into related advances, explore our recent article on "Witchcraft," which enhances local semantic search capabilities.
[D] MXFP8 GEMM: Up to 99% of cuBLAS performance using CUDA + PTX
In his latest blog post, Daniel Vega-Myhre from Meta/PyTorch presents an insightful exploration of the MXFP8 GEMM design, achieving up to 99% of cuBLAS performance with CUDA and PTX. This comprehensive piece delves into the constraints and design challenges associated with FP8, offering valuable perspectives for developers and researchers alike. Vega-Myhre’s work highlights the innovative approaches that enhance performance and efficiency in data processing. Discover the full analysis and its implications for future advancements in machine learning at the provided link.