The attention mechanism has dominated deep learning in recent years, but its quadratic complexity makes it resource-hungry on long and multidimensional data. Previously proposed subquadratic alternatives, such as standard convolutions or recurrent models, forced compromises: the former lose the global receptive field and input dependency, while the latter require flattening images, volumes, or physical simulations into a one-dimensional scan order that violates their spatial structure.

The new HyenaND operator, presented by the researchers, flips the perspective: instead of adapting the data to the algorithm, it designs long multidimensional, input-dependent convolutions that respect the native geometry of the problem. The mathematical core is an implicit parametrization of global kernels, which captures long-range relationships without the quadratic complexity of attention. The computational cost scales as O(L log L), where L is the data size, making it ideal for very long sequences and high-dimensional data.

But the real novelty isn't just the algorithm. The team developed a CUDA implementation, called nSubQ, that fuses the FFT-convolution path to translate that O(L log L) into real speedups. In practice, it's not just a neat theorem: there's an optimized kernel that allows HyenaND to be used in inference and training pipelines on GPUs with gains in wall-clock time.

The tests cover areas where attention is most burdensome: long-context genomics, computer vision, medical imaging, and partial differential equation (PDE) modeling. In these domains, pure HyenaND stacks match the accuracy of top attention-based models. Even more interesting is the hybrid configuration: layers alternating HyenaND and attention outperform both pure transformers and hybrids with recurrent models.

For those thinking in terms of on-premise deployment or controlled enterprise hardware, the message is clear. The subquadratic efficiency reduces the computational cost per unit of data, which translates into lower hardware requirements for workloads that today force the use of high-end GPUs or the cloud. It's not just about energy savings but about making the processing of 3D volumes, genomic sequences, or physical simulations feasible on local infrastructure, where data sovereignty and TCO control are paramount. If HyenaND were integrated into mainstream deep learning frameworks, companies could rethink their compute architecture, shifting loads from the cloud to on-premise hardware without sacrificing performance.

Of course, the work is still at the research level and the maturity of the CUDA implementation will need to be verified at industrial scale. But the signal is structural: the race is not only toward larger models but toward more efficient operators that can process complex data directly in their native form. HyenaND shows it can be done, and it does so with an implementation that points straight to practical use.