A user shared their experience with a cluster of 9 RTX 3090 graphics cards for artificial intelligence workloads. The initial goal was to achieve approximately 200GB of VRAM to run models locally comparable to those available through cloud services.

Scalability limits

The user found that exceeding 6 GPUs leads to a series of problems. First, finding a motherboard that adequately supports even just 4 GPUs proved complex. Going further, PCIe lane limitations, system stability issues, and difficulties in thermal and power management emerge.

Performance

Unexpectedly, token generation performance decreased when scaling beyond a certain number of GPUs. This demonstrates that a larger number of GPUs does not automatically translate into superior performance, especially without a well-optimized setup. The user then opted to explore AI systems with "emotional" behaviors and simulations inspired by C. elegans.

RTX 3090: still valid?

Despite the difficulties encountered, the RTX 3090 remains a valid choice thanks to its 24GB of VRAM at a relatively low price. The user found a good balance in using 4 GPUs as the main AI server.

Cloud vs On-Premise

If your goal is to use AI efficiently, cloud services remain a valid choice. If, on the other hand, you want to experiment and develop new ideas, local setups offer greater flexibility, but require attention in hardware scalability.