Some doubts about dual 3090 and cooling

[jmr]

Hi,

building a workstation for machine learning. Initially buying a 3090 but thinking on buying one or two more in the future and placing them in the same cage.

Once said that I would like to see if anybody have an answer for some questions

1) the configurator does not seem to allow to put a second 3090 (it is ok to put some other GPUs though, up to 4). Is there a reason for that?

2) How many 3090 could I fit if I watercool them?

3) Do you see in the configuration something to improve/balance?

Thanks!





Case
FRACTAL DEFINE 7 XL BLACK QUIET TOWER CASE
Processor (CPU)
AMD Ryzen Threadripper PRO 3975WX 32 Core CPU (3.5GHz - 4.2GHz, 144MB CACHE)
Motherboard
ASUS® Pro WS WRX80E-SAGE SE WIFI (Intel dual 10 G LAN, USB 3.2 Gen 2x2 Type-C port, SLI)
Memory (RAM)
64GB Kingston DDR4 3200MHz ECC Registered (2 x 32GB)
Graphics Card
24GB NVIDIA GEFORCE RTX 3090 - HDMI, DP
1st M.2 SSD Drive
1TB SEAGATE FIRECUDA 530 GEN 4 PCIe NVMe (up to 7300MB/R, 6000MB/W)
1st M.2 SSD Drive
4TB CORSAIR MP400 NVMe PCIe M.2 SSD (up to 3480 MB/R, 3000 MB/W)
Power Supply
CORSAIR 1600W PRO SERIES™ TITANIUM AX1600i DIGITAL MODULAR PSU
Power Cable
1 x 1 Metre IEC C19 UK Power Cable
Processor Cooling
CoolerMaster MasterLiquid ML360 RGB TR4 High Performance Liquid Cooler
Thermal Paste
STANDARD THERMAL PASTE FOR SUFFICIENT COOLING
Sound Card
ONBOARD 6 CHANNEL (5.1) HIGH DEF AUDIO (AS STANDARD)
Network Card
2 x 10 GIGABIT ETHERNET PORTS
Wireless Network Card
WIRELESS INTEL® Wi-Fi 6 AX200 (2.4 Gbps) + BT 5.0
USB/Thunderbolt Options
MIN. 2 x USB 3.0 & 2 x USB 2.0 PORTS @ BACK PANEL + MIN. 2 FRONT PORTS
Operating System
Windows 10 Professional 64 Bit - inc. Single Licence [MUP-00003]

 

[barlew]

Hi,

building a workstation for machine learning. Initially buying a 3090 but thinking on buying one or two more in the future and placing them in the same cage.

Once said that I would like to see if anybody have an answer for some questions

1) the configurator does not seem to allow to put a second 3090 (it is ok to put some other GPUs though, up to 4). Is there a reason for that?

2) How many 3090 could I fit if I watercool them?

3) Do you see in the configuration something to improve/balance?

Thanks!





Case
FRACTAL DEFINE 7 XL BLACK QUIET TOWER CASE
Processor (CPU)
AMD Ryzen Threadripper PRO 3975WX 32 Core CPU (3.5GHz - 4.2GHz, 144MB CACHE)
Motherboard
ASUS® Pro WS WRX80E-SAGE SE WIFI (Intel dual 10 G LAN, USB 3.2 Gen 2x2 Type-C port, SLI)
Memory (RAM)
64GB Kingston DDR4 3200MHz ECC Registered (2 x 32GB)
Graphics Card
24GB NVIDIA GEFORCE RTX 3090 - HDMI, DP
1st M.2 SSD Drive
1TB SEAGATE FIRECUDA 530 GEN 4 PCIe NVMe (up to 7300MB/R, 6000MB/W)
1st M.2 SSD Drive
4TB CORSAIR MP400 NVMe PCIe M.2 SSD (up to 3480 MB/R, 3000 MB/W)
Power Supply
CORSAIR 1600W PRO SERIES™ TITANIUM AX1600i DIGITAL MODULAR PSU
Power Cable
1 x 1 Metre IEC C19 UK Power Cable
Processor Cooling
CoolerMaster MasterLiquid ML360 RGB TR4 High Performance Liquid Cooler
Thermal Paste
STANDARD THERMAL PASTE FOR SUFFICIENT COOLING
Sound Card
ONBOARD 6 CHANNEL (5.1) HIGH DEF AUDIO (AS STANDARD)
Network Card
2 x 10 GIGABIT ETHERNET PORTS
Wireless Network Card
WIRELESS INTEL® Wi-Fi 6 AX200 (2.4 Gbps) + BT 5.0
USB/Thunderbolt Options
MIN. 2 x USB 3.0 & 2 x USB 2.0 PORTS @ BACK PANEL + MIN. 2 FRONT PORTS
Operating System
Windows 10 Professional 64 Bit - inc. Single Licence [MUP-00003]

I would imagine the configurator wont allow it because the card are absolutely huge. I doubt you would fit two in that case let alone 3!

Just to caveat this, its my opinion not a definitive.

 

[DarTon]

First, I'd have to say I have no personal experience running 2 or more 3090s in a single machine. I have only a A6000 and a 3090 in my current work machine which is somewhat less onerous.

I have no idea why PCS doesn't allow selecting more one 3090. It may well be that because PCS don't guarantee a specific 3090 card, that getting the Nvlink bridge to work across different types of 3090 is non-trivial. It's perfectly doable but I know at work we've had far more issues getting 3090s to cooperate than the A series cards. I'd ring up PCS and ask them.

In terms of the number of 3090s, the Fractal Define XL case has 9 standard and 3 vertical expansion slots. Given most 3090s are 3 slot cards, then 3 might be doable but getting 4 in seems rather doubtful.

Personally, while the FD XL case is a very good one, I'd be cautious given that it's not the best case from an airflow perspective. Plus with 3x 3090s all drawing possibly up to 450W, then the 1600W PSU is going to be taking a lot of strain unless you power limit them.