7,612
edits
Changes
Jump to navigation
Jump to search
*I picked a rating such that another GPU or other extension could be added without a new PSU. If no other components will be added, 1000W will be more efficient (PSUs are most efficient at 50-75% of peak draw)
*PSU: Rosewill - 1200W 80+ Platinum Certified Fully-Modular ATX Power Supply [https://www.newegg.com/Product/Product.aspx?Item=9SIA0723W13835&nm_mc=AFC-C8Junction&cm_mmc=AFC-C8Junction-PCPartPicker,%20LLC-_-na-_-na-_-na&cm_sp=&AID=10446076&PID=3938566&SID=]
* Network card?
no edit summary
{{Project|Has project output=Content|Has sponsor=McNair ProjectsCenter
|Has title=GPU Build
|Has owner=Oliver Chang,Kyran Adams
}}
==Final Decision==
We decided to clone the NVIDIA [[DIGITS DevBox]]: https://developer.nvidia.com/devbox
To start with we are trying to use our existing ASUS Z10 server board, rather than switching to the Asus X99-E WS workstation class motherboard, and rather than Four TITAN X GPUs, we've got a TITAN XP and a TITAN RTX.
Note that the Asus X99-E WS is available from NewEgg for $500 now.
==Single vs. Multi GPU==
*[https://www.nvidia.com/en-us/geforce/products/10series/geforce-gtx-1080-ti/ GTX 1080 Ti Specs]
* Since we are using Tensorflow, it doesn't scale well to multiple GPUs for a single model
Cons of multiple GPUs:
*Adds a lot of complexity.
=== K80, NVLink ===
*NVLink can link between CPU and GPU for increase in speed, but only with the CPU IBM POWER8+.
*NVLink can link between GPU and GPU as a replacement for SLI with other CPUs, but this is not super relevant to tensorflow, even if trying to parallelize across one model.
*[https://www.quora.com/Which-GPU-is-better-for-Deep-Learning-GTX-1080-or-Tesla-K80 This source] says to get the 1080 because the K80 is basically two K40s, which have less memory bandwidth than the 1080. [https://www.reddit.com/r/deeplearning/comments/5mc7s6/performance_difference_between_nvidia_k80_and_gtx/ This source] agrees.
==Misc. Parts==
*Cases: Rosewill 1.0 mm Thickness 4U Rackmount Server Chassis, Black Metal/Steel RSV-L4000[https://www.amazon.com/gp/product/B0056OUTBK/ref=oh_aui_detailpage_o04_s00?ie=UTF8&psc=1]
*Consider this case: Corsair Carbide Series Air 540 High Airflow ATX Cube Case [https://www.amazon.com/dp/B00D6GINF4/ref=twister_B00JRYFVAO?_encoding=UTF8&psc=1]
*DVDRW (Needed?): Asus 24x DVD-RW Serial-ATA Internal OEM Optical Drive DRW-24B1ST [http://www.amazon.com/Asus-Serial-ATA-Internal-Optical-DRW-24B1ST/dp/B0033Z2BAQ/ref=sr_1_2?s=pc&ie=UTF8&qid=1452399113&sr=1-2&keywords=dvdrw]
*Keyboard and Mouse: AmazonBasics Wired Keyboard and Wired Mouse Bundle Pack [http://www.amazon.com/AmazonBasics-Wired-Keyboard-Mouse-Bundle/dp/B00B7GV802/ref=sr_1_2?s=pc&rps=1&ie=UTF8&qid=1452402108&sr=1-2&keywords=keyboard+and+mouse&refinements=p_72%3A1248879011%2Cp_85%3A2470955011]
* [https://www.slideshare.net/PetteriTeikariPhD/deep-learning-workstation Deep Learning Computer Build] useful tips, long
* [https://www.tooploox.com/blog/deep-learning-with-gpu Another box]
* [http://graphific.github.io/posts/building-a-deep-learning-dream-machine/ Expensive deep learning box]
==Double GPU Server Build==
[https://pcpartpicker.com/user/kyranadams/saved/gDzFdC PC Partpicker build]
*[https://www.quora.com/Can-I-double-the-PCIe-lanes-in-a-dual-CPU-motherboard This article] says that it may be necessary to get both CPUs to get all of the PCI lanes
==Double GPU Build==
===Motherboard===
*Should have Needs enough PCIe slotsto support both GPUs and other units
*Motherboards: MSI - Z170A GAMING M7 ATX LGA1151 Motherboard [https://www.newegg.com/Product/Product.aspx?Item=9SIA85V4SC7911&nm_mc=AFC-C8Junction&cm_mmc=AFC-C8Junction-PCPartPicker,%20LLC-_-na-_-na-_-na&cm_sp=&AID=10446076&PID=3938566&SID=], LGA 1151, 3x PCIe 3.0 x 16, 4 x PCIe 3.0 x 1, 6 x SATA 6GB/s, also used in [https://medium.com/@SocraticDatum/getting-started-with-gpu-driven-deep-learning-part-1-building-a-machine-d24a3ed1ab1e this build]
*Does not have to be fast for deep learning: "CPU-RAM-to-GPU-RAM is the true bottleneck – this step makes use of direct memory access (DMA). As quoted above, the memory bandwidth for my RAM modules are 51.2GB/s, but the DMA bandwidth is only 12GB/s!"[http://timdettmers.com/2015/03/09/deep-learning-hardware-guide/]
* Crucial - 32GB (2 x 16GB) DDR4-2133 Memory [https://www.newegg.com/Product/Product.aspx?Item=9SIA8PV5HF1514&nm_mc=AFC-C8Junction&cm_mmc=AFC-C8Junction-PCPartPicker,%20LLC-_-na-_-na-_-na&cm_sp=&AID=10446076&PID=3938566&SID=], SATA 6 GB/s interface
* If not enough, should be able to extend this by buying two more cards
===PSU===
*Some say PSU should be 1.5x-2x wattage of system, some say wattage+100W
*PSU: EVGA - SuperNOVA G2 1000W 80+ Gold Certified Fully-Modular ATX Power Supply [https://www.newegg.com/Product/Product.aspx?Item=N82E16817438010&ignorebbr=1&nm_mc=AFC-C8Junction&cm_mmc=AFC-C8Junction-PCPartPicker,%20LLC-_-na-_-na-_-na&cm_sp=&AID=10446076&PID=3938566&SID=]
===Storage===
*SSD should be fast enough, no need for M.2 [http://timdettmers.com/2015/03/09/deep-learning-hardware-guide]
*SSD: Samsung - 850 EVO-Series 500GB 2.5" Solid State Drive [https://www.newegg.com/Product/Product.aspx?Item=N82E16820147373&ignorebbr=1&nm_mc=AFC-C8Junction&cm_mmc=AFC-C8Junction-PCPartPicker,%20LLC-_-na-_-na-_-na&cm_sp=&AID=10446076&PID=3938566&SID=]
*HDD: Seagate - Barracuda 3TB 3.5" 7200RPM Internal Hard Drive [https://www.newegg.com/Product/Product.aspx?Item=9SIADG25GT7889&nm_mc=AFC-C8Junction&cm_mmc=AFC-C8Junction-PCPartPicker,%20LLC-_-na-_-na-_-na&cm_sp=&AID=10446076&PID=3938566&SID=]
===Other things to consider===
* Water cooling? [http://timdettmers.com/2015/03/09/deep-learning-hardware-guide/ this] has a good section on cooling
* Is case actually big enough?Case is not rack mounted ==Software tips==* PCIe slots work Setting up Ubuntu and Docker [https://medium.com/@SocraticDatum/getting-started-with GPUs?-gpu-driven-deep-learning-part-2-environment-setup-fd1947aab29]
