Difference between revisions of "GPU Build"

From edegan.com
Jump to navigation Jump to search
Line 8: Line 8:
  
 
==Motherboard/CPU/Fan==
 
==Motherboard/CPU/Fan==
*Not a huge deal, but used for data prep
+
*Not a huge deal, but used for data preparation
 +
*If using multiple GPUs, at least one core (two threads) per GPU
 +
*Should have enough PCIe slots
 
*Motherboards: ASUS Z10PE-D16 [http://www.newegg.com/Product/Product.aspx?Item=N82E16813132257&Tpk=N82E16813132257], Dual LGA 2011 R3, DDR4 - Up to 32GB RDIMM, 16 slots
 
*Motherboards: ASUS Z10PE-D16 [http://www.newegg.com/Product/Product.aspx?Item=N82E16813132257&Tpk=N82E16813132257], Dual LGA 2011 R3, DDR4 - Up to 32GB RDIMM, 16 slots
 
*Chips: Intel Haswell Xeon e5-2620v3, 6 core @ 2.4ghz, 6x256k level 1 cache, 15mb level 2 cache, socket LGA 2011-v3 [https://www.amazon.com/gp/product/B00M1BUUMO/ref=oh_aui_detailpage_o04_s00?ie=UTF8&psc=1]
 
*Chips: Intel Haswell Xeon e5-2620v3, 6 core @ 2.4ghz, 6x256k level 1 cache, 15mb level 2 cache, socket LGA 2011-v3 [https://www.amazon.com/gp/product/B00M1BUUMO/ref=oh_aui_detailpage_o04_s00?ie=UTF8&psc=1]
Line 32: Line 34:
  
 
==RAM==
 
==RAM==
 +
*At least as much RAM as GPUs, probably twice that much (2 * Number of cards * 11 GB [GTX 1080 Ti size])
 
*RAM: Crucial DDR4 RDIMM [http://www.newegg.com/Product/Product.aspx?Item=9SIA0ZX39C3002], 2133Mhz , Registered (buffered) and ECC, comes in packs of 4 x 32GB
 
*RAM: Crucial DDR4 RDIMM [http://www.newegg.com/Product/Product.aspx?Item=9SIA0ZX39C3002], 2133Mhz , Registered (buffered) and ECC, comes in packs of 4 x 32GB
  
 
==PSU==
 
==PSU==
 +
*Some say 1.5x-2x wattage of GPU+CPU, some say GPU+CPU+100W
 
*PSUs: Corsair RM Series 850 Watt ATX/EPS 80PLUS Gold-Certified Power Supply - CP-9020056-NA RM850 [https://www.amazon.com/gp/product/B00EB7UIXM/ref=oh_aui_detailpage_o03_s00?ie=UTF8&psc=1]  
 
*PSUs: Corsair RM Series 850 Watt ATX/EPS 80PLUS Gold-Certified Power Supply - CP-9020056-NA RM850 [https://www.amazon.com/gp/product/B00EB7UIXM/ref=oh_aui_detailpage_o03_s00?ie=UTF8&psc=1]  
  
Line 44: Line 48:
 
==Misc.==
 
==Misc.==
 
*Cases: Rosewill 1.0 mm Thickness 4U Rackmount Server Chassis, Black Metal/Steel RSV-L4000[https://www.amazon.com/gp/product/B0056OUTBK/ref=oh_aui_detailpage_o04_s00?ie=UTF8&psc=1]
 
*Cases: Rosewill 1.0 mm Thickness 4U Rackmount Server Chassis, Black Metal/Steel RSV-L4000[https://www.amazon.com/gp/product/B0056OUTBK/ref=oh_aui_detailpage_o04_s00?ie=UTF8&psc=1]
*DVDRW: Asus 24x DVD-RW Serial-ATA Internal OEM Optical Drive DRW-24B1ST [http://www.amazon.com/Asus-Serial-ATA-Internal-Optical-DRW-24B1ST/dp/B0033Z2BAQ/ref=sr_1_2?s=pc&ie=UTF8&qid=1452399113&sr=1-2&keywords=dvdrw]
+
*DVDRW (Needed?): Asus 24x DVD-RW Serial-ATA Internal OEM Optical Drive DRW-24B1ST [http://www.amazon.com/Asus-Serial-ATA-Internal-Optical-DRW-24B1ST/dp/B0033Z2BAQ/ref=sr_1_2?s=pc&ie=UTF8&qid=1452399113&sr=1-2&keywords=dvdrw]
 
*Keyboard and Mouse: AmazonBasics Wired Keyboard and Wired Mouse Bundle Pack [http://www.amazon.com/AmazonBasics-Wired-Keyboard-Mouse-Bundle/dp/B00B7GV802/ref=sr_1_2?s=pc&rps=1&ie=UTF8&qid=1452402108&sr=1-2&keywords=keyboard+and+mouse&refinements=p_72%3A1248879011%2Cp_85%3A2470955011]
 
*Keyboard and Mouse: AmazonBasics Wired Keyboard and Wired Mouse Bundle Pack [http://www.amazon.com/AmazonBasics-Wired-Keyboard-Mouse-Bundle/dp/B00B7GV802/ref=sr_1_2?s=pc&rps=1&ie=UTF8&qid=1452402108&sr=1-2&keywords=keyboard+and+mouse&refinements=p_72%3A1248879011%2Cp_85%3A2470955011]
*KVM Switch: IOGEAR 4-Port USB Cable KVM Switch GCS24U (Black) [https://www.amazon.com/gp/product/B001S2PJO6/ref=oh_aui_detailpage_o02_s00?ie=UTF8&psc=1]
 
  
  
==Other Builds==
+
==Other Builds/Guides==
  
 
* [https://news.ycombinator.com/item?id=14438472 Deep learning box for $1700] (links to https://blog.slavv.com/the-1700-great-deep-learning-box-assembly-setup-and-benchmarks-148c5ebe6415)
 
* [https://news.ycombinator.com/item?id=14438472 Deep learning box for $1700] (links to https://blog.slavv.com/the-1700-great-deep-learning-box-assembly-setup-and-benchmarks-148c5ebe6415)
 +
* [http://timdettmers.com/2015/03/09/deep-learning-hardware-guide/ A Full Hardware Guide to Deep Learning]
 +
* [https://www.oreilly.com/learning/build-a-super-fast-deep-learning-machine-for-under-1000 Cheap build]
 +
* [https://medium.com/@SocraticDatum/getting-started-with-gpu-driven-deep-learning-part-1-building-a-machine-d24a3ed1ab1e]

Revision as of 16:25, 23 October 2017


McNair Project
GPU Build
Project logo 02.png
Project Information
Project Title GPU Build
Owner Oliver Chang, Kyran Adams
Start Date
Deadline
Primary Billing
Notes
Has project status Active
Copyright © 2016 edegan.com. All Rights Reserved.


PC Partpicker build missing a lot of pieces

Motherboard/CPU/Fan

  • Not a huge deal, but used for data preparation
  • If using multiple GPUs, at least one core (two threads) per GPU
  • Should have enough PCIe slots
  • Motherboards: ASUS Z10PE-D16 [1], Dual LGA 2011 R3, DDR4 - Up to 32GB RDIMM, 16 slots
  • Chips: Intel Haswell Xeon e5-2620v3, 6 core @ 2.4ghz, 6x256k level 1 cache, 15mb level 2 cache, socket LGA 2011-v3 [2]
  • CPU Fans: Intel Thermal Solution Cooling Fan for E5-2600 Processors BXSTS200C [3]

GPU

  1. "I quickly found that it is not only very difficult to parallelize neural networks on multiple GPUs efficiently, but also that the speedup was only mediocre for dense neural networks. Small neural networks could be parallelized rather efficiently using data parallelism, but larger neural networks... received almost no speedup."
  2. Possible other use of multiple GPUs: training multiple different models simultaneously, "very useful for researchers, who want try multiple versions of a new algorithm at the same time."
  3. This source recommends GTX 1080 Tis and does cost analysis of it
  4. If the network doesn't fit in the memory of one GPU (11 GB),
  1. Might want to get two graphics cards, one for development, one (crappy card) for operating system
  1. Intra-model parallelism: If a model has long, independent computation paths, then you can split the model across multiple GPUs and have each compute a part of it. This requires careful understanding of the model and the computational dependencies.
  2. Replicated training: Start up multiple copies of the model, train them, and then synchronize their learning (the gradients applied to their weights & biases).

TL;DR

Using multiple GPUs adds a lot of complexity. It has a few benefits: possible speed ups if the network can be split up (and is big enough), able to train multiple networks at once (either copies of the same network or modified networks), more memory for huge batches. Some frameworks have much better performance with multiple GPUs (pytorch, caffe 2) while others are catching up.

Todo: ask what software, dataset size, development approach

RAM

  • At least as much RAM as GPUs, probably twice that much (2 * Number of cards * 11 GB [GTX 1080 Ti size])
  • RAM: Crucial DDR4 RDIMM [4], 2133Mhz , Registered (buffered) and ECC, comes in packs of 4 x 32GB

PSU

  • Some say 1.5x-2x wattage of GPU+CPU, some say GPU+CPU+100W
  • PSUs: Corsair RM Series 850 Watt ATX/EPS 80PLUS Gold-Certified Power Supply - CP-9020056-NA RM850 [5]

Storage

  • M.2 Drives: Samsung 950 PRO -Series 512GB PCIe NVMe - M.2 Internal SSD 2-Inch MZ-V5P512BW [6]
  • Solid State Drives: Intel Solid-State Drive 750 Series SSDPEDMW400G4R5 PCI-Express 3.0 MLC - 400GB [7] or 800GB [8]
  • Regular Hard drives: WD Red 3TB NAS Hard Disk Drive [9] - 5400 RPM Class SATA 6 Gb/s 64MB Cache 3.5 Inch

Misc.

  • Cases: Rosewill 1.0 mm Thickness 4U Rackmount Server Chassis, Black Metal/Steel RSV-L4000[10]
  • DVDRW (Needed?): Asus 24x DVD-RW Serial-ATA Internal OEM Optical Drive DRW-24B1ST [11]
  • Keyboard and Mouse: AmazonBasics Wired Keyboard and Wired Mouse Bundle Pack [12]


Other Builds/Guides