Changes

Jump to navigation Jump to search
528 bytes added ,  15:35, 15 May 2019
no edit summary
==DocumentationIntroduction== This page details the build of our [[DIGITS DevBox]].
The documentation from NVIDIA is here:
*https://docs.nvidia.com/dgx/digits-devbox-user-guide/index.html
*https://developer.nvidia.com/devbox
 
Hardware specs from other builds:
*https://www.azken.com/download/DIGITS_DEVBOX_DESIGN_GUIDE.pdf
**https://www.reddit.com/r/buildapc/comments/3gewmz/build_complete_nvidia_digits_devbox/
*https://cellmatiq.com/?p=155
*http://graphific.github.io/posts/building-a-deep-learning-dream-machine/
However, unfortunately, the form to get help from NVIDIA is closed [https://info.nvidianews.com/early_access_nvidia_3_15.html][https://www.reddit.com/r/buildapc/comments/3gewmz/build_complete_nvidia_digits_devbox/][https://www.pyimagesearch.com/2016/06/06/hands-on-with-the-nvidia-digits-devbox-for-deep-learning/]. And most of the other specs are limited to just the hardware [https://www.reddit.com/r/buildapc/comments/3gewmz/build_complete_nvidia_digits_devbox/][https://cellmatiq.com/?p=155][http://graphific.github.io/posts/building-a-deep-learning-dream-machine/][https://pcpartpicker.com/b/FGP323]. The best instructions that I could findwere:
*https://medium.com/yanda/building-your-own-deep-learning-dream-machine-4f02ccdb0460
The DevBox is currently unavailable from Amazon [https://www.amazon.com/Lambda-Deep-Learning-DevBox-Preinstalled/dp/B01BCDK1KC], and at around $15k buying one is prohibitive for most people. Some firms, including Lamdba Labs, Bizon-tech, are selling variants on them, but the details on their specs are limited (the MoBo and config details are missing entirely): *[https://lambdalabs.com/deep-learning/workstations/4-gpu**https://pcpartpicker.com/b/FGP323*], Bizon-tech [https://bizon-tech.com/us/bizon-g3000], are selling variants on them, but their prices are high too and the details on their specs are limited (the MoBo and config details are missing entirely).*http://deeplearningbox.com/*https://wwwBut the parts cost is perhaps $4-5k now for the original spec! So this page goes through everything required to put one together and get it up and running.amazon.com/Lambda-Deep-Learning-DevBox-Preinstalled/dp/B01BCDK1KC (currently unavailable)
Unfortunately, the form to get help from NVIDIA is closed:*https://info.nvidianews.com/early_access_nvidia_3_15.html*https://www.reddit.com/r/buildapc/comments/3gewmz/build_complete_nvidia_digits_devbox/*https://www.pyimagesearch.com/2016/06/06/hands-on-with-the-nvidia-digits-devbox-for-deep-learning/==Hardware==
At around $15k (the Lamdba variants go from $10k to $23k), buying one is prohibitive for most people. But the parts cost is perhaps $5k now for the original spec.===Description===
==Hardware==We mostly followed the original hardware spec from NVIDIA, updating the capacity of the drives and other minor things, as we had many of these parts available as salvage from other boxes. We had to buy the ASUS X99-E WS motherboard (we got the ASUS X99-E WS/USB variant as the original wasn't available and this one has USB3.1), as well as some new drives, just for this project.
We mostly followed opted to use a Xeon e5-2620v3 processor, rather than the Core i7-5930K. We had both available and both support 40 channels and mount in the original hardware spec from NVIDIALGA 2011-v3 socket, updating and have 6 cores, 15mb caches, etc. Although the capacity of i7 has a faster clock speed but the drives and other minor thingsXeon takes registered (buffered), ECC DDR4 RDIMMs, as which means we had many of these parts available as salvage from other boxescan put 256Gb on the board, rather than just 64Gb. Though For the GPUs we had have a TITAN RTX and an older TITAN Xp available to start, and we can add a 1080Ti later, or buy some additional GPUs if needed. We also put the ASUS X99whole thing in a Rosewill RSV-E WS motherboard (as well as some new drives) just for this projectL4000 case.
We opted to use a Xeon e5-2620v3 processor, rather than the Core i7-5930K (which we did have available). Both support 40 channels and mount in the LGA 2011-v3 socket, and both have 6 cores, 15mb caches etc. The i7 has a faster clock speed but the Xeon takes registered (buffered), ECC DDR4 RDIMMs, which means we can put 256Gb on the board, rather than just 64Gb. For the GPUs we have a TITAN RTX and an older TITAN Xp available to start, and we can add a 1080Ti later, or buy some additional GPUs if needed. We also put the whole thing in a Rosewill RSV-L4000 case.===Parts List===
{| class="wikitable sortable"
| 2 || ARCTIC F8 PWM Fluid Dynamic Bearing Case Fan, 80mm PWM Speed Control, 31 CFM at 22dBA
|}
 
===Build notes===
Old notes on a prior look at a [[GPU Build]] are on the wiki too.
There weren't any particularly noteworthy things about the hardware build. The GPUs need to go in slots 1 and 3, which means they sit tight on each other. I We put the Titan XP in slot 1 (and plugged the monitor into its HDMI port), because then the fans for the Titan RTX (which I we expect will get heavier use) are in the clear.The fans were set up on a push and pull arrangement, and the hot-swap bay was put in the center position to allow as much airflow past the GPUs as possible. ===BIOS===
The initial BIOS boot was weird - the machine ran at full power for a short period then powered off multiple times before finally giving a single system beep and loading the BIOS. It may have been memory checking or some such.
==We did NOT update the BIOS==. It didn't need it. The m.2 drive is visible in the BIOS and will be used as a cache for the RAID 5 array (using bcache). The GPUs are recognized as PCIe devices in the tool section. And all of the SATA drives are being recognized.
The machine boots to BIOS. I We then made the following changes:*The GPUs are being recognized - see the tool section!*All of the SATA drives are being recognized
*Set the three hard disks to hotswapable enable
*Set the fans to PWM, which drastically cuts down the noise, and set the lower thresholds to 200 (not that it seemed to matter, they seem to be idling at around 1k)
*Listed List the OS as "Other OS " rather than windows, and set enhanced mode to disabled
*Delete the PK to disable secure boot
*Change the boot order to be CD first (not as UEFI, and then the Samsung 850)
 
Notes:
*We will do RAID 5 array in software, rather using X99 through the BIOS
*The m.2 drive is visible in the BIOS and will be used as a cache for the RAID 5 array (using bcache)
==Software==

Navigation menu