High speed and cost effective computing for Artificial Intelligence applications

In this article, let’s discuss how to boost speed and cost effective computing for Artificial Intelligence applications.

Artificial intelligence solutions are making its way into practically every aspect of our life. This is primarily due to the fact that we live in a data-driven ecosystem now. We generate so much data that we are incapable of handling it manually. This calls for deep learning and machine learning based solutions that can do all the heavy lifting. But have you ever thought about why we see a lot of these solutions being introduced nearly every day now? Why not before? This isn’t a new concept; we had the baseline research outlined years before. So, why now? Because of the introduction of powerful hardware at very low prices.

Deep learning and machine learning research require highly capable hardware for its learning process. The learning or training phase is the most time-consuming part of AI research and the choices a researcher makes regarding this procedure ultimately decides how good the model is. The silicon industry is abuzz with new and improved hardware tailor-made for deep learning applications. Deep learning research has fuelled the development and introduction of newer breeds of GPUs and ASICs into the market.

Processor chips are an integral part of the hardware used for machine learning application, as it is tasked with the role of executing the logic behind the algorithm. A CPU is designed to execute only a few, very complex and intricate tasks effectively. This is what often causes them to perform poorly when it comes to training deep models. Enter the GPU or Graphics Processing Unit.

GPUs are designed to process multiple yet simpler instructions more efficiently. Most machine learning and deep learning algorithms perform matrix multiplication which are simple but extensive and in such cases the GPUs always outperform CPUs. On the back of this ability was introduced newer deep learning libraries with GPU computing as one of its most advertised features.

Tensorflow from Google has special versions with GPU compatibility that lets you train models with your GPU memory. This trend accelerated when Nvidia came out with their neural network computation platform called CUDA along with support for their GPUs. Now with adequate proprietary support from the vendor, these chips performed better and training was more efficient. Parallel processing was smoother and model development became easier. But the user was often confused as to what the right hardware for their task would be and how can they set up the necessary environment to do AI research. Often times they are confused by the varying prices in the hardware and the poor technical support from third-party OEM vendors. These concerns were addressed with the introduction of deep learning development boards and this article focuses specifically on the Nvidia Jetson TX2 board.

At Accubits, we develop AI solutions for a multitude of scenarios like real-time video analytics, chatbots, and stock market analysis. For us, the key is to prepare and deliver the best solution in the shortest time possible. Most of the existing solutions we developed have CUDA compatibility and the Nvidia’s latest Jetson TX2, with its out-of-the-box capability for neural computing, seemed like the appropriate choice. But the initial setup and configuration of the module were not easy and this is the main focus of this post.

Through this article, we would like to share with you the issues we faced during the initialization and how we solved those as well as how Accubits is accelerating productivity with the new hardware so that your journey with the Jetson boards are as smooth and efficient as possible.

Any hardware is as good as the software. For the Jetson, Nvidia offers a souped-up version of Ubuntu, called Jetpack, which comes with pre-built support for CUDA, OpenCV, CuDNN libraries etc, which saves us the hassle of having to manually install and configure them. trust me it’s tedious. Jetpack 3.2 is the latest and the installation gets you Ubuntu 16.04 with all the aforementioned packages. Anyone who wishes to get started on development with Jetson boards, the github page of ‘Jetsonhacks’  is a great place to find a lot of support materials and the forum is a great place to find solutions for your problems. Detailed tutorials on installing Jetpack can also be found there.
So let’s break down the issues we faced and some bizarre out-of-the-box configurations within the Jetson TX2.

1.Installing python packages on boards based on ARM architecture

We had issues with installing tensorflow and Anaconda packages on the Jetson TX2 as it’s based on the ARM architecture and Anaconda or miniconda packages are not available for those. On top of that, tensorflow wheel files are not configured for them. To fix the tensorflow install issue, we sourced a prebuilt wheel file specifically designed for Jetpack 3.2. Alternatively, you can build it from source. For installing all the packages offered by anaconda, we created a list of essential library files and squeezed it into a text file so that it can be installed via the pip package installer in one go. It is available here.

2. One single core is active as default.

We noticed that the Jetson TX2 showed 4 CPU cores in System monitor, but only one was active. This can be fixed by running the following commands,

echo 1 > /sys/devices/system/cpu/cpu1/online
echo 1 > /sys/devices/system/cpu/cpu2/online

This will enable the rest of the CPU cores. This feels a bit strange and should have been the default configuration.

3. Pre-installed OpenCV was not functioning.

Jetpack 3.2 comes packed with OpenCV, but Python was unable to access it. The simple solution was to use the script provided on “jetsonhacks”, that builds and runs OpenCV from source, with compatibility for Jetpack OSs.

Those were some of the issues we faced while configuring our newly purchased Jetson TX2 and we thought this article could help save time for at least some of you out there and you wouldn’t have to go through the agony and countless cups of coffee like us during the initialization. These are slight niggles but could be a major headache when you have a deadline to meet.

Accubits is already driving development on purpose-built systems for deep learning research and plan to expand our inventory with more robust and capable hardware from multiple vendors. This helps us make scalable and tailor-made solutions for our clients as per their requirements and usage scenarios. Such developer-focused hardware platforms ease the process of pushing solutions and helps us deliver our best.