Itzikr's Blog
Dell Storage, PowerStore, PowerFlex PowerMax & PowerScale, Virtualization & Containers Technologies
A guest post by Nomuka Luehr Inspired by the pain of standing in long grocery checkout lines, David Humble invented the self-service till in 1984. As artificial intelligence (AI) research […]
A guest post by Nomuka Luehr
Inspired by the pain of standing in long grocery checkout lines, David Humble invented the self-service till in 1984. As artificial intelligence (AI) research regained popularity in the 1980s, these basic tellers transformed into intelligent devices with AI-powered cameras that can analyze facial and hand expressions to detect misscanned items and theft.
At this rate of mainstream adoption of AI and other data science applications at edge locations like your local grocery store, one can only imagine the high-tech user experience awaiting future consumers. Whatever comes next, it’s no doubt that IT infrastructure will be critical to providing the necessary support to enable such developments. That’s why we wanted to explore these infrastructure technologies and their many applications across data analytics, AI, Machine Learning (ML), and Deep Learning (DL) through this new series.
To kick off the series, this blog will explore recent AI developments occurring at the edge, the role of 5G, and the benefits of deploying related workloads with PowerEdge.
The Edge, AI and 5G oh my!
Deploying AI at the edge brings a world of opportunities, especially with the arrival of 5G networks. With edge deployments of AI, you can get real-time actionable insights at the point of decision with lower cost and latency when compared to traditional data transfer architectures. Through 5G networks, service providers can gain insights faster via IoT devices, making consumer applications like real-time monitoring (like the self-checkout till from earlier) possible and more reliable.
Depending on the use case, AI is deployed on a broad range of smart devices like phones, drones, speakers, appliances, and industrial machines. The intelligence in these “edge” deployments usually resides on an edge server, not the smart device itself. Therefore, choosing the right servers for your specific edge deployment case is critical to maximizing performance. Beyond complex use cases like Augmented/Virtual Reality experiences, 5G will heavily impact enterprise use cases like self-service applications and dynamic warehouse inventory planning through capabilities like ubiquitous machine-to-machine connectivity and ultra-low-latency application execution. Ultimately, as AI and 5G technologies continue to mature and bring endless new opportunities, now is the perfect time to explore the edge.
PowerEdge Edge Essentials
Let’s explore how PowerEdge can help you leverage new data sources and achieve your unique edge and AI goals
Get the best of edge and cloud
While the increase of new distributed edge solutions is quickly rising, there are still IoT use cases that are best suited for cloud-centric and hybrid architectures. Therefore, most organizations will need a strategy that embraces different types of edge implementations and multiple cloud options to provide the best services at the lowest cost. As a leader in edge, cloud, and hybrid technologies, PowerEdge is well suited to provide a comprehensive, customized approach to help organizations thrive in this diverse market.
Access a broad portfolio of edge optimized servers
The PowerEdge portfolio includes multiple servers that are optimized for the edge. These servers are designed to bring Tier 1 data center features to your edge & telecom environments, additionally, full PowerEdge features built with a minimum footprint and full enterprise compute capabilities optimize valuable edge space and deliver a targeted virtual experience. These servers include:
Simple management for a diverse portfolio
The broad portfolio also comes with easy management tools to help deploy your server fleet and keep them secure. OpenManage Enterprise can be installed on the data center and can connect to edge devices using a WAN solution. OpenManage Enterprise can be used to discover and maintain server inventory, monitor the health of the server as well perform BIOS and firmware updates. Also, you can use the Integrated Remote Access Controller (iDRAC) for secure remote server management of the edge servers. In addition to using virtual media and console for deployment as explained earlier, iDRAC can also be used to update BIOS and firmware and monitor PowerEdge servers.
PowerEdge Edge Integrations
Beyond the essentials, PowerEdge has worked with a multitude of partner companies to bring an enhanced edge experience.
Easily Scale AI at the Edge with Nvidia Fleet Command
While the edge provides many opportunities, limited staffing and harsh environments can make it difficult to maintain AI deployments. To address this issue, Dell has partnered with NVIDIA to enable their Fleet Command deployments on compatible PowerEdge servers. NVIDIA Fleet Command is a hybrid cloud platform that enables admins to manage and scale AI deployments from dozens to million of servers and edge devices.
NVIDIA Fleet Command offers three key management benefits:
If you’re interested in powering your edge with PowerEdge and the NVIDIA Fleet Command, you have access to a wide selection of servers that are qualified by Dell through NVIDIA Certified Systems and NGC Ready Systems validation programs. Beyond the edge optimized servers we saw earlier (R650, R750, XE2420, XR2, XR11 and XR12), you can choose from the following additional servers to be a part of your NVIDIA fleet command deployment: R650, R750xa, R6515, R6525, R7525, R940xa, DSS 8440, XE2420, R640, R740 and R740xd servers.
Take HCI to the edge with VxRail
Operating at the edge, in space constrained, remote, and sometimes harsh environments, without compromising functionality or performance comes with many challenges. In these remote locations, systems must be especially easy to operate, manage, and update. Hyperconverged Infrastructure (HCI) offers integrated server & storage technology and provides full lifecycle management and data protection at the edge, thereby replicating the operational efficiency of a traditional data center. The new Dell EMC VxRail D Series is the first and only ruggedized HCI appliance developed with and optimized for VMware environments. The solution delivers a simple, agile, and, more importantly, a familiar user experience that integrates seamlessly with existing VMware infrastructure. Users can manage traditional and cloud-native applications across a consistent infrastructure.
Centralize data with the Confluent platform
The Confluent Platform enables data science and IT practitioners to collaborate on building real-time data pipelines and streaming applications by integrating data from multiple sources and locations into a single, central event streaming platform. Confluent simplifies connecting data sources to Kafka, building applications with Kafka services, and securing, monitoring, and managing Kafka infrastructure. Sample configurations for PowerEdge server deployments ensure your Confluent Kafka architecture is robust and takes advantage of the most recent advancements in server technology. These include:
Want to know more?
As we briefly explored, PowerEdge is well equipped to take your AI workloads to the edge. To learn more, check out the following resources:
Data Science Part 2: Training Deep Learning Models with PowerEdge
Are you smarter than an AI? Well, it is widely agreed upon that AI intelligence is currently on par with that of adult humans. This means that we and machines perform comparably on tasks that require skills like logical reasoning, problem-solving, and abstract thinking. This long-awaited achievement for AI development is primarily attributed to Deep Learning (DL) advancements.
What is Deep Learning?
Unlike other machine learning (ML) methods, DL uses a more complex structure of layered algorithms called neural networks that analyze data with a logical design similar to how our brains draw conclusions and learn over time. If you’re interested in reading more about the different types of neural networks and how they work, check out this article. While other ML methods rely on engineers to adjust and increase the accuracy of their models, neural networks can learn from previous right and wrong predictions by employing a process called backpropagation where they manipulate the weights and biases of their algorithms until the correct output occurs. DL’s high accuracy and less need for human intervention ultimately means that we can tackle more complex use cases with better outcomes.
Although it requires less maintenance over time, DL typically requires more computing power than other ML methods. We may take it for granted but mimicking the billions of interconnected cells within our brain that work parallelly to take in new data, recognize patterns, and make decisions is no easy feat for computers. DL requires vast amounts of data and complex software simulations to ensure consistently accurate results. Nevertheless, companies are more than willing to invest in the necessary hardware and software systems, as continued AI augmentation through DL is expected to generate trillions in business value through more effective and reliable methods of data analysis and decision making.
How can organizations make the best use of Deep Learning?
Optimal application of DL techniques has already enabled great successes in many fields, such as computer vision through object detection, commerce through recommendations, and autonomous driving through combining a multitude of techniques. However, reaping such rewards is challenging without the right tools. DL workloads are not simple to carry out, and related system components must be carefully selected and tuned for each unique use case. Therefore, organizations are faced with many complex choices, including those related to data, software, performance analytics, and infrastructure components—each with varying impact on accuracy, ease and time of deployment, and business impact.
With these pain points in mind, I thought we’d take some time to explore how engineers can use available benchmarking tools to improve the efficiency of their models, particularly when it comes to choosing and sizing the appropriate infrastructure elements. Before we check out the components that are tasked with running these demanding workloads, let’s explore the process of implementing a DL model.
How is Deep Learning implemented?
Just like how we attend school, DL models must undergo training on sample data sets to discover patterns and modify their algorithms for the desired output before tackling new instances. The training phase involves several iterations and a substantial amount of data, which usually requires multi-core CPUs or GPUs to accelerate performance for high accuracy. After training, the model moves onto inferencing, where it can be deployed on FPGAs, CPUs, or GPUs to perform its specific business function or task.
As DL models cycle between training and inferencing to continuously adapt the model to changes in data, ensuring the efficiency of both phases is critical to the system’s overall performance. In this blog, we’ll explore how to evaluate and choose system components to help enhance the efficiency of the DL training phase. Stay tuned for the next blog when we explore similar methods for ensuring optimal performance for DL inferencing!
How can I pick the right infrastructure tools to train my Deep Learning model?
Optimizing a platform for DL requires the contemplation of many variables as it allows for a range of statistical, hardware, and software optimizations that can significantly alter the model’s learning processes, training time, and accuracy. Additionally, the software and hardware systems available are so diverse that comparing their performance is difficult even when using the same date, code, and hyperparameters.
Thankfully, there is MLPerf, an industry-standard performance benchmark system for ML that overcomes related challenges to help fairly evaluate the performance of different DL systems. To do so, MLPerf provides an agreed-upon process for measuring how quickly and efficiently different types of accelerators and systems can train a given model. MLPerf is especially useful and has gained widespread popularity for providing accurate benchmarking results for multiple DL domains, including image classification, NLP, object detection, and more:
For each given domain, MLPerf will measure performance by assessing and comparing the total amount of time that it takes to train a neural net model for a given domain from the list above to reach target accuracy. To help data scientists pick the appropriate infrastructure components that will achieve their DL goals, Dell submitted the following servers for benchmarking across various domains:
Here’s a quick glimpse at the benchmarking results of the various Dell EMC systems for the Natural Language Processing (NLP) domain, a field of ML that’s focused on interpreting, responding, or manipulating (i.e., translating) human language. Click here to find benchmarking results for all available domains.
As you can see, varying the server and system setup, processor type and count, and accelerator type and count can significantly impact a model’s required training time despite using the same software and dataset.
As you explore the MLPerf benchmark results, I recommend keeping the following two things in mind:
What’s next?
This blog explored how data scientists can use MLPerf Benchmarks to optimize their DL training process. Stay tuned for Data Science Part 3 as we move on from training to discover how to choose the right infrastructure components for optimal DL inference performance!
Want to know more about Deep Learning? Check out the resources below
