Itzikr's Blog

We know today that GPT applications that taken off and it’s caused an inflection point of IT DMs. A Gartner study polled their customers, almost half of them had already increased their AI investment just because of the fact on the potential of chat GPT and generative AI applications. We know that large enterprises are already using AI improve their own efficiency and quality from their results.

And three years from now, Gartner expects over sixty percent of applications like phone apps, web apps or Windows apps to be generated automatically from AI coding, programming generating functions. That’s only three years from now which means companies need to jump on the bandwagon today because AI and Generative AI potential is changing things for the enterprise.

Furthermore, a quote from a Bloomberg call where the CEO of ArK Invest Funds was mentioning that the killer app for any kind of AI is really automating and delivering productivity gains that businesses are looking for. She said that the real winners are those that really have these huge masses of proprietary data kind of like the Bloomberg GPT application (https://www.bloomberg.com/company/press/bloomberggpt-50-billion-parameter-llm-tuned-finance/) and high quality data that can turn be turned into value quickly. Adopting this approach is going to put a lot of these companies who are already developing and deploying AI further into a leadership position.

And AI leaders have already spent the time and effort to leverage AI, build AI teams with skillsets and more quickly plan for converting their data into value, into competitive advantage.

The challenge for many companies on-the-fence with regards to AI is to jump on the AI bandwagon to increase their business transformation and boost productivity while minimizing the learning curve, trial-and-error adoption of AI and now other forms of AI including Generative AI to farm value from their data. Not doing so risks remaining behind AI leaders.

Use cases for Generative AI are expanding as the potential of digital humans and other assistants emerges, as a consequence of the compute potential now available.

There are companies already deploying generative AI in their own program software products for their customers. While Generative AI can be used to reduce human interaction and therefore reduce the workforce, analyst guidance has tended to favor opportunities where Generative AI can identify and automate repetitive tasks rather than human-necessary tasks.

• Assistants (including Customer retail, Business operations, intelligent sales, security and infrastructure, Legal, HR and hiring, business security and data risk mitigation, call center) can reduce time spent looking for the right data needed to be acted on, as well as be trained on business data, thereby facilitating and expediting not only searches, but recommendations, insights, risks, and customer preferences.
• Developer: Code generation and code efficiency, UI/UX design are key areas that not only increase productivity but help the business transform with AI helping code and choice of designs well in advance.
• Creative, sales and marketing: from sales scripting to marketing and creative content generation, users have a wider range of inputs and outputs, not just text but also audio, video, imagery, extended to podcast content, video for games and applications and music,
• Other business and sciences: from thought leadership to technical papers, driving to faster content delivery, helps reduce the time to format and perhaps process for submission and review with real-time suggested feedback.

Today, businesses are seeing the value of traditional AI coming together as taking data set inputs and focusing on detecting the patterns, driving classifying data across thousands to a few million data points.

Now comes generative AI, which is another aspect of the overall AI landscape that is very prompt driven, driving outputs based on trained models, producing new texts, new graphics, new video from trained models, and the aspect of being very large data sets and billions of data points implies that you need a much larger infrastructure at the core.

Both AI approaches rely on the business proprietary data to drive applications. Both rely on

• a capable, performance infrastructure, that is capable of handling not just AI but also generative AI ;
• A larger, new set of framework of models for generative AI,
• Which provides a GenAI developer experience that enables them access the resources and tools quickly and in a more integrated approach
• to help them develop their AI applications, accordingly, serving the needs of the business: driving transformation and increasing productivity

Dell can help customers design for Generative AI initiatives today, offering best in class infrastructure platforms. From servers to storage to other foundational platforms, Dell solutions for AI and Generative AI align to

• enabling customers to accelerate their business outcomes so they can move quickly past GenAI deployment and start extracting value from their data
• Enabling them to drive and streamline their AI journey, from defining the data that needs to be modeled, finding out the models,, and streamlining, making repeatable processes and work streams based on generative AI.
• securing and trusting, making trusted decisions and actions from a basis of a trusted infrastructure, taking all the trusted proprietary company data to provide the right outcomes and recommendations that impacts business decision makers to change business direction and strategies.
• 
• Innovative approaches require the enterprise to rethink their AI status quo. With new ways, new models and much larger data sets, businesses are able to adopt large language model training and benefit from Generative AI applications and methodology. Their traditional AI approaches can now be bolstered and complemented with Enterprise-grade Generative AI foundations.
• Dell Technologies and NVIDIA and are excited to announce Project Helix, a unique collaboration that will make the promise of Generative AI real for the enterprise.
• Project Helix will deliver a full-stack solution, built on Dell hardware and utilizing the award-winning software stack and accelerator technology of Nvidia. Bringing together the deep knowledge and creativity of Nvidia with the global customer knowledge and technology expertise of Dell, Project Helix  will:
• Deliver full-stack scalable solutions: Deliver full-stack generative AI solutions built on the best of Dell infrastructure and software, in combination with the latest NVIDIA accelerators, AI software and expertise.
• Provide expertise and deployment guidance: Provide expertise and guidance for  enterprises to build, train, and model-tune Generative AI on-premise to help solve specific business challenges.
• Assist from development to multi-site deployment: Assist enterprises with the entire GenAI lifecycle, from infrastructure provisioning, large model development and training, pre-trained model fine tuning, to multi-site, multi-cloud model deployment (and consumption) and large model inferencing.
• Deliver trusted AI outcomes with integrity: Helix is aimed at helping deliver Trusted AI outcomes, that the decision outcomes are made from objective data inputs (assessed and sanitized according to a defined set of “guardrails”) which assures data integrity. These decisions elevate the quality of those results to being trusted as well as providing the rationale for the decisions/recommendations provided by the model.
• Types of guardrails
• Topical guardrails prevent apps from veering off into undesired areas.
• Safety guardrails can filter out unwanted language
• Security guardrails enforce that confidential information stays that way.
• Ensure security and privacy of business data: Ensure security and privacy of sensitive and proprietary company data, as well as compliance with government regulations.
• Flexible Capex/Opex consumption with APEX flex consumption approaches
• To that end, project Helix is about

• Accelerating business outcomes with generative AI, enabling
• Best-in-class infrastructure
• Full-stack integrated AI suite for LLM
• Drive rapid business transformation
• Scaling GenAI for business-wide impact: Delivering GenAI at scale for business-wide impacts
• Develop rapidly into results
• Validated design reduces complexity
• Convert data into value
• And Securing Foundations for trusted decisions and approaches to enable trusted decisions that impact the business direction
• Secure, high-quality on-premises
• Reduce data loss/compromise risk
• Trusted decisions, refined with guardrails

• With Project Helix, Dell and NVIDIA enable organizations to automate complex business processes and workstreams, automatically extract value from data lakes, improve customer interactions and unlock new possibilities with better machine intelligence.
• Together, Dell and NVIDIA are leading the way in driving the next wave of innovation in the enterprise AI landscape with AI-grade Enterprise solutions..

The PowerEdge XE9680 is our highest performance server in the portfolio which is designed to have the fastest Time to value
and is our no compromise AI Acceleration product.

The XE9680 is Dell’s first entry of a 8 way GPU SXM server that will offer both Nvidia’s next generation H100 sxm GPU’s as well as their current A100 offering, providing our customers optionality of performance and power.

This server can handle the largest and most demanding AI, ML, DL training/HPC modeling & simulation initiatives, and now Generative AI, all in a 6U air cooled package.

Once again this will feature next gen technologies with Intel’s 4^th Generation Xeon processors, DDR5 memory, E3 storage but also has 10 PCIe Gen 5 slots enabling 1:1 GPU: NIC ration which means you get max GPU utilization with smooth data transfer in and out resulting in minimizing latency.

This product is great for HPC super computing clusters and the largest data sets for AI, used for large scale natural language processing, recommendation engines and neural network. Also great for conversational AI, chatbots, voice recognition and audio applications

Verticals: Generative AI/ChatGPT, CRISP, CSP Finance, HPC, Healthcare, Federal, Research/Universities

1- Dell MLCommons MLPerf Inference v3.0 vs v2.0, BERT 99.9 v3.0 Offline scenario renders 843 percent more improvement compared to PowerEdge XE8545 four-way NVIDIA A100 SXM. March 2023, https://infohub.delltechnologies.com/p/dell-servers-excel-in-mlperf-tm-inference-3-0-performance/

2- Dell PowerEdge XE9680 with 8x Nvidia H100 GPUs delivers 6.4x (+543%) more TeraFLOPS than the PowerEdge XE8545 with 4x Nvidia A100 GPUs in HPL benchmark. Testing conducted by Dell in March of 2023. Performed on PowerEdge XE9680 with 8x Nvidia H100 SXM5-80GB and PowerEdge XE8545 with 4x Nvidia A100-SXM-80GB.

3-NVIDIA, LLM training test, For a cluster of 4K GPUs: 1T parameter model on A100 takes 1 month, while only 1 week for same size H100 cluster. NVIDIA conducted test, 2023.

• 175B parameter model on A100 takes 6 days, for H100 1.6 days
• 40B parameter model days on A100 become hours on H100

4-Dell white paper, Launching & Running Large Language Models on a single Dell server produces outstanding results, 2023. Throughput in tokens per second of the BLOOM-176B large language model (LLM) on the Dell Technologies PowerEdge XE9680 server vs. the Jean Zay supercomputer using the same benchmarking code and same model instantiation with HuggingFace Accelerate and float16 precision. In this instance, the XE9680 demonstrated a 70% increase in throughput as compared and the Jean Zay supercomputer, regardless of batch size. https://www.delltechnologies.com/asset/en-us/products/servers/industry-market/launching-llms-on-poweredge-xe9680.pdf

1. Based on Dell Technologies internal analysis comparing the max bandwidth of the ECS EXF900 (511 MB/s) to the max bandwidth of the ECS EX300 (24 MB/s) for 10KB writes, November 2020. Actual performance will vary.

2. Based on an Enterprise Strategy Group Technical Review commissioned by Dell EMC, “Accelerating the Artificial Intelligence Journey with Dell EMC Ready Solutions for AI”, August 2018.

3-Dell spec sheet. Raw capacity up to exabytes (ECS) or up to 186PB in a 252-node cluster (PowerScale)

CloudIQ delivers intelligent infrastructure insights by combining machine intelligence and human intelligence, enabling algorithms to be continuously updated leveraging Dell Technologies product and subject matter expertise, combined with industry best practices to address the most potentially impactful issues. CloudIQ helps organizations:

• Reduce risk by leveraging machine learning to identify and avoid issues and expedite troubleshooting
• Plan ahead by anticipating business needs and avoiding outages
• And improve staffing and infrastructure productivity via a single pane of glass view across the environment

The PowerEdge integrated Dell Remote Access Controller (iDRAC) delivers advanced, agent-free local and remote server administration.

• Embedded, remote, function-rich, agent-free management.
• 1:1 out-of-band Baseboard Management controller on each PowerEdge server
• Monitor > 180 server metrics.

OpenManage Enterprise (OME), 1: many on-prem systems management and automation.

• Server Life Cycle Management
• Manage Up to 8,000 devices
• Easy server deployment, configuration, firmware updates, and real-time health monitoring.
• Comprehensive RESTful API enables customizable automation and solution integration

OpenManage Enterprise Power Manager plugin: Analyze and prevent a server-related power and/or thermal outage

• Analyze historical and present power consumption and thermal readings
• Define power policy and budgeting for automation
• Platform subcomponent power reporting. Including Fans, storage, NICs, GPU, FPGA, etc.
• View power, virtual machine energy, facility energy, overhead energy, and associated cost of VMs and VM groups.

Address sustainability

• Diminish carbon footprint
• Identify zombie servers
• Reduce server TCO

Get a server into data center production faster

• Know where there is available power/thermal headroom immediately
• Provide accurate power consumption for chargeback

Storage

For nearly two decades, we’ve been successfully helping our customers solve their unstructured data challenges. The market data and industry analyst commentary reflect that commitment and proven track record. To date, we’ve shipped over 17 exabytes of capacity, including a single customer managing more than 1EB of unstructured data on OneFS. The bottom-line: We have unmatched experience in addressing these challenges, and every day we think about how to help our customers overcome them in a sustainable way.

OneFS has always been a highly flexible software platform which was designed from the ground up to provide agility when managing data at scale. In fact, OneFS has historically provided many of the benefits you would usually associate with a “software defined” solution, such as: single pool aggregation of multiple types of media, policy-based management and automation, dynamic scaling of performance and capacity, and a distributed architecture – just to name a few.

We provide cloud consumption options not available on scalable file solutions until now. Our multi-cloud offering delivers the full fidelity experience of OneFS in a hosted, managed service with simultaneous connectivity on the same IP range with Google, AWS and Azure. Our native integration with Google enables a cloud-native experience where Google handles all provisioning, management and billing. All these solutions can be run as cloud-only or interoperate with on-prem instances of OneFS for a hybrid deployment.

Dell Technologies Services helps customers drive their businesses forward by meeting them where they are in their digital journey. We collaborate with customers on IT solutions that give them an advantage today and prepare them for what’s ahead.

• We provide forward-thinking strategies, flexible choices and consistent execution to ensure the customer’s transformation stays on course.
• Our deep expertise and connected technology fuel the services that empower businesses to be resilient and thrive.
• We offer a full, integrated suite of services to help customers accelerate deployment and adoption of technology, while minimizing costs with planning, training and residents.
  • Services for the full lifecycle – capabilities from edge to core to cloud (portfolio of services that stretch from Consumer to SMB to Global Enterprise)
  • Connected to the largest tech ecosystem on the planet – unparalleled leverage from the industry’s largest data set, and machine learning that helps predict issues, navigate complexity, and streamline operations
  • Different ways to consume – flexibility to choose traditional or as-a-Service
  • The simplicity and ultimate experience of fully managed cloud-based capabilities
  • Custom capabilities when a customer requires something unique
• We have massive, global Services coverage with more than x number of services professionals and partners serving customers in 180 locations.

Services offerings:

• Consulting: Our Consulting Services accelerate time-to-value for our customers’ transformations. Consultants partner with customers to advise, plan and execute their digital, IT and workforce transformations – whether they are modernizing their data centers, optimizing their application environments, or transforming their workforce.
  • 10,000 transformation projects completed
  • 1,000+ IT strategy workshops completed
• Deployment: Our deployment experts help customers get the most out of new technology quickly – supplementing their IT resources and accelerating technology adoption without disrupting user productivity. Whether adding new equipment to a data center or migrating end users to the newest laptops, customers can trust our experts to lead deployments from basic hardware installation through planning, configuration, complex integrations and data migration. ProDeploy Plus for PC customers can reduce deployment time by up to 50%. We configure more than 11M PCs annually.
  • Deploy up to 66% faster with ProDeploy/ProDeploy Plus for Enterprise
• Support: Our support services are built on a foundation of artificial intelligence (AI), machine learning and data analytics, changing the way customers look at saving time and increasing availability. With the support expertise, insights and technologies we’re known for across the globe, we’re able to help customers maximize productivity and uptime, extending the customer’s IT team and enabling them to address issues before they impact their business.
  • We have the most complete PC support available in the industry including automated, proactive and predictive support.
    • And with recent upgrades, IT admins can now use telemetry-driven insights and recommendations to remotely act on developing IT issues, anticipate end-user needs and reduce resolution time.
  • Our enterprise-class support is focused on quality, speed to resolution, and collaboration:
    • 96%+ customer satisfaction
• Managed Services: Our managed services portfolio spans from end user devices to the data center and into the cloud. We have decades of experience delivering managed services and support over 120 million devices in the market. With proven processes and best practices, we can quickly implement best-in-class solutions that will optimize performance in the IT environment. Our services encompass all aspects of IT operations – from employee service desks, to data backup and recovery solutions, to cloud implementation and management as well as high-performance computing. This allows customers to shift focus from time-consuming, daily tasks to strategic business priorities.

• Education Services: Dell Technologies Education Services offers training and certifications to help customers develop the skills they need to transform their businesses. Skills development and validation include technology topics such as cloud, data storage, servers, networking, data protection and big data analytics.
• Residency Services: Dell Residency Services provide extensive options to meet business needs from infrastructure solutions and end-user technology to business outcomes in the areas of Multicloud, Modern Workforce, Security and more. Directed by the customer, residents act like an extension of the IT staff to enhance internal capabilities and resources. Bringing deep technical expertise, residents help businesses realize faster adoption and maximized ROI of new technology.  Resident engineers are notable in helping IT organizations quickly implement new features and functionality that improve technology utilization and value – delivering better performance for the business. In fact, a recent IDC study found that Residency Services provide measurable benefits including improvement in staff efficiency and reduced IT service delivery cost. With flexible engagement options and a rigorous selection process to match the best resource, Dell Residency Services are the perfect match to meet the unique needs of the business.
• Security: With the increasing digitization of businesses and the increasing value and scale of the data they generate, having a holistic cybersecurity strategy has never been more important. Organizations need to practice defense-in-depth to reduce business risk which encompasses everything from vulnerability management to how they’ll recover after a cyberattack. Dell Services focuses on helping customers reduce their attack surface and shift to a proactive mindset by continuously addressing vulnerabilities, protecting data and assets; and enabling continuous vigilance and response. By reducing business interruptions and being a trusted advisor to customers, they can focus more on their strategy and less on day-to-day tasks.
• Asset Recovery Services helps customers securely and responsibly retire their IT assets, regardless of brand – while protecting both their business and the planet. Planning for this early, when assets are at their maximum value, allows them to unlock funds that can be put toward future innovation.

Dell Technologies is a leader and helps customers with solutions that can drive their performance, harness AI and accelerate their initiatives.

With the right foundation, customers can accelerate their time to results with building blocks from high performance servers, fast GPU accelerators to complete turnkey, certified and validated solutions for their environment.

Deploying technology, innovation starting with the compute helps customers with the ample horsepower needed to solve larger and greater complexity problems, within HPC as well all other AI initiatives. With the continuous increase in data generation across the organization, more compute is needed to extract value for business initiatives.

With the best AI technology foundation as well as a framework of operations and suites to help virtualize AI, businesses can readily make available resources and democratize business wide. Suites like VMware vSphere and NVIDIA AI for Enterprise rapidly helps drive AI resources to those who need the access.

IDC found that AI leaders recognize that many of the demanding AI workloads require on-premise compute, shift AI-intensive algorithms to that infrastructure and thereby own their infrastructure for optimal on-premise performance. While businesses can also utilize the cloud, for massive data and compute resources, the operations remain on-premise for lowest latency and fastest performance.

These benefits help organizations be more competitive in their business, from faster insights which foster faster decision-making which drives more value sooner. Time-to-value is quickened furthering AI leaders as well as AI evaluators in their transformation

To understand how the new Dell solution is superior, watch the interview with Andrea Booker, Director of Product Management, Accelerator Portfolio, Dell Technologies & Steen Graham, CEO & Founder, Scalers AI, sit with Lisa Martin & Dave Vellante at Dell Technologies World 2023