Google Cloud Next 23 | Recap

A day after OpenAI launched the enterprise version of its ChatGPT model, Google Cloud Next 2023 kicked off, showcasing the company’s latest product launches and exemplifying the race for the most dominant position in the field of AI.

From Plain Concepts, we didn’t want to miss the event that, after three years, returned to its in-person format in the city of San Francisco. Here is a summary of some of the most relevant news.

AI platforms and tools

Duet AI

Duet AI, Google Cloud’s innovative artificial intelligence platform designed to revolutionize how users work with the cloud, aims to improve productivity, provide competitive advantages, and increase operational efficiency. Duet AI has been integrated into a wide variety of Google Cloud services and applications through its expansion.

One of the key areas of focus for Duet AI is application development. The platform offers expert assistance throughout the software development lifecycle, including code generation, source citation, testing, and API design. Developers can use natural language to understand and improve code, as well as to generate unit tests. This support is available in various development environments, helping maintain workflow and minimize disruption.

Duet AI also streamlines application modernization by helping with code refactoring. For example, it can accelerate the migration of legacy applications to the Google Cloud, simplifying tasks such as converting code from one language to another. This feature is especially valuable for companies that want to upgrade their technology systems more efficiently.

The platform integrates with API management, allowing APIs to be designed and published using natural language requests. This simplifies the orchestration of communications between services and facilitates the integration of applications in an enterprise environment.

Duet AI is also used to simplify the operation and management of infrastructure and applications. It helps automate deployments, configure applications correctly, and resolve problems efficiently. For example, Cloud Monitoring can translate natural language requests into PromQL queries to analyze time-series metrics, making it easier to identify and resolve problems.

In the field of data analysis, Duet AI is a valuable tool for analysts. In BigQuery it provides contextual assistance for writing SQL and Python, enabling faster and more effective analysis. It can generate complete functions, suggest code completions, and provide detailed explanations of SQL queries, making it easier to understand the data and its patterns. Duet AI also integrates with Vertex AI, further optimizing data queries and text analysis. This facilitates semantic search and recommendation queries based on BigQuery data.

Duet AI is also being introduced in Looker to help business users analyze data more efficiently. It offers conversational data analysis, automatic presentation generation, and report-based text summaries, enabling faster understanding of data and the creation of LookML models.

Finally, Duet AI plays an important role in cybersecurity by rapidly summarising and classifying threat intelligence, reducing the workload of security professionals, and improving incident detection and response. It is integrated into security products such as Chronicle Security Operations, Mandiant Threat Intelligence, and Security Command Center.

In short, Duet AI is an artificial intelligence platform that broadly impacts application development, infrastructure operation, data analytics, and cybersecurity. Its ability to understand natural language and provide contextual assistance promises to improve efficiency and productivity in a wide variety of areas within the Google Cloud.

Vertex AI

Vertex AI, the cloud-based machine learning platform developed by Google Cloud, offers a complete workflow for building, training, and deploying machine learning models. It supports various types of machine learning tasks, offers data processing and analysis tools, and includes pre-trained models for common use cases. Vertex AI will ease infrastructure management, allowing developers, data scientists, and researchers to focus on their machine learning tasks. With Vertex AI, users can train and deploy models on the Google Cloud infrastructure, which includes AI Platform, Kubernetes, and AutoML.

Vertex Platform

Enhancements to Vertex AI capabilities have been announced in Google Cloud Next, including new models in Model Garden, updates to proprietary models, and tools to customize and enhance models. New models such as Llama 2 and Claude 2 are highlighted, and models such as PaLM 2, Codey, and Imagen are enhanced.

In addition, digital watermarking functionality is introduced for Imagen. Vertex AI Extensions have been introduced, enabling the connection of models to APIs for real-time data and real-world actions. All are designed to facilitate experimentation and application building with base models, enterprise data customization, and application deployment with built-in privacy, security, and responsible AI features. These updates aim to cater to developers and data scientists, regardless of their level of AI expertise, and accelerate the adoption of generative AI in businesses across various industries.

Vertex AI Search and Conversation

Google Cloud’s Vertex AI Search and Conversation are here to power the creation of generative search and chat applications. These products enable developers with little AI experience to build search engines and chatbots to interact with customers and answer questions effectively. In addition to general availability, features like multi-shift search and extensions to take real-time actions have been added.

Vertex AI Search enables high-quality multimodal searches, and Vertex AI Conversation facilitates the creation of natural-sounding chatbots and voicebots. These tools are critical to accelerating the adoption of generative AI and improving the user experience in various enterprise applications.

Vertex Colab

Google launched Vertex AI three years ago to provide the best AI/ML platform for accelerating AI workloads. Since then, Vertex AI has expanded its capabilities, including support for generative AI and developer-friendly products for common generative AI use cases. Over 100 large models from Google, open-source contributors, and third parties have also been added. Despite this expansion, the focus on data science and machine learning engineering remains.

Colab Enterprise has been released in public preview, combining the ease of use of Google’s Colab notebooks with enterprise capabilities for security and compliance. Ray’s support for Vertex AI to efficiently scale AI workloads has also been announced. In addition, MLOps capabilities for generative AI are being advanced with model tuning, model evaluation, and a new version of the Vertex AI Feature Store with support for embeddings.

Colab Enterprise enables data scientists to collaborate and accelerate AI workflows with access to Vertex AI capabilities, BigQuery integration, and code generation. Ray on Vertex AI offers efficiency and scalability and is ideal for training generative AI models. In addition, features such as model evaluation and embedding support have been announced in the Vertex AI Feature Store to improve the management of generative AI models in production.

These new features and products are designed to help organizations advance their AI practice, especially in the context of generative AI, and focus on collaboration, scalability, and efficient model management.

Vertex AI with Colab Enterprise and MLOps for generative AI

As mentioned, there is significant progress in MLOps for generative AI with model fitting and evaluation and a new version of the Vertex AI Feature Store with support for embedding. With these capabilities, customers can leverage the features they need across the entire AI/ML workflow, from prototyping and experimentation to deployment and management of models in production.

For MLOps, key areas such as managing AI infrastructure, customizing with new techniques, managing new artifact types, monitoring generated output, and connecting to enterprise data are highlighted. A new MLOps Framework for Predictive and Generative AI is presented to address these challenges.

Security

At this Google Cloud Next event, another of the big scenarios that concern all types of companies could not be left behind: security. They detailed their approach to security and how they are tackling the most pressing IT challenges. Google Cloud uses artificial intelligence (AI), specifically Duet AI, to strengthen its security solutions and protect against growing cyber threats.

The holistic approach ranges from managing and controlling security in AI workloads to incorporating AI into their security products to make them more effective. In addition, they are providing tools and platforms, such as the Google Cloud Security AI Workbench, that enable their customers to leverage AI to improve security in their own applications and operations.

Duet AI in Mandiant Threat Intelligence

It helps identify tactics, techniques, and procedures (TTPs) used by threat actors against organizations, summarising Google threat intelligence comprehensibly. Provides information on the latest threats and how to make threat intelligence actionable across the organization.

Duet AI in Chronicle Security Operations

Simplifies detection, investigation, and response to cyber threats. Provides clear case summaries, context, guidance on important threats, and recommendations on how to respond. It also enables natural language searches to speed up results.

Duet AI in Security Command Center

It facilitates the analysis of security findings and possible attack paths with near-instant analysis of security conclusions. This simplifies complex issues so that even non-specialists can defend their organizations.

Google Cloud

Google Cloud is also introducing additional capabilities to enhance cybersecurity in Google Cloud environments, such as agentless vulnerability scanning, Cloud Firewall Plus with next-generation firewall capabilities, and Network Service Integration Manager.

Data security

Confidential Computing Private Preview was announced on 4th generation Intel Xeon Scalable CPUs with TDX technology, along with enhanced integrations for protecting sensitive data in services such as Dataplex and Dialogflow.

Modernization of cloud infrastructures

Google finally recognized that we are at an inflection point in computing, where the demands of workloads such as generative AI and large language models (LLMs) are growing exponentially, and traditional infrastructure is no longer sufficient. And it is in this light that the following new developments fell into place:

AI-optimised infrastructure

Google Cloud has led AI for two decades and has created infrastructure solutions optimized for AI. Complete AI solutions, from compute infrastructure to software and services, are offered to train, tune, and serve models globally.

New Infrastructure Enhancements:

• Cloud TPU v5e: Google announced Cloud TPU v5e, which is highly cost-efficient and versatile for large-scale training and inference. It offers up to 2x higher training performance per dollar and up to 2.5x higher inference performance per dollar for LLMs and AI gen models compared to TPU v4. It is less expensive than its predecessor, enabling more organizations to train and deploy larger, more complex AI models. Offers great flexibility with support for eight different virtual machine configurations.
• A3 VMs: These virtual machines based on NVIDIA H100 GPUs will be available soon. They are ideal for training and serving demanding AI workloads. They offer three times the performance and ten times the network bandwidth compared to the previous generation. A3 VMs run at a massive scale, allowing users to scale models to tens of thousands of NVIDIA H100 GPUs.

Operability of TPUs

Google Cloud is making it easier to operate TPUs with the general availability of Cloud TPUs on Google Kubernetes Engine (GKE). Customers can improve AI development productivity by using GKE to manage the orchestration of large-scale AI workloads on Cloud TPU v5e.

Extended support for AI frameworks

Google Cloud supports several AI frameworks, such as JAX, PyTorch, and TensorFlow, in addition to popular open-source tools. They also announced strengthening PyTorch support with the upcoming PyTorch/XLA 2.1 release.

Training Scalability

Multislice” technology was introduced in the preview, allowing AI models to scale beyond the limits of physical TPU pods easily.

In summary, Google Cloud is providing a more cost-efficient and scalable AI-optimised infrastructure that will enable organizations to address the ever-increasing demands of generative AI and LLMs, accelerating progress in the field of AI and deep learning.

Dockers Infrastructure

Some advances in container infrastructure and artificial intelligence (AI) were announced with a focus on GKE Enterprise, TPU in GKE, and Duet AI in GKE and Cloud Run.

GKE Enterprise

Google Cloud introduced GKE Enterprise, an integrated and intuitive container platform combining GKE and Anthos’s best. This edition includes a new feature called “fleets,” which allows platform engineers to group similar workloads into dedicated clusters, apply custom configurations and group-specific policies, isolate sensitive workloads, and delegate cluster management to other teams. GKE Enterprise also offers managed security features such as advanced workload vulnerability intelligence, governance, and policy controls, and a managed service mesh service. In addition, it supports hybrid and multi-cloud environments to run container workloads in GKE, other public clouds, or locally with Google Distributed Cloud.

TPU in GKE

Google introduced Cloud TPU v5e, a more cost-efficient and scalable AI accelerator that can scale to tens of thousands of chips. It offers up to 2x higher training performance and up to 2.5x higher inference performance per dollar compared to Cloud TPU v4, and can take advantage of features such as automatic scaling and workload orchestration. A3 VM support with NVIDIA H100 GPU was also announced.

Duet AI in GKE and Cloud Run

Duet AI, Google’s AI partner, is now available on GKE and Cloud Run. It helps platform teams reduce manual and repetitive work when running containers on Google Cloud.

These announcements show Google Cloud’s commitment to providing cutting-edge container infrastructure and AI capabilities to help businesses increase productivity, efficiency, and scalability. Customers, such as Equifax, have already experienced significant improvements in security and efficiency with GKE Enterprise, allowing them to manage hundreds of clusters efficiently. In addition, advances in TPU and Duet AI promise to accelerate the adoption of AI in various enterprise applications.