How to use Foundation Models in Azure Machine Learning?

In recent years we have seen how advances in artificial intelligence and Machine Learning have led to the emergence of large Foundation Models that are pre-trained with a large amount of data.

We analyze what they consist of, their advantages when using them in Azure Machine Learning, and how to use them.  

What are Foundation Models in Azure Machine Learning?

Foundation Models are a starting point for developing specialized models, which can be easily adapted to multiple applications in different industries. In fact, these models have positioned themselves as a unique opportunity for companies to create and use in their Deep Learning workloads.

Using them in Azure Machine Learning provides native Azure ML capabilities that allow these open-source models to be deployed at scale. They can be easily integrated into business applications and include capabilities such as:

• Discover: this allows you to review model descriptions, test sample inference, and search for code samples to evaluate, tune, or implement the model.
• Evaluate: This allows you to check if the model suits your specific workload by providing your own test data. This makes it easy to visualize the selected model.
• Finetune: allows you to organize your training work and find the model that best suits your needs.
• Deploy: you can deploy pre-trained base models or fitted models on online endpoints for real-time inference or to process large datasets.
• Import: you can use the latest models by importing models similar to those in the catalog.

Catalog of models and collections

This is a hub for finding Foundation Models in Azure Machine Learning and is a starting point for exploring these models. You will be able to search and filter models according to the tasks they are capable of. For now, only models work with text, but whispers that can work with audio have also been deployed.

This catalog has, at the moment, two collections of models: Open source models selected by Azure Machine Learning (ready for immediate use and optimized, natively supported, and easily migrated) and Transformers models from the HuggingFace center (thousands of models for real-time inference with online endpoints).

The latter service is the creator of the leading open-source library for creating state-of-the-art ML models. It allows you to deploy machine learning models on a dedicated connection point to Azure’s enterprise-grade infrastructure. It allows you to choose from tens of thousands of ML models for natural language processing, audio, and machine vision to accelerate your workload. It also streamlines inference with easy deployment and helps keep our data private and secure.

How to use the Foundation Models selected by Azure Machine Learning?

As mentioned above, the Foundation Models in Azure Machine Learning provide native functionality for discovering, evaluating, tuning, deploying, and running these open-source models.

To access these models, you’ll need to go to the Azure Machine Learning Studio, a hub for discovering the Foundation Models catalog. You’ll see the most popular models, open-source LLMs, and more tasks coming soon.

You will have the option to filter by task or license and then select a specific model name, where you can read a card describing the details of the model:

• Task: indicates the inference task for which this pre-trained model can be used.
• Finetuning tasks: lists the tasks for which this model can be adjusted.
• License: indicates the license information. 

Thanks to the model card, you can quickly test any model using the sample inference widget, which will give you your own sample input to test the result.  

How to evaluate Foundation Models using your own test data?

You can evaluate a model against your test data set in two ways: via the “Evaluate UI Wizard” or code-based examples.

In the UI Wizard evaluation, each model can be evaluated for a specific inference task:

• Test data: Pass the test data you want to evaluate by uploading a local file or selecting a set of data recorded in your workspace. Once selected, assign the input data columns according to the schema you need for each task.
• Compute: Provide the Azure ML cluster with what you want to use to tune the model (must run on CPU compute and with sufficient compute quota). Select “Finish” in the evaluation wizard. Once the work is complete, you can view the model metrics and decide if you want to tune the model using your own training data.

Advanced evaluation parameters: Besides the basic evaluation, the wizard includes several advanced evaluation parameters, including default values that can be customized through code-based samples.  

How to fit models with your own training data?

To improve the model’s performance in your workload, you can make adjustments using your own training data easily using the Finetune wizard or code-based examples linked from the model card.  

Each pre-trained model in the catalog can be adjusted for a specific set of tasks; just select it from the drop-down menu. Pass the training data by uploading a local file or selecting a dataset from your workspace.

Then pass the data to validate by selecting “Automatic split.” Also, pass any test data you want to use to evaluate the fitted model. An automatic split of the training data will be reserved for testing.

Next, provide the cluster of the process you want to tune, where we recommend using GPU A100/V100 compute SKUs. Finally, select “Finish” in the wizard to submit your fine-tuning job.

You will find several advanced tuning parameters, such as learning rate, epochs, batch size, etc.  

Machine Learning Movdels

At Plain Concepts, we help companies manage their Machine Learning projects by providing expert guidance on AI and MLOps, including assessing current capabilities and applying industry standard practices to maintain a production-ready ML environment.

We are one of the first companies to earn the AI and Machine Learning on Microsoft Azure Advanced Specialization, so we can assist you in implementing solutions for the lifecycle of machine learning and AI-driven applications.

If you are ready to start or advance your project but don’t know how we can help. Contact us, and our experts will study your case to find a way to get the most out of your business.