Using Phi-3 in Microsoft Foundry

With the development of Generative AI, we hope to use a unified platform to manage different LLM and SLM, enterprise data integration, fine-tuning/RAG operations, and the evaluation of different enterprise businesses after integrating LLM and SLM, etc., so that generative AI can Smart applications are better implemented. Microsoft Foundry is an enterprise-level generative AI application platform.

With Microsoft Foundry, you can evaluate large language model (LLM) responses and orchestrate prompt application components with prompt flow for better performance. The platform facilitates scalability for transforming proof of concepts into full-fledged production with ease. Continuous monitoring and refinement support long-term success.

We can quickly deploy the Phi-3 model on Microsoft Foundry through simple steps, and then use Microsoft Foundry to complete Phi-3 related Playground/Chat, Fine-tuning, evaluation and other related work.

1. Preparation

If you already have the Azure Developer CLI installed on your machine, using this template is as simple as running this command in a new directory.

Manual Creation

Creating a Microsoft Foundry project and hub is a great way to organize and manage your AI work. Here's a step-by-step guide to get you started:

Creating a Project in Microsoft Foundry

1. Go to Microsoft Foundry: Sign in to the Microsoft Foundry portal.
2. Create a Project:
   • If you're in a project, select "Microsoft Foundry" at the top left of the page to go to the Home page.
   • Select "+ Create project".
   • Enter a name for the project.
   • If you have a hub, it will be selected by default. If you have access to more than one hub, you can select a different one from the dropdown. If you want to create a new hub, select "Create new hub" and supply a name.
   • Select "Create".

Creating a Hub in Microsoft Foundry

1. Go to Microsoft Foundry: Sign in with your Azure account.
2. Create a Hub:
   • Select the Management center from the left menu.
   • Select "All resources", then the down arrow next to "+ New project" and select "+ New hub".
   • In the "Create a new hub" dialog, enter a name for your hub (e.g., contoso-hub) and modify the other fields as desired.
   • Select "Next", review the information, and then select "Create".

For more detailed instructions, you can refer to the official Microsoft documentation.

After successful creation, you can access the studio you created through ai.azure.com

There can be multiple projects on one AI Foundry. Create a project in AI Foundry to prepare.

Create Microsoft Foundry QuickStarts

2. Deploy a Phi model in Microsoft Foundry

Click the Explore option of the project to enter the Model Catalog and select Phi-3

Select Phi-3-mini-4k-instruct

Click 'Deploy' to deploy the Phi-3-mini-4k-instruct model

3. Playground Chat Phi in Microsoft Foundry

Go to the deployment page, select Playground, and chat with Phi-3 of Microsoft Foundry

4. Deploying the Model from Microsoft Foundry

To deploy a model from the Azure Model Catalog, you can follow these steps:

• Sign in to Microsoft Foundry.
• Choose the model you want to deploy from the Microsoft Foundry model catalog.
• On the model's Details page, select Deploy and then select Serverless API with Azure AI Content Safety.
• Select the project in which you want to deploy your models. To use the Serverless API offering, your workspace must belong to the East US 2 or Sweden Central region. You can customize the Deployment name.
• On the deployment wizard, select the Pricing and terms to learn about the pricing and terms of use.
• Select Deploy. Wait until the deployment is ready and you're redirected to the Deployments page.
• Select Open in playground to start interacting with the model.
• You can return to the Deployments page, select the deployment, and note the endpoint's Target URL and the Secret Key, which you can use to call the deployment and generate completions.
• You can always find the endpoint's details, URL, and access keys by navigating to the Build tab and selecting Deployments from the Components section.

5. Using Phi API in Microsoft Foundry

You can access https://{Your project name}.region.inference.ml.azure.com/swagger.json through Postman GET and combine it with Key to learn about the provided interfaces

You can get the request parameters very conveniently, as well as the response parameters.