How to set up an ASP.NET Core API to call an Azure OpenAI instance that is behind Azure API Management and requires both a subscription key & an OAuth2 access token using Semantic Kernel & C#

In this architecture, we are going to put the Azure API Management service in between our application & our Azure OpenAI instance(s).

In order to set up this system, we will need to configure the following systems

• APIM OpenAI endpoint (proxy through API Management)
• APIM Subscription key (to keep track of API calls through APIM, not for authorization)
• OAuth2 On-Behalf-Of flow (to exchange the user’s signed in token for a token to OpenAI)
  • Entra ID Service Principal for the OpenAI service
  • Entra ID Service Principal for the client application
• Semantic Kernel configured to use these components

APIM OpenAI endpoint

We want all requests to the OpenAI service to go through our API Management gateway. That means callers of the API will not know the endpoint of the actual OpenAI service. Instead, they will be given the FQDN of the API Management service. The rest of the OpenAI REST API call will be the same, just the top-level domain will change.

API Management Subscription Key

Azure API Management can require the use of an API key in order to call an API endpoint. This enables APIM to do things like rate limit requests. These API keys come from a unique subscription in APIM. Each instance of the consuming application (and/or instance of the application, like DEV, TST, PRD) will need to provide an API key on each call to the APIM service. This key is validated before the request continues on to the other APIM policies in effect for that API.

Typically, this API key is presented as a request header Ocp-Apim-Subscription-Key.

Follow the docs here to provision a subscription key.

OAuth2 Access Token

The Azure OpenAI service allows either API key or OAuth2 authorization. We don’t want to hand out the API key to each consuming application since that risks exposing the key. It is too easy for someone to accidentally (or maliciously) copy-paste that key on the Internet, check it into source control, share it with others, etc.

OAuth2 is a much more secure option, but we still don’t want calling applications to directly call the OpenAI service. Part of the value of going through the API Management gateway is that we can rate limit requests, log requests, returned cached responses, etc.

Therefore, we should create a new service principal to represent our OpenAI service. This service principal (also called a resource server in OAuth2) is the one that client applications will request an access token to. Our APIM policies will validate this token and then allow requests to go to the backend.

<validate-jwt header-name="Authorization" failed-validation-httpcode="401" failed-validation-error-message="Unauthorized. Access token is missing or invalid.">
    <openid-config url="https://login.microsoftonline.com/{{tenant-id}}/.well-known/openid-configuration" />
    <audiences>
        <audience>{{openai-service-principal-audience}}</audience>
    </audiences>
    <issuers>
        <issuer>https://sts.windows.net/{{tenant-id}}/</issuer>
    </issuers>
</validate-jwt>

In our APIM backend, it will use a Managed Identity to actually authorize a request to the OpenAI service. Only the Managed Identity has access to the actual OpenAI service (via the Cognitive Services OpenAI Contributor RBAC role).

1. Our GenerativeAI powered app will sign users in with their enterprise Single-Sign-On (SSO) (Entra ID)
2. When calling the LLM, we will take the user’s original access token and exchange it for a token to the OpenAI service principal (using the On-Behalf-Of OAuth2 flow)
3. APIM will validate that this token has an audience of the OpenAI service principal & has a valid APIM subscription key
4. APIM will then use its managed identity to request a new access token to the actual OpenAI service
5. APIM will then call the OpenAI service

These measures ensure there is no way anyone can bypass the APIM gateway since they would not have the API key and they would not have a way to request a valid OAuth2 token.

Entra ID OpenAI Service Principal setup

For setting up the Entra ID app registration & service principal that represents the OpenAI service, we need to make a specific scope to expose to the client application. By convention, since we are impersonating the user, we can call this “user_impersonation”. Add this scope to the Expose an API blade on the app registration.

Copy the following information for use in the following steps.

• Client ID (c512fc8d-181e-4a16-8807-b75be129095e in my case)
• Tenant ID
• “user_impersonation” scope

Entra ID Client App Service Principal setup

For setting up the Entra ID app registration & service principal that will be used by the client, we need to request the specific “user_impersonation” scope specified in the previous step. Requesting this permission will also require admin consent. We can also add the client app client_id to the previous OpenAI app registration so users will not see the consent screen.

You should also create a “user_impersonation” scope for use by the client application (similar to the one you created for the OpenAI service).

Copy the following information for use in the client application.

• Client ID (c2fc3bb0-f18b-4d37-a4bb-0f5e9ee4692b in my case)
• Client Secret (provision one if not already created)
• Tenant ID
• Redirect URI (set this if not already set)
• “user_impersonation” scope

Original OAuth2 access token provided by the client application

Note that the audience of this JWT is the client app application scope and the appid is the client app client_id.

This JWT will be provided as the user_assertion when setting up the OnBehalfOnCredential object.

Exchanged OAuth2 access token (obtained by the On-Behalf-Of flow)

Note that the audience of this JWT is the OpenAI app application scope whereas the appid has remained as the client app client_id.

Raw REST API call to Azure OpenAI through APIM

Putting everything together, here is what a properly formatted request to the Azure OpenAI Service through APIM would look like. The important part is the Ocp-Apim-Subscription-Key & Authorization headers. I’m using the REST Client in VS Code to easily test this API call.

# @name openai_request
POST https://{{apiManagementName}}.azure-api.net/openai/deployments/{{chatDeploymentName}}/chat/completions?api-version=2024-06-01
Content-Type: application/json
Ocp-Apim-Subscription-Key: {{subscriptionKey}}
Authorization: Bearer {{token}}

{
    "model":"gpt-4o",
    "messages":[
        {
            "role":"user",
            "content":"Hello"
        }
    ]
}

The REST Client lets you specify variables (either local variables or secrets stored in the settings.json file).

C# ASP.NET Core changes using Semantic Kernel

Specifying these values in the low-level REST API calls to OpenAI is easy to do. However, most applications don’t directly call the REST APIs, but go through an SDK.

Semantic Kernel is Microsoft’s 1st party SDK for easily creating AI enabled applications. It abstracts a lot of the low-level calls so you can focus on building your application.

By default, Semantic Kernel can use the actual OpenAI API key or get an OAuth2 access token. However, we need to heavily control this process because we need to specify the APIM API key via a header called Ocp-Apim-Subscription-Key (not the api-key header that is the OpenAI default). We also need to override the default OAuth2 access token request mechanism to specify a custom API scope (since we created our own identity to represent the OpenAI service).

Luckily, Semantic Kernel supports ASP.NET’s dependency injection and inversion of control system & gives us the ability to control the creation of the client.

Here is an example in Python of the same code below that is in C#.

First, we need to create a OnBehalfOfCredential object that will exchange the original token the user got when they signed in for a token to the backend OpenAI service that is proxied through APIM. The most important field is the userAssertion which is the user’s original access token. In my case, I am going to get this access token from the client application as a header from Easy Auth. We also need to specify the authority in case this is being run in an Azure sovereign cloud (like Azure Government)

 var credential = new OnBehalfOfCredential(
     tenantId: "tenant_id",
     clientId: "client_app_client_id,
     clientSecret: "client_app_client_secret,
     userAssertion: httpContextAccessor.HttpContext?.Request?.Headers["X-MS-TOKEN-AAD-ACCESS-TOKEN"],
     new OnBehalfOfCredentialOptions {
         AuthorityHost = new Uri("https://login.microsoftonline.com/")
     }
);

Next, we need to configure a HttpClient to specify the right OAuth2 scope that represents our OpenAI resource server identity. We also need to provide the Ocp-Apim-Subscription-Key header with all requests. We can do this by configuring a HttpClient and inject it into Semantic Kernel

 httpClient.DefaultRequestHeaders.Add("Ocp-Apim-Subscription-Key", "12345-6789-0123-456789");
 
 openAIClient = new AzureOpenAIClient(
   new Uri(standardServiceEndpoint), 
   credential, 
   new AzureOpenAIClientOptions {
     Audience = "api://<openai-service-principal-application-id>/user_impersonation",
     Transport = new HttpClientPipelineTransport(httpClient)
   }
);

Finally, we can set up the kernel. Note that I have to manually add the IChatCompletionService and the ITextGenerationService instead of using the helper function AddAzureOpenAIChatCompletion. This is because the helper function registers the services as singletons, but I need them to be scoped so they are specific to the signed in user.

Func<IServiceProvider, object?, AzureOpenAIChatCompletionService> factory = (serviceProvider, _) => new(standardChatGptDeployment, standardChatGptClient, null, serviceProvider.GetService<ILoggerFactory>());

var kernelBuilder = Kernel.CreateBuilder();
kernelBuilder.Services.AddKeyedScoped<IChatCompletionService>(null, factory);
kernelBuilder.Services.AddKeyedScoped<ITextGenerationService>(null, factory);

kernel = kernelBuilder.Build();

Now, all requests to OpenAI will go through our APIM endpoint, have the correct APIM API key added as a header and the appropriate OAuth2 access token will be requested and provided.