Send out extra headers when using AzureChatOpenAI in Langchain python

Question

I have put my Open AI service behind Azure API Management gateway, so if the client has to access the Open AI service they have to use the gateway URL. Lets say the gateway URL is [email protected]. Before the gateway forwards the request to the OpenAI service it needs to authenticate the client by looking at the Authorization header. So the request from the client should contain an Authorization Header.I was able to achieve this using the openai official python library. Code is shown below:

import openai
import logging
import os

os.environ["AZURE_OPENAI_API_TYPE"] = "azure"
os.environ["AZURE_OPENAI_API_KEY"] = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
os.environ["API_BASE"] = f"[email protected]"
os.environ["AZURE_OPENAI_API_VERSION"] = "2022-12-01"
os.environ["AZURE_OPENAI_EMBEDDING_DEPLOYMENT"] = "model-deployment-name"

def embed_documents(text):
        logging.info("embed_documents [" + text + "]")
        openai.api_type    = os.getenv("AZURE_OPENAI_API_TYPE")
        token              = os.getenv("AZURE_OPENAI_API_KEY")    
        openai.api_key     = token
        openai.api_base    = os.getenv("API_BASE")
        openai.api_version = os.getenv("AZURE_OPENAI_API_VERSION")
        
        AZURE_OPENAI_EMBEDDING_DEPLOYMENT = os.getenv("AZURE_OPENAI_EMBEDDING_DEPLOYMENT")
        response = openai.Embedding.create(input=text, 
                                           engine=AZURE_OPENAI_EMBEDDING_DEPLOYMENT,
                                            headers={
                                                'Authorization': f'{token}'
                                                }
                                           )
        logging.info("embed_documents complete")
        embeddings = response['data'][0]['embedding']
        print(embeddings)
        return embeddings

        if __name__ == "__main__":
          embed_documents("MY first text")

In the above code snippet I could generate the embedding using the gateway URL because I was able to add Authorization Header in the request. I wish to do the same with the following code.

import os
from langchain.chat_models import AzureChatOpenAI
from langchain.schema import HumanMessage

model = AzureChatOpenAI(
    openai_api_base="https://openai003.openai.azure.com/",
    openai_api_version="2023-05-15",
    deployment_name="semantic-query-expansion",
    openai_api_key="XXXXXXXXXXXXXXXXXXXXXXXXXX",
    openai_api_type="azure",
)


model(
    [
        HumanMessage(
            content="Translate this sentence from English to Bengali. I love programming."
        )
    ]
)

Please note the above code is taken from the link https://python.langchain.com/docs/integrations/chat/azure_chat_openai and works fine when I hit azure openAI URL directly, but I haven't found any way to add Authorization Header to it, because of which I have not been able to use the gateway URL. Does anyone have some experience in doing something similar?

Answer 1

you can use LangChain connected to Azure APIM. The trick is to use a different header to pass the api-key

llm = AzureOpenAI(    
    headers={"Ocp-Apim-Subscription-Key": os.environ["OPENAI_API_KEY"]},    
    openai_api_base=APIM_BASE_URL,    
    model_name=COMPLETION_MODEL,    
    deployment_name=COMPLETION_DEPLOYMENT,    
    max_tokens=SUMMARY_MAX_TOKENS,    
    temperature=SUMMARY_TEMPERATURE    
)

Answer 2

The best way to transparently place Azure API Management in front of Azure OpenAI is to change the subscription key from the default Ocp-Apim-Subscription-Key to api-key to match the AOAI syntax.

You can configure the header name used by APIM under the API settings > Subscription > Header name.

Answer 3

You can use default_headers with new version of langchain.

llm = AzureChatOpenAI(
    default_headers={"Ocp-Apim-Subscription-Key": <your-subscription-key-here>},
    deployment_name="gpt-35-turbo",
    model_name="gpt-35-turbo",
    temperature=0.1
)