Trying a sample implementation from: https://python.langchain.com/docs/integrations/llms/azureml_endpoint_example Dolly with LLMChain
Added deployment_name="<DOLLY_ENDPOINT_URL_deployment_name>",
from langchain import PromptTemplate
from langchain.llms.azureml_endpoint import DollyContentFormatter
from langchain.chains import LLMChain
formatter_template = "Write a {word_count} word essay about {topic}."
prompt = PromptTemplate(
input_variables=["word_count", "topic"], template=formatter_template
)
content_formatter = DollyContentFormatter()
llm = AzureMLOnlineEndpoint(
endpoint_api_key="<DOLLY_ENDPOINT_API_KEY>",
endpoint_url="<DOLLY_ENDPOINT_URL>",
deployment_name="<DOLLY_ENDPOINT_URL_deployment_name>",
model_kwargs={"temperature": 0.0, "max_tokens": 300},
content_formatter=content_formatter,
)
chain = LLMChain(llm=llm, prompt=prompt)
print(chain.run({"word_count": 100, "topic": "how to make friends"}))
No matter what I do, I am always getting this error:
---------------------------------------------------------------------------
ValidationError Traceback (most recent call last)
Cell In[109], line 22
13 llm = AzureMLOnlineEndpoint(
14 endpoint_api_key="<API key>",
15 endpoint_url="<enpoint>",
(...)
18 content_formatter=content_formatter,
19 )
21 chain = LLMChain(llm=llm, prompt=prompt)
---> 22 print(chain.run({"word_count": 100, "topic": "how to make friends"}))
...
File ~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\langchain\llms\base.py:606, in LLM._generate(self, prompts, stop, run_manager, **kwargs)
600 for prompt in prompts:
601 text = (
602 self._call(prompt, stop=stop, run_manager=run_manager, **kwargs)
603 if new_arg_supported
604 else self._call(prompt, stop=stop, **kwargs)
605 )
--> 606 generations.append([Generation(text=text)])
607 return LLMResult(generations=generations)
File ~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\langchain\load\serializable.py:74, in Serializable.__init__(self, **kwargs)
73 def __init__(self, **kwargs: Any) -> None:
---> 74 super().__init__(**kwargs)
75 self._lc_kwargs = kwargs
File ~\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\LocalCache\local-packages\Python311\site-packages\pydantic\main.py:341, in pydantic.main.BaseModel.__init__()
**ValidationError: 1 validation error for Generation
text
str type expected (type=type_error.str)**
Also this endpoint works with both:
https://github.com/Azure/azureml-examples/blob/main/sdk/python/foundation-models/system/inference/text-generation/text-generation-online-endpoint-dolly.ipynb (5) Test the endpoint with sample data:
OR
using urllib.request.Request(url, body, headers)
So, the endpoint is actually accessible.
Really appreciate if someone can guide me to fix this. I need the langchain solution for embeddings.