Fine-Tuning Azure OpenAI Model for a custom dataset

639 views Asked by At

I want to fine-tune chatgpt davinci model on my own dataset. I have written some code but its giving me the API-Key error. I am using windows and below is code:

import os
import openai
import pandas as pd
import config.config as config
import base64 
import subprocess

engine = config.ENGINE
openai_completion_engine = config.COMPLETION_ENGINE
openai.api_key = config.API_KEY
openai.api_base = config.API_BASE
openai.api_type = config.API_TYPE
openai.api_version = config.API_VERSION

df = pd.read_excel("my_dataset.xlsx")
prepared_data = df.loc[:,['question','answer']]
prepared_data.rename(columns={'question':'prompt', 'answer':'completion'}, inplace=True)
prepared_data.to_csv('prepared_openai_data.csv',index=False)

## prepared_data.csv --> prepared_data_prepared.json
subprocess.run('openai tools fine_tunes.prepare_data --file prepared_openai_data.csv --quiet'.split())

## Start fine-tuning
subprocess.run('openai api fine_tunes.create --training_file prepared_data_prepared.jsonl --model davinci --suffix "SuperHero"'.split())

Below is the error I am getting: enter image description here

1

There are 1 answers

0
Pat On

Looks like you are not providing a valid API key. You may want to add a print statement for debugging.

openai.api_key = config.API_KEY
print('api key: ' + openai.api_key)

For some reason your variable for the api_key is not being set to a valid key.