Integrate with ChatGPT

For Dummies

OpenAI currently (July/2023) offers these three products:

OpenAI apps

We’re going to look into how to integrate OpenAI models into our applications.

First things first understand how to work with the API…

OpenAI’s API exposes various models that can be used to solve virtually any task that involves processing language.

GPT-3 models can understand and generate natural language. The more powerful GPT-3.5 generation models superseded these models, however, the original GPT-3 base models (davinci, curie, ada, and babbage) are currently the only models that are available to fine-tune.

Available models

You will interact with the model by sending it a piece of text (also known as a prompt). When the model receives the prompt it will break it into tokens (from a high-level perspective tokenization is the process of breaking a sentence into words).

Prices are per 1,000 tokens…

Pricing model

The latter is important because you’re going to be charged per number of tokens. Pricing is pay-as-you-go per 1,000 tokens, with $5 in free credit that can be used during your first 3 months.

⚠️ The total number of tokens processed in a single request is comprised of both prompt and completion.

To get a sense of measure one token generally corresponds to ~4 characters of text for common English text.

Tokens

OpenAI offers multiple language models, each with different capabilities and therefore different pricing.

Models pricing

At the end of the day, depending on your needs (content generation, summarization, translation, sentiment analysis) you’re going to interact with one or many of the endpoints exposed by OpenAI.

Endpoints

In this example, we’re going to use /completions endpoint, which will allow us to POST a prompt, and receive one or more completions predicted by the model, and also the probabilities of alternative tokens at each position.

completions model compatibility

Getting started

Head to your OpenAI account API keys and click create a new secret key, give the key a name, and copy and save the value. This is the API key that you’re going to use to authenticate (all API requests should include the API key in an Authorization HTTP header).

Install the Python package:

pip install openai

Bellow I’ve declared my API key as OPEN_API_KEY environment variable:

!/usr/bin/env python3

import openai
import os

openai.api_key = os.getenv("OPEN_API_KEY")

# get the models names from the list
available_models = openai.Model.list()
model_names = [model['id'] for model in available_models['data']]
print(model_names)

You should have an output with the available models:

available models

Now, when making a request two things are mandatory the ID of the model to use and the prompt to generate completions for.

Below is an example that usestext-davinci-003 model and reads the prompt from STDIN.

#!/usr/bin/env python3

import openai
import os

openai.api_key = os.getenv("OPEN_API_KEY")

# get the models names from the list
available_models = openai.Model.list()
model_names = [model['id'] for model in available_models['data']]
print(f"{model_names} \n Prompt: ")

iprompt = input()

response = openai.Completion.create(
            model="text-davinci-003",
            prompt=iprompt,
            temperature=0.5,
        )
print(response)

Next, I’ll test the model with a simple prompt Suggest a funny name for a cat and we can see that the returned completion is Sneaky McMeowface but a more important thing is the finish_reason which in this case is stopand it shows that the API returned the full completion generated by the model.

If I would give it another prompt e.g. What is a LLM we can see that now the finish_reason is length .

finish_reasons

You can “fix” this by using max_tokens (which default is 16) and now if it’s set to 100, we can clearly see a difference in the completion for the same What is a LLM prompt.

response = openai.Completion.create(
            model="text-davinci-003",
            prompt=iprompt,
            temperature=0.5,
            max_tokens=100,
        )

max_tokens 100

Use carefully the settings for max_tokens and stop, because these parameters directly influence the completions, and it can quickly consume your token quota.

Now there are many parameters for completion, some of them:

• temperature = can take values between 0 and 2, by default is 1, and if as a rule of thumb set the temperature high for more creative results or set the temperature low for more predictable results
• echo = boolean default to False, if set to True returns back the prompt

But after tinkering with them, you would like to return:

response.choices[0].text

Closing Notes

Using OpenAI’s ChatGPT API is rather straightforward and simple, the completionsendpoint is flexible enough to solve most of any language processing task (content generation, summarization, semantic search, topic tagging, sentiment analysis, etc.)

Currently a limitation for most models, a single API request can only process up to 4,096 tokens between your prompt and completion. Also don’t forget to read API data usage policies 😉