Edit online

Impose Rate Limits for OpenAI™ API Keys

It is possible to impose rate limits (requests per minute and tokens per minute) for Open AI projects where API keys are configured in Oxygen Feedback Enterprise. It is recommended to define the API Keys at a project level. They should be defined in a project other that the organization's default project. This allows you to impose custom requests and tokens per minute limits. You can also impose a custom budget on the associated project and custom alerts when a certain budget limit has been reached.

Set Tokens per Minute and Requests per Minute Limits at Project Level

OpenAI provides built-in mechanisms for limiting usage at the project level for tokens per minute (TPM) and requests per minute (RPM). These limits can be set through the OpenAI organization dashboard:
  1. Access your Organization’s settings:
    • Go to the OpenAI Dashboard.
    • Under the Organization tab, navigate to the API Keys or Projects section, depending on your specific configuration.
  2. Select the project where you want limits set.
  3. Set rate limits:
    • Open the Rate Limits section for the particular project.
    • Specify the maximum tokens per minute (TPM) and requests per minute (RPM) that you want to allow for that project.
  4. Save the settings and the limits will be enforced automatically by OpenAI.

Set a Budget Limit and Alerts

While the rate limits control the API’s throughput, you can also set a budget limit at the organization level:

  1. Go to the Billing section of your OpenAI account.

  2. Under the Usage Limits tab, you can set:
    • Soft Limit - This sends you alerts when your spending reaches a specified percentage (e.g., 80%) of the limit.
    • Hard Limit - This enforces a cap, stopping API requests when your budget is exhausted.

Budget Alarms and Notifications

OpenAI automatically provides email notifications as you approach the soft limit or when you hit the hard limit. You can customize these notifications in the Usage Limits section.