Skip to content

Latest commit

 

History

History
88 lines (59 loc) · 4.6 KB

fireworksai.md

File metadata and controls

88 lines (59 loc) · 4.6 KB
Raw

Fireworks AI

Fireworks AI

Fireworks AI is a platform designed to empower developers and businesses to leverage the power of generative AI. It offers a comprehensive suite of tools and services, including fast and affordable text and image model inference, fine-tuning capabilities, and on-demand private GPU inference. This enables developers to build innovative products and applications with generative AI while benefiting from optimized performance and customizable solutions. Fireworks AI is committed to accelerating product innovation and making generative AI accessible to a wide range of users.

Interface Name

  • fireworksai

Example Usage

const { LLMInterface } = require('llm-interface');

LLMInterface.setApiKey({'fireworksai': process.env.FIREWORKSAI_API_KEY});

async function main() {
  try {
    const response = await LLMInterface.sendMessage('fireworksai', 'Explain the importance of low latency LLMs.');
    console.log(response.results);
  } catch (error) {
    console.error(error);
    throw error;
  }
}

main();

Model Aliases

The following model aliases are provided for this provider.

  • default: accounts/fireworks/models/llama-v3-8b-instruct
  • large: accounts/fireworks/models/llama-v3-70b-instruct
  • small: accounts/fireworks/models/phi-3-mini-128k-instruct
  • agent: accounts/fireworks/models/llama-v3-8b-instruct

Embeddings Model Aliases

  • default: nomic-ai/nomic-embed-text-v1.5
  • large: nomic-ai/nomic-embed-text-v1.5
  • small: nomic-ai/nomic-embed-text-v1.5

Options

The following parameters can be passed through options.

  • context_length_exceeded_behavior: Details not available, please refer to the LLM provider documentation.
  • frequency_penalty: Penalizes new tokens based on their existing frequency in the text so far, reducing the likelihood of repeating the same line. Positive values reduce the frequency of tokens appearing in the generated text.
  • max_tokens: The maximum number of tokens that can be generated in the chat completion. The total length of input tokens and generated tokens is limited by the model's context length.
  • n: Specifies the number of responses to generate for each input message. Note that costs are based on the number of generated tokens across all choices. Keeping n as 1 minimizes costs.
  • name: Details not available, please refer to the LLM provider documentation.
  • presence_penalty: Penalizes new tokens based on whether they appear in the text so far, encouraging the model to talk about new topics. Positive values increase the likelihood of new tokens appearing in the generated text.
  • prompt_truncate_len: Details not available, please refer to the LLM provider documentation.
  • response_format: Defines the format of the AI's response. Setting this to { "type": "json_object" } enables JSON mode, ensuring the message generated by the model is valid JSON.
  • role: Details not available, please refer to the LLM provider documentation.
  • stop: Up to 4 sequences where the API will stop generating further tokens.
  • stream: If set, partial message deltas will be sent, similar to ChatGPT. Tokens will be sent as data-only server-sent events as they become available, with the stream terminated by a data: [DONE] message.
  • temperature: Controls the randomness of the AI's responses. A higher temperature results in more random outputs, while a lower temperature makes the output more focused and deterministic. Generally, it is recommended to alter this or top_p, but not both.
  • tools: A list of external tools available for the AI to use in generating responses.
  • top_k: The number of highest probability vocabulary tokens to keep for top-k sampling.
  • top_p: Controls the cumulative probability of token selections for nucleus sampling. It limits the tokens to the smallest set whose cumulative probability exceeds the threshold. It is recommended to alter this or temperature, but not both.
  • user: Identifier for the user making the request.

Features

  • Native JSON Mode
  • Streaming
  • Tools
  • Embeddings

Getting an API Key

Free Tier Available: The Fireworks AI API offers a free developer tier and commercial accounts. No credit card is required for the free tier.

To get an API key, first create a Fireworks AI account, then visit the link below.

Fireworks AI Documentation

Fireworks AI documentation is available here.