Skip to main content

Generic LLM

The Generic LLM component provides a flexible interface to interact with various Large Language Models, including multimodal models capable of processing both text and images. It offers comprehensive configuration options to control the model's behavior and memory management.

Credit Cost​

Varies by model. Each model has specific credit costs for input tokens (shown in blue) and output tokens (shown in orange) per million tokens. For example:

  • MiniMax-01: 2,000 credits input / 11,000 credits output
  • Mistral Codestral 2501: 3,000 credits input / 9,000 credits output

Usage​

The Generic LLM component has a single input handle that accepts multiple variables of type 'text' or 'image', and a single output handle that produces the model's text response. You can reference text variables in both system and user prompts using the {{variable}} syntax.

As a core component of the platform, the Generic LLM offers exceptional flexibility in handling and transforming data within your workflows. You can:

  • Combine and process data from multiple upstream components
  • Generate structured outputs (like JSON) that can be parsed by downstream components
  • Perform various operations like:
    • Text summarization and analysis
    • Content generation and transformation
    • Problem-solving and decision making
    • Data structuring and reorganization
    • Natural language processing tasks

This flexibility makes it a powerful hub for connecting different parts of your workflow, allowing you to process data exactly as needed for your specific use case.

Variable Handling​

The component accepts:

  • Multiple text variables that can be referenced in prompts using {{variable}} syntax
  • Multiple image variables that can be enabled for model "viewing" through the Images property
  • Output is always a single text variable containing the model's response

If a referenced variable contains empty data, it will be replaced with an empty string in the prompts.

Properties​

Title​

  • Type: text
  • Description: An identifier for the component. Used for organizational purposes only and doesn't affect execution.
  • Default: "LLM"

Model​

  • Type: model selector
  • Description: Search and select from available LLM models. Each model displays:
    • Name and creation date
    • Modality (text->text or text+image->text)
    • Credit costs (input/output per million tokens)
    • Context length
    • Brief description

Memory Type​

  • Type: option
  • Options:
    • No Memory: The model starts fresh each time, with no recollection of previous interactions
    • Cycle Memory: The model remembers interactions within a single execution cycle (useful for loops), but forgets between executions
    • Full Memory: The model maintains memory across multiple executions within the same conversation
  • Default: "Full Memory"

System Prompt​

  • Type: text
  • Description: Defines the model's role and objective. This is where you should establish the model's behavior and purpose, as it significantly impacts performance. You can use {{variable}} syntax to include dynamic content.
  • Best Practices:
    • Be specific about the model's role
    • Define clear objectives and constraints
    • Establish tone and style expectations
    • Specify output format requirements

User Prompt​

  • Type: text
  • Description: Contains the context or data for the model to work with. With the role already established in the system prompt, this should focus on the specific task or query. Supports {{variable}} syntax for dynamic content.
  • Best Practices:
    • Provide relevant context
    • Ask clear questions or give specific instructions
    • Reference connected variables as needed

Images​

  • Type: checkbox list
  • Description: Only appears when image variables are connected. Select which images the model should "see" and process.
  • Note: Only available for multimodal models that support image processing

Advanced Parameters​

Optional parameters to fine-tune the model's behavior:

  • Temperature (0.0-2.0): Controls response randomness
  • Top P (0.0-1.0): Limits token choices by cumulative probability
  • Top K (0 or above): Limits the number of tokens considered
  • Frequency Penalty (-2.0-2.0): Reduces repetition based on token frequency
  • Presence Penalty (-2.0-2.0): Reduces repetition of input tokens
  • Max Tokens (1 or above): Limits response length
  • View all parameters

Examples​

{
  "title": "Product Description Generator",
  "system_prompt": "You are a professional product description writer. Create compelling and accurate descriptions.",
  "user_prompt": "Write a description for: {{a}}\nFeatures: {{b}}",
  "memory_type": "no_memory"
}