Provider Selection Guide¶

This guide helps you choose the best LLM provider based on your requirements.

Decision Flowchart¶

flowchart TD
    A[Start] --> B{Privacy Required?}
    B -->|Yes| C[Local Providers]
    B -->|No| D{Budget?}

    C --> C1{Hardware Available?}
    C1 -->|GPU 8GB+| C2[Ollama / vLLM]
    C1 -->|CPU Only| C3[Ollama with small models]

    D -->|Free| E[Groq / Gemini Free]
    D -->|Low $10-50/mo| F[Together / DeepSeek]
    D -->|Production| G{Priority?}

    G -->|Quality| H[OpenAI GPT-4o / Claude]
    G -->|Speed| I[Groq / Cerebras]
    G -->|Cost| J[GPT-4o-mini / Together]
    G -->|Compliance| K[Azure / Bedrock]

Quick Recommendations¶

By Primary Requirement¶

By Industry¶

LLMConfig(provider="openai", model="gpt-4o-mini")
# or
LLMConfig(provider="groq", model="llama-3.3-70b-versatile")

LLMConfig(
    provider="azure",
    deployment_name="gpt-4o",
    azure_endpoint="https://your-resource.openai.azure.com"
)

LLMConfig(provider="ollama", model="llama3.1:70b")

LLMConfig(provider="groq", model="llama-3.3-70b-versatile")

LLMConfig(provider="anthropic", model="claude-3.5-sonnet")

Cost Optimization¶

Free Options¶

1. Groq Free Tier

   • 30 requests/min
   • 100K tokens/day
   • Great for development

2. Google Gemini Free

   • 15 requests/min
   • Good for low volume

3. Ollama (Local)

   • Unlimited (hardware cost only)
   • Best for privacy

Budget Tiers¶

Cost Comparison Table¶

*Estimated for typical sentiment analysis workload

Quality Comparison¶

Sentiment Analysis Accuracy¶

Based on internal benchmarks:

Summarization Quality¶

Latency Comparison¶

Average response time for typical requests:

Feature Requirements¶

Vision Support¶

For analyzing images (product photos, screenshots):

• OpenAI: GPT-4o, GPT-4-turbo
• Anthropic: Claude 3.5 Sonnet, Claude 3
• Google: Gemini 1.5 Pro, Gemini 2.0
• Together: LLaVA models
• Ollama: LLaVA, BakLLaVA

Long Context¶

For analyzing many reviews at once:

Structured Output¶

For reliable JSON responses:

• Best: OpenAI (JSON mode), Anthropic
• Good: Groq, Together, Fireworks
• Variable: Local models (depends on prompting)

Fallback Strategies¶

Recommended Fallback Chain¶

from sentimatrix.config import SentimatrixConfig, LLMConfig

config = SentimatrixConfig(
    llm=LLMConfig(
        provider="groq",
        model="llama-3.3-70b-versatile",
        fallback=[
            # Fallback 1: Another fast provider
            {"provider": "together", "model": "meta-llama/Llama-3-70b-chat-hf"},
            # Fallback 2: Premium provider
            {"provider": "openai", "model": "gpt-4o-mini"},
            # Fallback 3: Local (always available)
            {"provider": "ollama", "model": "llama3.2"},
        ]
    )
)

By Reliability Requirement¶

LLMConfig(
    provider="openai",
    model="gpt-4o-mini",
    fallback=[
        {"provider": "anthropic", "model": "claude-3-haiku"},
        {"provider": "groq", "model": "llama-3.3-70b"},
    ],
    max_retries=5,
    retry_delay=1.0,
)

LLMConfig(
    provider="groq",
    model="llama-3.3-70b",
    fallback=[
        {"provider": "deepseek", "model": "deepseek-v3"},
        {"provider": "ollama", "model": "llama3.2"},
    ]
)

LLMConfig(
    provider="anthropic",
    model="claude-3.5-sonnet",
    fallback=[
        {"provider": "openai", "model": "gpt-4o"},
        {"provider": "openai", "model": "gpt-4o-mini"},
    ]
)

Summary Recommendations¶

For Most Users¶

Start with Groq - Free tier, fast, good quality:

LLMConfig(provider="groq", model="llama-3.3-70b-versatile")

For Production¶

Use GPT-4o-mini - Best balance of quality, cost, reliability:

LLMConfig(provider="openai", model="gpt-4o-mini")

For Best Quality¶

Use Claude 3.5 Sonnet - Best reasoning and safety:

LLMConfig(provider="anthropic", model="claude-3.5-sonnet")

For Privacy¶

Use Ollama - Fully local, no data leaves your machine:

LLMConfig(provider="ollama", model="llama3.1:70b")