Together AI

What is Together AI?

Together AI is a cloud platform for running and customising open models such as DeepSeek, Qwen, Llama, GLM and Gemma. You call models through a pay-as-you-go API or reserve dedicated GPUs. The same account covers fine-tuning, batch jobs and GPU clusters.

Founders include CEO Vipul Ved Prakash, CTO Ce Zhang and chief scientist Tri Dao. Reviews list it as a San Francisco firm founded in 2022. Its homepage names customers such as Zoom and Cursor.

Key Features

• Serverless inference: Together AI runs open models on demand and bills per token, with no servers to manage. The library covers chat, image, code, vision and video models from families such as DeepSeek, Qwen, Llama and GLM.
• Dedicated endpoints: Dedicated endpoints place a model on single-tenant GPUs billed by the hour, from $5.49 for an H100. This suits steady production traffic that needs predictable latency without sharing hardware.
• GPU clusters: Teams can book H100, H200 or B200 clusters, from instant self-serve setups to thousands of GPUs. H100 clusters start at $3.99 per hour on demand or $1.99 preemptible, which suits pre-training and large experiments.
• Fine-tuning: The platform offers LoRA and full fine-tuning, with supervised and preference-based (DPO) training methods. Supported bases include Qwen, DeepSeek, Gemma and Llama variants, and LoRA jobs start from $0.34 per million tokens.
• Batch inference: The Batch API processes large offline workloads asynchronously at reduced rates on selected models. Together AI says batch jobs can scale to 30 billion tokens per model, which suits extraction, classification or synthetic data runs.
• Code Sandbox: Code Sandbox provides customisable virtual machines for running AI-generated code, billed per vCPU and per GiB of RAM each hour. A separate Code Interpreter API executes model-written code for $0.03 per 60-minute session.
• Evaluations: Evaluations measure model quality inside the same account. Teams can compare a fine-tuned model with its base version before deployment, rather than exporting results to a separate testing tool.
• OpenAI-compatible API: The API accepts OpenAI SDK calls for chat and embeddings once you change the base URL and key. Native Python and TypeScript libraries are also available for features beyond the OpenAI format.
• Managed storage: Together AI offers managed storage for AI workloads with zero egress fees, according to its site. Keeping datasets and checkpoints next to the GPUs avoids extra transfer charges when moving data between jobs.

Pros

• Wide model choice: Together AI hosts a large open model library across chat, image, code, vision and video. The official site lists 50+ models, while reviews cite 100+, all reachable through one API key.
• Low token prices: Small serverless models cost cents per million tokens, such as $0.09 input for Qwen3.8 Flash. Batch rates cut costs further on selected models, which helps high-volume jobs that are not time-sensitive.
• Easy migration: OpenAI SDK code works with few changes, usually just the base URL and API key. Teams already built on OpenAI can trial open models without rewriting their application.
• Full lifecycle: Inference, fine-tuning, evaluations, sandboxes and GPU clusters sit in one account. A project can start with a single serverless call and later move to dedicated GPUs or custom training without changing provider.
• Research-backed speed: The FlashAttention team tunes the serving kernels, and the company claims 2x faster inference. Futurepedia reviewers also rate its speed highly, which matters for chat and agent apps where latency is visible.
• Enterprise compliance: SOC 2 Type 2 controls and HIPAA support are listed. That helps regulated firms in healthcare or finance pass security reviews before moving workloads onto the platform.

Cons

• No free trial: A $5 prepayment is required before any API call, and the official docs list no free credits. One blog mentions $25 in free credits, but that conflicts with the official documentation.
• Pricey dedicated GPUs: Dedicated endpoints cost $5.49 per hour for an H100 and $8.99 for a B200. Light or bursty traffic rarely justifies that spend, so serverless is usually cheaper until volume is steady.
• Open models only: Together AI hosts open models and does not serve closed models such as GPT or Claude. Teams wanting one provider for every model will still need separate accounts.
• Partial OpenAI parity: The OpenAI-compatible API covers chat and embeddings but not the Assistants API. Apps built on Assistants or other OpenAI-specific features will need some rework, and testing, before switching providers.
• Complex pricing: Rates differ by model, hardware, batch status and fine-tuning method, and they change often. Budgeting takes careful tracking, and Futurepedia reviewers also flag pricing transparency and cost-efficiency as weaker points.
• Sales-led support: Detailed help for larger accounts runs through sales and enterprise contracts. Self-serve users rely mainly on documentation, and Futurepedia rates support lower than speed or accuracy.

Who is Using Together AI?

• AI startups: Coding tools and agents.
• Enterprises: Private endpoints with SOC 2 controls.
• ML researchers: GPU clusters for pre-training.
• App developers: Chat or speech features via API.

Pricing

Together AI is prepaid and usage-based, with a $5 minimum credit purchase and no free trial (official prices, October 2026).

One blog cites $25 in free credits, which conflicts with official docs. Rates change often, so recheck before you budget.

What Makes Together AI Unique?

Research-built infrastructure is the real differentiator here. The FlashAttention team tunes its own kernels, so new open models arrive quickly and run fast. You can begin with one serverless call and later move the same workload to dedicated GPUs.

Compatibilities and Integrations

• API and SDKs: Python and TypeScript libraries.
• OpenAI SDK: Drop-in chat and embeddings support.
• Ecosystem partners: NVIDIA and Hugging Face.
• Compliance: SOC 2 Type 2 and HIPAA support.

Together AI Tutorials

Getting a first response takes only a few minutes after billing setup.

1. Sign up and add $5 credit.
2. Create an API key.
3. Install the Python or TypeScript SDK, or point the OpenAI SDK at the Together base URL.
4. Pick a model from the library and send a test chat request.
5. Track spend, then move busy workloads to a dedicated endpoint if needed.

How We Rated It

Summary

Together AI suits developers and companies that want fast, affordable access to open models without running servers. Its main limitation is scope, since it hosts open models only and offers no free trial. Start on serverless with a small model, measure token use, and only then reserve dedicated capacity.