July 27, 2026

Everyone's selling the same model, but are they?

Inkling, from Thinking Machines, the hottest new entry into the open weights category, is available on five vendors on launch day instead of primarily being a first party API. We picked three for this comparison and took them through the same tests we did for Kimi, Deepseek and GLM last time. In theory, we're buying access to the same weights down to the bits of the same model. As you're about to find out, your mileage might vary depending on your inference provider.

Compare vendors:BasetenFireworks AITogether AI

At Isotopes AI, we don't always fine tune models for our general agents, but we do have a small loop within NeoCortex, which would benefit from an open weights frontier model which was built ground up with fine-tuning in mind. Last time we evaluated the Chinese open-weight models, we found them usable on quality and dramatically cheaper. But open weights isn't a Chinese lab thing.

Inkling (from Thinking Machines) has emerged as a credible challenger: 975B mixture-of-experts, open weights, and available across multiple vendors. We wanted to see if it lives up to the hype, but more importantly, we liked the licensing. Instead of the the non-competes or "do no evil" licensing schemes from the other open weight providers, Inkling came with a proper Apache 2.0 license which is a license near and dear to all of us ex-ASF folks. Throw in a discount to Tinker, it would be an understatement to say, we had big plans.

Before we got too excited about fine tuning, we tried to look around for a good inference provider for our fine tuned models. We wanted to know: if we pick inkling, does it behave the same across vendors: same accuracy, quality, latency and speed. Or does the vendor you choose actually change what you get? To check, each model below is run through the identical coding task on every vendor that serves it (same weights, different infrastructure), and we compare quality, latency, cost and tokens, scored by a fixed neutral judge (Claude Opus 4.8). This was a quick, two-person, exploratory evaluation to guide that decision, not a formal product comparison. For example, we tried out Modal, but couldn't get it to work with our existing harness easily.

Quality: One stands out

Baseten stands out with the highest quality score, and the gap persists despite all three vendors running the same model weights. Baseten and Together publish identical per-token pricing, but Baseten came out cheaper because it needed fewer tokens to complete the same tasks. Turns out quality has a quantity of its own. Fireworks runs on dedicated hardware with per-GPU-second billing, a completely different cost model that resulted in much higher expenses for lower quality. The pattern is clear: serverless + fewer retries wins on both quality and cost.

Caveat: Inkling launched just days before this benchmark was run. These vendors' infrastructure, optimizations, and configurations will evolve rapidly. This snapshot captures early deployments that may look quite different by the time you read this. Fireworks, for instance, may offer serverless soon, which would reshape the cost and latency picture entirely. To run a cheap comparison, reasoning is turned off for this benchmark, so don't think this is a less capable model compared to the last blog post about chinese models.

Judge overall (/5) per vendor. If the weights are the same, quality should be identical.

Inkling (Thinking Machines)

Baseten ★

3.81

Fireworks AI

3.33

-13%

Together AI

3.29

-14%

Scale 0–5. ★ = native baseline; Δ badges are vs native.

Capability Fingerprint

Each model's vendors overlaid across all dimensions at once. Same-model polygons should overlap if the vendors were truly interchangeable — where one caves inward (usually Speed/Turnaround), that's where the vendor diverges. (Speed = TTFT; Throughput = decode tok/s — a vendor can be slow to start yet generate at a normal rate.)

Inkling (Thinking Machines)

Baseten★Fireworks AITogether AI

Each axis is normalized so the best vendor for that model reaches the outer edge (1.0); lower-is-better axes (Speed=TTFT, Cost, Turnaround=e2e) are inverted first. Quality sits near the edge for everyone (same model) — the Speed/Turnaround axes are where vendors pull apart.

Value / Pareto (X / Y)

Plot any two metrics against each other and filter by model. Cost-vs-latency is the default view (where each vendor sits on the price/speed trade-off), but you can swap either axis to quality, throughput, TTFT, or token count. Changing a dropdown re-renders the chart.

ModelX axisY axis

Each point is one vendor · ★ = native · hover a point for every metric. Axis ends are labeled with the data min/max; pick any two metrics to compare.

Cost per task

Combined $/task (writer + critic + judge), computed from raw tokens × per-vendor pricing.

Inkling (Thinking Machines)

Baseten ★

$0.0936

Fireworks AI

$2.0154

+2053%

Together AI

$0.1257

+34%

Per-model scale. Lower = better. An arm whose writer was unpriced shows only judge/critic cost (footnoted in methodology).

Fireworks AI — dedicated-hardware cost. This arm ran on a dedicated deployment (GPU-time billed, not per-token), so it has no $/token rate — its cost throughout this report is derived from GPU time. Total real spend was $54 over the full benchmark run — 553s (9.2 min) of wall time — i.e. $0.0976/s (≈ $352/hr). Each turn is priced as generation time × $/s (this is what the cost bars & heatmap above now use):

turn	e2e (median)	dedicated $
q1	13.0s	$1.27
q2	17.8s	$1.74
q3	17.2s	$1.68
q4	11.8s	$1.15
q5	43.4s	$4.24

Serverless hosts (Baseten/Together) bill per token (~$0.03–0.06/task) — a dedicated deployment only wins at high sustained utilization.

Latency: TTFT vs decode

Latency outliers — reasoning-off runaways. Some turns ran far longer than the vendor's typical turn: with reasoning off, inkling can loop on the hardest question until it hits max_tokens. This drags the mean latency up sharply, so the tables above report per-arm medians (robust to these). The raw outliers:

Together AI q5: 319.2s end-to-end (30,901 output tokens, near the token cap) — ~13× this vendor's median turn.

Total latency splits into two phases: TTFT (time-to-first-token — the wait before generation starts: request queue + prompt prefill) and decode (streaming the output tokens). The split is the whole story here: decode speed was fairly similar across every vendor (~100–200 tok/s — it's the same model generating), so the latency differences below are driven almost entirely by TTFT. On a slow arm the model isn't generating slower — you simply wait far longer for the first token. (TTFT dominates a streaming UX; for long outputs the shared decode time dilutes it in the end-to-end total, which is why the e2e ratios look smaller than the TTFT ratios.)

Full breakdown — TTFT · decode · e2e

vendor	TTFT	× native	worst TTFT	decode	tok/s	e2e	× native	out tok
Inkling (Thinking Machines) — native: Baseten
Baseten ★	680ms	1.0×	782ms	6.9s	227	7.6s	1.0×	4,208
Fireworks AI	1.3s	1.9×	1.7s	16.0s	153	17.2s	2.3×	3,043
Together AI	1.1s	1.6×	1.5s	24.3s	101	25.4s	3.3×	7,706

TTFT / e2e are per-arm MEDIANS (robust to the tail); worst TTFT is the slowest turn (red = ≥2× the median = heavy tail). decode = e2e − TTFT (generation after the first token). × native = ratio vs the ★ first-party arm.

End-to-end (writer, total)

Inkling (Thinking Machines)

Baseten ★

7.6s

Fireworks AI

17.2s

+125%

Together AI

25.4s

+233%

Per-model scale (bars compare vendors within a model). Lower = better.

Time-to-first-token (TTFT)

Inkling (Thinking Machines)

Baseten ★

680ms

Fireworks AI

1.3s

+91%

Together AI

1.1s

+59%

Per-model scale. Lower = better.

Token usage: same tokenizer, diverging context

Tokens are not expected to be identical across every arm, and that isn't a same-weights problem. On turn 1 the input is identical across a model's vendors (same prompt, same tokenizer) — the clean proof it's the same model. It drifts on later turns because this is a multi-turn conversation: each turn's input includes the previous turn's output, and each vendor produced a slightly different-length answer, so accumulated context diverges. Output tokens vary for the same reason (sampling yields different-length answers). Across different models the counts differ more — different tokenizers. So: near-equal ⇒ same model; small drift = conversation state, not different weights.

Input tokens (writer)

Inkling (Thinking Machines)

Baseten ★

16,277

Fireworks AI

19,556

Together AI

27,801

Output tokens (writer)

Inkling (Thinking Machines)

Baseten ★

4,208

Fireworks AI

3,043

Together AI

7,706

Why the token counts jump: retries, not verbosity

BasetenFireworks AITogether AI

How to read the token numbers. Each number is the total of all writer attempts in that turn, not one long answer — so a big number usually means the model had to retry, not that it is wordy.

Together’s 30,901 tokens on q5 is not one huge reply. The writer was called 8 times there — the code kept failing to run and the critic kept rejecting it. Each attempt was a normal ~3–4k tokens; eight of them add up.
Tries on q5: Baseten 3, Fireworks 4, Together 8. Output tokens follow the number of tries almost exactly (correlation ≈ 0.92, see chart).
So the token count here shows how many tries a vendor needed to get working code, not how chatty the model is.
For Fireworks (dedicated hardware) it counts twice: more retries = more GPU time = more cost.

Quantization & the observability gap

Same open weights, three vendors — but served at the same precision? Here is what can and cannot be verified.

What each vendor serves. We kept each vendor's defaults — the only setting we changed was turning reasoning off (all three allow it), and we did not ask for any particular precision. The model itself matches across the three: Fireworks, Baseten and Together all publish inkling as a 975B mixture-of-experts with 41B active, so on paper it is the same checkpoint everywhere. (Together does list a 524K context window on its page, against the 1M quoted elsewhere.) The real difference is in how it is served: Fireworks offered inkling only in dedicated mode — there was no serverless, per-token option.

Quantization itself cannot be measured. The token-level signals that would isolate precision — per-token logprobs, perplexity, top-k margin, greedy divergence — are not emitted by these managed serving stacks. So there is no way to publish a "% quality loss" figure, and a user cannot actually verify the precision they are being served. That opacity is itself the finding: an observability gap.

A hardware clue. We observed that Baseten's inference endpoint (inference.baseten.co/v1) resolves to Google Cloud, so Baseten may be running on different hardware than the other two vendors. Their blog does mention BF16, but only as a memory requirement (~2 TB for BF16 vs ~600 GB for NVFP4) — it does not confirm which precision they actually serve. Still, if their setup is serving the full-precision BF16 weights, that would plausibly explain why Baseten was both faster and higher quality in our runs. We cannot confirm this; it is a plausible suspicion for why Baseten came out ahead.

Vendor Scorecard

A vendor scorecard of the mean per metric — read down a column to rank the vendors on that metric, across a row for a vendor's whole profile.

vendor	Quality ↑	Cost ↓	TTFT ↓	Latency ↓
Baseten ★	3.81	$0.0936	672ms	19.0s
Fireworks AI	3.33	$2.0154	1.3s	20.6s
Together AI	3.29	$0.1257	1.0s	79.8s

Mean per metric, colored INDEPENDENTLY per column (green = leading vendor for that metric → red = trailing). ↑ higher-is-better · ↓ lower-is-better.

Per-metric heatmaps (breakdown)

The raw per-query values behind the means above — one heatmap per metric (vendors × turns + arm mean). Cell color runs green (leading) → red (trailing).

Quality (/5) — higher is better

vendor	q1	q2	q3	q4	q5	mean
Baseten ★	4.93	4.73	3.67	3.80	1.93	3.81
Fireworks AI	2.33	4.80	3.47	4.67	1.40	3.33
Together AI	4.27	3.33	2.80	4.07	2.00	3.29

Cost ($/task) — lower is better

vendor	q1	q2	q3	q4	q5	mean
Baseten ★	$0.0714	$0.0447	$0.1016	$0.0653	$0.1850	$0.0936
Fireworks AI	$1.2727	$1.7387	$1.6768	$1.1507	$4.2378	$2.0154
Together AI	$0.0896	$0.0496	$0.0872	$0.0580	$0.3442	$0.1257

TTFT — lower is better

vendor	q1	q2	q3	q4	q5	mean
Baseten ★	722ms	601ms	577ms	680ms	782ms	672ms
Fireworks AI	1.4s	1.7s	1.1s	1.2s	1.3s	1.3s
Together AI	1.2s	556ms	1.1s	837ms	1.5s	1.0s

Latency — end-to-end, lower is better

vendor	q1	q2	q3	q4	q5	mean
Baseten ★	6.7s	5.6s	18.0s	7.6s	57.0s	19.0s
Fireworks AI	13.0s	17.8s	17.2s	11.8s	43.4s	20.6s
Together AI	29.9s	12.5s	25.4s	12.0s	319.2s	79.8s

What we actually paid for

The three arms run the same open weights, but we did not buy the same product from each vendor. This section pins down the four things that actually differ — deployment mode, hardware, precision, and price — and cites the source for every cell. Where a vendor publishes nothing, the cell says so: “not disclosed” is itself a result, and it is the same observability gap described in §9.

	Baseten ★	Fireworks AI	Together AI
Deployment
Mode	Model API — serverless, per-token¹	Dedicated deployment, per-GPU-second^4,6 Serverless not offered for this model⁴	Serverless, per-token^7,8 Dedicated not offered⁸
Model id	thinkingmachines/inkling¹	accounts/xxxxx/ deployments/zzzzzzz⁶	thinkingmachines/Inkling⁷
Scaling	Managed by vendor³	Autoscale on, min 0 / max 1 replica, scale-to-zero when idle⁶	Managed by vendor
Hardware & precision
GPU	B200¹ count per replica not published	NVIDIA B200 180GB 8 GPUs/replica, shape “Minimal”⁶	Not disclosed
Cloud (inference)	Google Cloud `inference.baseten.co` → 35.244.129.243 (GCP range, likely GCLB anycast). Note the control plane differs: `app.baseten.co` / `api.baseten.co` sit on EKS behind a shared ALB in AWS us-west-2 — the inference hostname is the one that matters here.¹⁰	AWS us-west-2¹⁰	Not resolved
Precision served	NVFP4 (inferred)¹ Baseten's own example response returns `inferact/inkling-nvfp4` as the model string. Not a runtime guarantee for our calls, but it is the only precision signal Baseten publishes.	FP4 (confirmed)⁶ Stated outright in the deployment console. Auto-tune on; long-prompt optimization off.	Not disclosed^7,8
Model & limits (as the vendor states them)
Parameters	975B total / 41B active^1,3	975B total / 41B active⁴	975B total / 41B active⁷ Console panel reads 552.8B — conflicts with Together's own model page; treated as a UI artifact.⁸
Context window	256K¹	1040K⁴	524K^7,8
Max output tokens	Not published	131,072 requested by our harness⁶	131,072⁹
Modalities	text · image · audio in → text out³	text · image · audio in → text out⁴	text · image · audio in → text out⁸
Cost
Billing basis	Per token²	Per GPU-second, whether busy or idle⁵	Per token⁸
Input / 1M	$1.00²	—	$1.00⁸
Cached input / 1M	$0.17²	—	$0.17⁸
Output / 1M	$4.05²	—	$4.05⁸
Hardware rate	n/a (serverless)	$10.00 / B200-hour list⁵ × 8 GPUs = $80/hr while a replica is up	n/a (serverless)
What we actually spent	Metered per token — see §6	$60.18 total account spend, $5.82 credits remaining⁶	Metered per token — see §6

Every cell is sourced below. “Not disclosed” = the vendor publishes no such figure; (inferred) = derived from vendor documentation rather than stated as a served configuration.

This weakens our BF16 explanation. §9 and the Observations note offer “Baseten serves BF16, Fireworks serves FP4” as the leading suspicion for why Baseten scored higher and ran faster. The table above does not support that. Baseten's model library documents its example response with the model string inferact/inkling-nvfp4 and lists the hardware as B200¹ — i.e. the same 4-bit checkpoint family, on the same GPU generation, as the Fireworks arm. The BF16 figure in Baseten's blog appears only as a memory footprint (~2 TB BF16 vs ~600 GB NVFP4)³, never as a claim about what they serve. We have no Baseten console access to check directly, so this is not conclusive either — but on published evidence, precision is no longer our best explanation for the quality gap, and the earlier notes should be read with that correction. What survives is the cloud/infrastructure difference, which is independently evidenced by DNS¹⁰.

Identical list price, different bill. Baseten and Together publish exactly the same rates for inkling — $1.00 in, $0.17 cached, $4.05 out^2,8. So the ~34% cost gap between them in §6 is entirely a token-count effect, not a pricing effect: Together consumed more input and output tokens for the same work (largely via retries, §7). That is a cleaner finding than a price comparison — at matched rates, the cheaper vendor is simply the one that needed fewer tries.

Reconciling the Fireworks hardware rate. Fireworks' list rate is $10.00 per B200-hour⁵, so an 8-GPU replica costs $80/hr while it is up. The $0.0976/s (≈$352/hr) figure used in §6 is not that rate — it is total spend amortised over generation seconds only, which loads all the idle replica time onto the ~553s that actually produced tokens. Both views are defensible and we have left §6 unchanged, but they answer different questions: §6 says what this benchmark cost us, while $80/hr says what the hardware costs at steady utilisation. The gap between the two is the price of a dedicated box sitting idle during development — which is the real argument against dedicated deployments for exploratory work.

Sources

Baseten — Inkling model library — hardware (B200), size (975B), 256k context, model id, and the example response carrying inferact/inkling-nvfp4.
Baseten — pricing — Inkling at $1.00 input / $0.17 cached input / $4.05 output per 1M tokens.
Baseten — “Meet Inkling” blog — 975B/41B MoE, 1M context, modalities, and the BF16 ~2 TB vs NVFP4 ~600 GB memory footprints.
Fireworks AI — Inkling model page — 975B parameters, context length 1040k, Serverless: not supported, fine-tuning not supported.
Fireworks AI — pricing — on-demand B200 180GB at $10.00/hr, billed per GPU-second. Verify on the live pricing page before publishing — this rate is widely reported by third-party pricing trackers but we have not screenshotted it ourselves.
Fireworks AI deployment console, zkjsn36d (“inkling test2”), captured 23 Jul 2026 — GPU type and count, precision FP4, auto-tune, replica policy, max_tokens, and total spend $60.18. Internal screenshot.
Together AI — Inkling model page — 975B/41B, 6-of-256 expert routing plus 2 shared experts, 524K context.
Together AI console model panel, captured 23 Jul 2026 — pricing ($1.00 / $4.05 / $0.17 cached), 524.3K context, serverless-only deployment, modalities, and the anomalous 552.8B parameter reading. Internal screenshot.
Together endpoint spec (third-party listing) — 524,288 context, 131,072 max output tokens.
Our own DNS resolution, 23 Jul 2026 — inference.baseten.co → 35.244.129.243 (Google Cloud range, likely GCLB anycast); app.baseten.co / api.baseten.co → EKS behind a shared ALB, AWS us-west-2; api.fireworks.ai → AWS us-west-2. Internal measurement.
Thinking Machines Lab — Inkling announcement — upstream reference for 975B/41B, 1M context, and the launch partner list.

Methodology & caveats

What ran

The same codebench coding task (write → run → self-critique Python for a SaaS-finance analyst question), sent to every vendor serving a given model. A fixed neutral judge (Claude Opus 4.8 (neutral)), kept out of the competing arms, scores each final output against a hand-verified golden. Vendors use different model-ids for the same weights (e.g. Baseten thinkingmachines/inkling vs Together …/Inkling vs a Fireworks dedicated-deployment id) — collapsed to one model here.

Read the deltas as

Quality — absolute /5 vs native. Same model ⇒ ≈ 0.
Cost / TTFT / Latency — % vs native; green = parity/better, red = worse.
Tokens — near-identical across vendors confirms the same underlying model.

Caveats (read before quoting a number)

Multi-try run; deltas are averaged over repeats.
Cost is frozen at run time — records written before a price was wired keep the old cost. An arm showing an unusually low cost may have had an unpriced writer (e.g. gpt-5.4 native before its price was added) — re-run that arm to refresh.
No fake scores — an arm whose credentials were missing records an infra-error and is shown as skipped in the matrix, never as a zero.
Latency is provider infrastructure, not the model — it reflects the vendor's serving stack (queueing, region, cold start), which is exactly Post-3's question.

All raw data — per-query metrics

Every metric for each turn (q1–q5), for every model × vendor — the complete raw numbers behind this report, to lift straight into a write-up (respects the vendor toggle above). Think tok ≈ 0 means reasoning was off on that turn; a non-zero cell is a turn where the host reasoned. The bottom mean row is the arm average; infra-error turns are flagged.

Inkling (Thinking Machines)

Baseten thinkingmachines/inkling

turn	Quality	Exec OK	Critic OK	In tok	Out tok	Think tok	TTFT	e2e	Writer $
q1	4.93	100%	100%	2,430	1,360	0	722ms	6.7s	$0.0079
q2	4.73	100%	100%	4,310	1,121	0	601ms	5.6s	$0.0089
q3	3.67	100%	100%	13,205	3,351	626	577ms	18.0s	$0.0293
q4	3.80	100%	100%	15,594	1,690	0	680ms	7.6s	$0.0224
q5	1.93	67%	67%	45,848	13,517	0	782ms	57.0s	$0.1006
mean	3.81	93%	93%	16,277	4,208	125	672ms	19.0s	$0.0338

Fireworks AI accounts/user-xxxxxx/deployments/xxxxxx

turn	Quality	Exec OK	Critic OK	In tok	Out tok	TTFT	e2e	Writer $
q1	2.33	33%	100%	6,162	1,664	1.4s	13.0s	$1.2727
q2	4.80	100%	100%	4,311	2,767	1.7s	17.8s	$1.7387
q3	3.47	100%	100%	5,586	2,750	1.1s	17.2s	$1.6768
q4	4.67	100%	100%	21,951	1,594	1.2s	11.8s	$1.1507
q5	1.40	33%	33%	59,769	6,442	1.3s	43.4s	$4.2378
mean	3.33	73%	87%	19,556	3,043	1.3s	20.6s	$2.0154

Together AI thinkingmachines/Inkling

turn	Quality	Exec OK	Critic OK	In tok	Out tok	TTFT	e2e	Writer $
q1	4.27	100%	67%	4,957	2,924	1.2s	29.9s	$0.0168
q2	3.33	100%	100%	4,310	1,395	556ms	12.5s	$0.0100
q3	2.80	100%	100%	9,402	2,177	1.1s	25.4s	$0.0182
q4	4.07	100%	100%	9,269	1,134	837ms	12.0s	$0.0139
q5	2.00	33%	0%	111,065	30,901	1.5s	319.2s	$0.2362
mean	3.29	87%	73%	27,801	7,706	1.0s	79.8s	$0.0590

Quality: One stands out

Inkling (Thinking Machines)

Capability Fingerprint

Inkling (Thinking Machines)

Value / Pareto (X / Y)

Cost per task

Inkling (Thinking Machines)

Latency: TTFT vs decode

Full breakdown — TTFT · decode · e2e

End-to-end (writer, total)

Inkling (Thinking Machines)

Time-to-first-token (TTFT)

Inkling (Thinking Machines)

Token usage: same tokenizer, diverging context

Input tokens (writer)

Inkling (Thinking Machines)

Output tokens (writer)

Inkling (Thinking Machines)

Why the token counts jump: retries, not verbosity

Quantization & the observability gap

Vendor Scorecard

Per-metric heatmaps (breakdown)

Quality (/5) — higher is better

Cost ($/task) — lower is better

TTFT — lower is better

Latency — end-to-end, lower is better

What we actually paid for

Sources

Methodology & caveats

What ran

Read the deltas as

Caveats (read before quoting a number)

All raw data — per-query metrics

Inkling (Thinking Machines)

Baseten thinkingmachines/inkling

Fireworks AI accounts/user-xxxxxx/deployments/xxxxxx

Together AI thinkingmachines/Inkling

aidnn is the most sophisticated AI agent for analytics, delivering verified, source-traced answers from wherever your data lives.