Start free trial

Meta: Llama 3.1 8B Instruct

meta-llama/llama-3-1-8b-instruct
CompareAPITry this model

Meta's latest class of model (Llama 3.1) launched with a variety of sizes & flavors. This 8B instruct-tuned version is fast and efficient.

Modalities
In / out per 1M$0.05 / $0.08
Context128K
ReleasedJul 23, 2024

Providers

5 providers serve this model upstream.

Providers serving this model
ProviderInput /MOutput /MCache read /MLatency P50Throughput P50Uptime
Groq$0.05$0.08$0.030.11s451 tps100%
Cloudflarefp8$0.15$0.29Variable0.47s18 tps100%
CoreWeavebf16$0.22$0.22$0.220.29s143 tps100%
DeepInfraus · fp8$0.02$0.04Variable0.64s30 tps99.92%
NovitaAIfp8$0.02$0.05Variable0.49s108 tps99.57%

Market-wide measurements, refreshed every 15 minutes. They describe who serves Llama 3.1 8B Instruct in general, not how ProjectCOZY routes your requests.

Pricing

List rates per 1M tokens — what Llama 3.1 8B Instruct costs pay-as-you-go elsewhere, not what you pay here.

Input
$0.05
per 1M tokens
Output
$0.08
per 1M tokens
Cache read
$0.03
per 1M cached tokens
Included in every pass
Llama 3.1 8B Instruct is included in every ProjectCOZY pass at no extra cost — one flat rate, no per-token metering. The rates above are there for comparison.

Performance

Time to first token and output speed across every provider, over the last 3 days.

0.0s1.0s2.0s3.0s4.0s
Oct 29, 12 AMOct 29, 8 AMOct 29, 4 PMOct 30, 12 AMOct 30, 8 AMOct 30, 4 PMOct 31, 12 AMOct 31, 8 AMOct 31, 4 PMNov 1, 12 AMNov 1, 8 AM
CoreWeave Cloudflare NovitaAI Groq DeepInfra (us)
0 tok/s200 tok/s400 tok/s600 tok/s800 tok/s
Oct 29, 12 AMOct 29, 8 AMOct 29, 4 PMOct 30, 12 AMOct 30, 8 AMOct 30, 4 PMOct 31, 12 AMOct 31, 8 AMOct 31, 4 PMNov 1, 12 AMNov 1, 8 AM
CoreWeave Cloudflare NovitaAI Groq DeepInfra (us)

Uptime

100% of requests were served successfully over the last 3 days, across all providers.

Last 3 daysAvailability 99.98%
Oct 29, 6:00 PMNow
Availability over the last 24 hoursAvailability 99.96%
100% 98% 97% 95%
Oct 31, 6:10 PMNow

Apps

Tools that most often run Llama 3.1 8B Instruct, by tokens this week.

Xreacher43.3% of tokens6.7BJob Agent - LinkedIn Job Matcher26.4% of tokens4.1BSwoony15.1% of tokens2.3BvoCleaner7.8% of tokens1.2BLeadFluence7.4% of tokens1.1B

Activity

201.0B tokens processed market-wide in the last month.

05.0B10.0B15.0B20.0B
Oct 2Oct 7Oct 12Oct 17Oct 22Oct 27Nov 1
Tokens

FAQ

Which model id do I use for Llama 3.1 8B Instruct?
meta-llama/llama-3.1-8b-instruct. Send it as the model field from any OpenAI-compatible client, against https://minuykh.app/v1.
How much context does Llama 3.1 8B Instruct hold?
131,072 tokens per request, of which up to 117,964 can be the response.
What can it accept and return?
Input: text. Output: text.
What does Llama 3.1 8B Instruct cost on a pass?
Nothing beyond the pass itself. Pay-as-you-go elsewhere it lists at $0.05 per 1M input tokens and $0.08 per 1M output tokens.
How current is its knowledge?
Training data runs to Dec 2023. Anything later has to come from your prompt or a tool call.
Who built Llama 3.1 8B Instruct?
Meta, in the Llama family.

Explore

Other models your pass can call.

Aion 3.5aion-labs/aion-3.5256KAionAion 3.5 Miniaion-labs/aion-3.5-mini256KAionAion-2.0aion-labs/aion-2.0128KAionAion-3.0aion-labs/aion-3.0128KAionAion-3.0-Miniaion-labs/aion-3.0-mini128KAionAion-RP 1.0 (8B)aion-labs/aion-rp-llama-3.1-8b32KAion