SpeedInfer SpeedInfer Developer Platform
Swagger Spec ↗ Open Console ↗

SpeedInfer Developer API Reference

SpeedInfer provides an OpenAI-compatible developer platform and control plane for high-throughput LLM inference, fine-tuning orchestration (LoRA / QLoRA), object storage, dataset management, and scoped key governance. Open-source models are served via connected GPU workers, while proprietary closed-source models route through external providers.

Authentication: All API requests require a Bearer token in the Authorization header.
Authorization: Bearer sk-speedinfer-xxxxxxxxxxxxxxxxxxxx

Public API Base URL

Use the public HTTPS endpoint for SDK and API requests.

EnvironmentBase URLDescription
Public APIhttps://speedinfer.com/v1SpeedInfer API gateway

Granular Permission Scopes

When creating API keys via POST /v1/keys or Console, you can restrict permissions to minimal privileges:

ScopeGrants Access To
inference (or chat:completions)Chat completions and legacy text completions
models:readModel discovery, architectures, pricing, and weights manifest
files:write / files:readUpload, read, and delete dataset files for fine-tuning
storage:write / storage:readCreate storage buckets and upload object artifacts
fine_tuning:write / fine_tuning:readLaunch, monitor, and cancel model fine-tuning jobs
keys:write / keys:readIssue, list, and revoke sub-API keys
usage:readQuery consumption metrics and credit balance
adminUniversal administrative superuser bypass

Retrieve Current User & Balance

GET /v1/auth/me

Returns the authenticated user's profile, organization role, verified email status, and current USD credit balance.

Model Pricing & Context Limits

GET /v1/workspace/pricing

Returns current token pricing per 1M input/output tokens, context windows, and storage rates across all supported architectures.

Usage & Rate Limits

GET /v1/usage

Retrieves aggregated token metrics, hourly request counts, and remaining rate limit quotas.

List Available Models

GET /v1/models

Lists currently active and warm models available for instant inference, complete with context windows and token pricing.

Retrieve Model Details

GET /v1/models/{model_id}

Retrieves detailed metadata for a specific base model or fine-tuned checkpoint.

Download & Export Model Weights

GET /v1/models/{model_id}/weights

Export model weights manifest with SHA-256 checksums or stream direct safetensors weights artifacts.

Query ParamTypeDescription
downloadbooleanSet true to initiate binary file download stream.
formatstringTarget format (e.g. safetensors, gguf). Default: safetensors.

Create Chat Completion

POST /v1/chat/completions

Creates a model response for the given chat conversation. Fully OpenAI SDK drop-in compatible.

FieldTypeDescription
modelrequiredstringModel ID to query (e.g. Qwen/Qwen2.5-7B-Instruct).
messagesrequiredarrayList of conversation turns (role and content).
temperaturefloatSampling temperature between 0.0 and 2.0. Default 0.7.
max_tokensintegerMax tokens to generate. Max: 32,768.
streambooleanStream partial tokens via Server-Sent Events (SSE).

Chat Completion Streaming (SSE)

POST /v1/chat/completions (stream=true)

Streams chunked token deltas via standard Server-Sent Events (SSE) protocol (text/event-stream).

Legacy Text Completions

POST /v1/completions

OpenAI-compatible raw prompt text generation endpoint.

Upload a File

POST /v1/files

Upload a training dataset or object. Automatically validates JSONL schema when purpose is fine-tune.

Form FieldTypeDescription
filerequiredfile (binary)The multipart file to upload.
purposestringTarget purpose: fine-tune, storage. Default: fine-tune.

List Files

GET /v1/files

Returns a list of all files uploaded by the caller's organization.

Retrieve File Metadata

GET /v1/files/{file_id}

Returns metadata about a specific file, including size, status, and creation date.

Download File Content

GET /v1/files/{file_id}/content

Streams the raw binary or text contents of an uploaded file.

Delete a File

DEL /v1/files/{file_id}

Permanently deletes a file and frees allocated storage space.

Create Storage Bucket

POST /v1/buckets

Creates an isolated storage bucket for holding model weights, datasets, RAG vector indexes, and checkpoint blobs.

List Storage Buckets

GET /v1/buckets

Returns a list of all object storage buckets owned by the account.

Retrieve Bucket Details

GET /v1/buckets/{bucket_id}

Retrieves metadata for a specific storage bucket, including total object count and cumulative byte size.

Delete Storage Bucket

DEL /v1/buckets/{bucket_id}

Deletes a bucket along with all objects stored within it.

Upload Object to Bucket

POST /v1/buckets/{bucket_id}/objects

Uploads an arbitrary binary blob or dataset file into the specified bucket.

List Objects in Bucket

GET /v1/buckets/{bucket_id}/objects

Lists all objects stored inside the designated bucket.

Create Fine-Tuning Job

POST /v1/fine_tuning/jobs

Launch a parameter-efficient fine-tuning (LoRA / QLoRA) run on a specified base model using an uploaded dataset.

FieldTypeDescription
modelrequiredstringBase model name (e.g. Qwen/Qwen2.5-7B-Instruct).
training_filerequiredstringID of the uploaded JSONL dataset file (file-...).
methodstringTraining strategy: lora, qlora, or full. Default: lora.
hyperparametersobjectConfig for n_epochs, batch_size, lora_rank, lora_alpha.

List Fine-Tuning Jobs

GET /v1/fine_tuning/jobs

Lists all training and fine-tuning runs created under the caller's account.

Retrieve Fine-Tuning Job

GET /v1/fine_tuning/jobs/{job_id}

Get current execution status, trained token counts, and resulting model identifier.

Cancel Fine-Tuning Job

POST /v1/fine_tuning/jobs/{job_id}/cancel

Gracefully halts execution of a running fine-tuning job.

List Job Checkpoints

GET /v1/fine_tuning/jobs/{job_id}/checkpoints

Retrieves checkpoint checkpoints saved during training steps with loss metrics.

List Job Events & Logs

GET /v1/fine_tuning/jobs/{job_id}/events

Real-time log stream and milestones emitted during model optimization.

Create Scoped API Key

POST /v1/keys

Programmatically issue API keys with custom permission scopes and rate limits for different microservices or applications.

List API Keys

GET /v1/keys

Lists all API keys associated with the account along with their scopes and usage status.

Retrieve API Key Details

GET /v1/keys/{key_id}

Returns configuration and granted scopes for a designated API key ID.

Revoke API Key

DEL /v1/keys/{key_id}

Immediately invalidates and deactivates an API key.

⚡ Interactive API Playground Ready 

          
Example 200 Response

          

        


      

    

  


  
  

    
    

      
المقالات والدروس التقنية (Engineering Guides)

      

        Production-ready architectures, benchmarks, and step-by-step guides for developers building AI products on SpeedInfer.
      

    


    
    

      Quickstart
      
From Zero to First Inference in 60 Seconds (التشغيل السريع)

      

        

          SpeedInfer provides 100% drop-in API compatibility with the official OpenAI SDKs in Python and Node.js. 
          You only need to change your base_url to https://speedinfer.com/v1.
        

        
Python Example

        
from openai import OpenAI

client = OpenAI(
    base_url="https://speedinfer.com/v1",
    api_key="sk-speedinfer-your-api-key-here"
)

response = client.chat.completions.create(
    model="Qwen/Qwen2.5-7B-Instruct",
    messages=[{"role": "user", "content": "Explain KV-cache optimization in 2 sentences."}]
)
print(response.choices[0].message.content)

      

    


    
    

      Model Training
      
Fine-Tuning Qwen 2.5 with LoRA and QLoRA on SpeedInfer (تدريب النماذج)

      

        

          SpeedInfer allows training and fine-tuning custom adapters without managing raw GPU infrastructure or Docker containers. Upload your JSONL dataset, configure LoRA parameters (e.g. r=16, alpha=32), track metrics in real time, and deploy your fine-tuned model directly to an inference endpoint.
        

        
1. Prepare your dataset (train.jsonl)

        
{"messages": [{"role": "system", "content": "You are a customer support agent."}, {"role": "user", "content": "Where is my order?"}, {"role": "assistant", "content": "Please provide your order ID."}]}
{"messages": [{"role": "system", "content": "You are a customer support agent."}, {"role": "user", "content": "Can I return a damaged item?"}, {"role": "assistant", "content": "Yes, within 30 days."}]}

        
2. Upload and Launch Training

        
# 1. Upload dataset
curl -X POST https://speedinfer.com/v1/files \
  -H "Authorization: Bearer $SPEEDINFER_KEY" \
  -F "[email protected]" \
  -F "purpose=fine-tune"

# 2. Launch Job with LoRA Hyperparameters
curl -X POST https://speedinfer.com/v1/fine_tuning/jobs \
  -H "Authorization: Bearer $SPEEDINFER_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen2.5-7B-Instruct",
    "training_file": "file-xxxxxxxxxxxx",
    "method": "qlora",
    "suffix": "support-v1",
    "hyperparameters": { "n_epochs": 3, "batch_size": 4, "lora_rank": 16, "lora_alpha": 32 }
  }'

      

    


    
    

      Low Latency
      
Ultra-Low Latency Streaming with Server-Sent Events (SSE) (البث الفوري)

      

        

          To deliver instant real-time typing experiences, SpeedInfer gateway disables internal buffering and streams raw token chunks directly 
          via chunked HTTP/2 streaming. TTFT (Time To First Token) is minimized through continuous batching.
        

        
import { OpenAI } from "openai";

const client = new OpenAI({
  baseURL: "https://speedinfer.com/v1",
  apiKey: process.env.SPEEDINFER_API_KEY
});

const stream = await client.chat.completions.create({
  model: "Qwen/Qwen2.5-7B-Instruct",
  messages: [{ role: "user", content: "Tell me a story about high-speed GPUs." }],
  stream: true
});

for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content || "");
}

      

    


    
    

      Security & Keys
      
Zero-Trust Scoped API Keys for Microservices (صلاحيات المفاتيح)

      

        

          Never share master account credentials across environments. SpeedInfer allows creating scoped keys restricted to 
          specific tasks, such as inference-only for production frontend proxies or storage-only for ETL pipelines.
        

        
curl -X POST https://speedinfer.com/v1/keys \
  -H "Authorization: Bearer $SPEEDINFER_MASTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "production-inference-bot",
    "permissions": "chat:completions,models:read",
    "rpm_limit": 300,
    "tpm_limit": 200000
  }'

      

    


    
    

      Storage & Buckets
      
Scalable Object Storage for Checkpoints and RAG Datasets (التخزين السحابي)

      

        

          SpeedInfer Buckets provide high-speed S3-compatible storage integrated with our model serving and fine-tuning pipelines. Eliminate data egress costs and latency when feeding datasets to training workflows and inference endpoints.
        

        
# Create bucket
curl -X POST https://speedinfer.com/v1/buckets \
  -H "Authorization: Bearer $SPEEDINFER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name": "rag-embeddings", "description": "Vector indices for search"}'

# Upload binary artifact
curl -X POST https://speedinfer.com/v1/buckets/bkt-xxxxxxxx/objects \
  -H "Authorization: Bearer $SPEEDINFER_API_KEY" \
  -F "[email protected]"