Runpod Python SDK

The runpod Python package is the single SDK for building on Runpod. It includes the decorator-based app SDK that was previously shipped as Flash, the rp command-line interface, the client for calling existing Serverless endpoints, the Serverless worker SDK, and the API wrapper for managing Pods and other resources.

import runpod

app = runpod.App("hello")

@app.queue(gpu="H100", dependencies=["torch"])
def hello():
    import torch
    return {"gpu": torch.cuda.get_device_name(0)}

@runpod.local_entrypoint
def main():
    print(hello.remote())

Run the file with rp flash dev main.py. The hello function runs on a Runpod GPU, and main runs on your machine.

Flash is now part of the Runpod SDK. If you have existing code that uses the runpod-flash package, it continues to work. See What changed for how the old Flash commands and imports map to the unified SDK.

Get started

Quickstart

Install the SDK and run a function on a GPU.

What changed

Map Flash commands and imports to the unified SDK.

CLI reference

Use the rp CLI to log in, develop, and deploy.

Install

The SDK requires Python 3.10 or higher.

pip install runpod

# If using uv:
uv add runpod

Installing the package also installs the rp CLI. The same CLI is available under the longer name runpod.

Authenticate

rp login

This opens your browser to approve access. Credentials are saved to ~/.runpod/config.toml and are shared by the SDK and the CLI. See rp login for other options, including named profiles.

You can also set the RUNPOD_API_KEY environment variable. This is the only environment variable the SDK reads for authentication.

What’s in the SDK

The runpod package covers four areas. Each one is independent, so you can use any of them on its own.

Area	Use it to	Main entry points
Apps	Define functions and classes in Python that run on Runpod GPUs or CPUs, then deploy them as endpoints.	`runpod.App`, `@app.queue`, `@app.task`, `@app.api`, `rp flash`
Endpoint client	Call a Serverless endpoint that already exists, by ID.	`runpod.Endpoint`
Serverless worker	Write the handler that runs inside a custom Serverless worker container.	`runpod.serverless.start`
API wrapper	Create and manage Pods, endpoints, templates, and other resources.	`runpod.create_pod`, `runpod.get_pods`, and related functions

Apps

An app is a collection of Python functions and classes that run on Runpod. Create one with runpod.App, then attach resources to it with decorators. Each decorator defines a different kind of remote compute.

Resource types

Decorator	What it creates	Use it for
`@app.queue`	An autoscaling, queue-based Serverless endpoint.	Inference, batch jobs, and other request/response work.
`@app.task`	One ephemeral Pod per call. The Pod provisions, runs to completion, and terminates.	Training, fine-tuning, and other long-running jobs.
`@app.api`	A load-balanced Serverless endpoint that serves a Python class over HTTP.	Web services and APIs with multiple routes.

The following example defines one of each:

import runpod
from runpod import App, Model, NetworkVolume, Secret

app = App("inference")

models = NetworkVolume("models", size=100)
llama = Model("meta-llama/Llama-3.1-8B-Instruct")

# Autoscaling job queue on H100s, scales to zero when idle
@app.queue(
    gpu="H100",
    workers=(0, 3),
    dependencies=["vllm"],
    mounts={"/runpod-volume": models},
    model=llama,
    env={"HF_TOKEN": Secret("hf-token")},
)
def chat(prompt: str):
    import vllm

    llm = vllm.LLM(model=str(llama.path))
    return llm.generate(prompt)

# One ephemeral Pod per call
@app.task(gpu="H100", gpu_count=2, mounts={"/models": models})
def finetune(steps: int = 1000):
    ...
    return {"loss": 0.1}

# Load-balanced HTTP service
@app.api(cpu="cpu3c-1-2")
class Counter:
    @runpod.init
    def setup(self):
        self.count = 0

    @runpod.post("/bump")
    async def bump(self, body: dict):
        self.count += body.get("by", 1)
        return {"count": self.count}

    @runpod.get("/value")
    async def value(self):
        return {"count": self.count}

In this example:

chat runs on an autoscaling queue endpoint with between 0 and 3 workers. Model weights are cached on the network volume, and the HF_TOKEN environment variable is read from a Runpod secret.
finetune provisions a two-GPU Pod for each call and terminates it when the function returns.
Counter is served over HTTP. The method marked with @runpod.init runs once when each worker starts, and methods marked with @runpod.get and @runpod.post become routes. Workers keep state in memory between requests.

Call remote functions

Decorated functions keep their Python identity. You call them through methods that control where and how they run:

Call	Behavior
`fn.remote(...)`	Runs the function on Runpod and blocks until the result is ready.
`await fn.remote.aio(...)`	The async form of `.remote()`.
`fn.spawn(...)`	Submits the work and returns a `Job` handle immediately, without waiting for the result.
`fn.stream(...)`	Streams results back as the function produces them.
`fn.local(...)`	Runs the function in your local Python process. Remote storage isn’t mounted.

For HTTP services defined with @app.api, call routes through the class, for example Counter.post("/bump", {"by": 3}) or Counter.get("/value").

Local entrypoints

Mark the function that drives your app with @runpod.local_entrypoint. It runs on your machine and is the entry point for rp flash dev:

@runpod.local_entrypoint
def main():
    print(chat.remote("Why is the sky blue?"))  # Blocks for the result
    job = finetune.spawn(steps=500)  # Returns a Job immediately

App discovery imports your modules synchronously. Keep blocking work inside an entrypoint or a remote function, not at the top level of a module.

Storage

Attach storage with the mounts parameter, which maps a path on the worker to a volume:

NetworkVolume(name_or_id, size=50, datacenter=None, create=True) references a network volume in a single datacenter. The app chooses a datacenter that works for every resource sharing the volume.
GlobalVolume(name_or_id, create=True) references global storage that isn’t tied to a datacenter.

Both types look up a volume by name or ID and create it if it doesn’t exist. Set create=False to require an existing volume. Mount rules depend on the resource type:

@app.task supports one network volume and one global volume, mounted at different paths that don’t overlap.
@app.queue and @app.api support one volume, mounted at /runpod-volume. Global volumes require a GPU endpoint.

Inside a remote function, volume.path returns the path where the volume is mounted. Read .path inside remote functions, not at the top level of a module.

Develop and deploy

Use the rp flash commands to iterate on an app and ship it:

rp flash dev main.py  # Live dev session: edit, re-run, logs stream back
rp flash deploy       # Deploy production endpoints

A dev session creates temporary endpoints and deletes them when you press Ctrl-C. See the CLI reference for all commands.

Endpoint client

To call a Serverless endpoint that already exists, such as one you deployed from the console or with rp flash deploy, use runpod.Endpoint with the endpoint ID. This class works the same way it did before the Flash consolidation.

import runpod

endpoint = runpod.Endpoint("ENDPOINT_ID")

# Wait for the result
result = endpoint.run_sync({"prompt": "Hello"})
print(result)

# Or submit a job and check on it later
run_request = endpoint.run({"prompt": "Hello"})
print(run_request.status())
print(run_request.output())

By default, the client uses the API key from rp login, RUNPOD_API_KEY, or runpod.api_key. To use a different key for one endpoint, pass it to the constructor: runpod.Endpoint("ENDPOINT_ID", api_key="API_KEY"). A key passed to the constructor takes precedence over the global key. See Send API requests for more examples.

Serverless worker

If you build your own worker container, use runpod.serverless.start to register the handler that processes jobs:

import runpod

def handler(job):
    job_input = job["input"]
    return {"output": job_input}

runpod.serverless.start({"handler": handler})

This part of the SDK is unchanged. See Handler functions to learn more.

API wrapper

The SDK includes functions for managing Runpod resources through the REST API:

import runpod

pods = runpod.get_pods()
pod = runpod.create_pod("test", "runpod/stack", "NVIDIA GeForce RTX 4090")
runpod.stop_pod(pod["id"])
runpod.terminate_pod(pod["id"])

Limitations

CPU resources are restricted to the EU-RO-1 datacenter.
Apps can scale workers quickly across several endpoints, so you may reach your account’s worker limit. Contact Runpod support to raise it.

Next steps

Run your first function with the SDK quickstart.
See what changed if you’re coming from Flash.
Learn about the runtime that runs your code on workers.
Browse the examples in the runpod-python repository.

Get started

Runpod SDK

Flash

Serverless

Pods

Storage

Public Endpoints

Instant Clusters

Accounts and billing

Organizations (Beta)

Integrations

Hub

Reference

Get started

Quickstart

What changed

CLI reference

Install

Authenticate

What’s in the SDK

Apps

Resource types

Call remote functions

Local entrypoints

Storage

Develop and deploy

Endpoint client

Serverless worker

API wrapper

Limitations

Next steps

​Get started

Quickstart

What changed

CLI reference

​Install

​Authenticate

​What’s in the SDK

​Apps

​Resource types

​Call remote functions

​Local entrypoints

​Storage

​Develop and deploy

​Endpoint client

​Serverless worker

​API wrapper

​Limitations

​Next steps

Get started

Install

Authenticate

What’s in the SDK

Apps

Resource types

Call remote functions

Local entrypoints

Storage

Develop and deploy

Endpoint client

Serverless worker

API wrapper

Limitations

Next steps