Get started with the Runpod SDK

This quickstart takes you from installing the Runpod SDK to running a function on a cloud GPU. You’ll write a small app, run it in a live development session, and deploy it as a Serverless endpoint.

Requirements

A Runpod account with a verified email address.
Python 3.10 or higher.

Step 1: Install the SDK

Create a project directory and a virtual environment, then install the runpod package:

mkdir hello-runpod && cd hello-runpod
python -m venv .venv
source .venv/bin/activate
pip install runpod

# If using uv:
uv init && uv add runpod

Installing the package also installs the rp CLI.

Step 2: Log in

Authenticate with your Runpod account:

rp login

# If using uv:
uv run rp login

This opens your browser so you can approve access. Your credentials are saved to ~/.runpod/config.toml, and both the SDK and the CLI use them from then on.

To skip the browser, run rp login --api-key API_KEY with a key from the API keys page, or set the RUNPOD_API_KEY environment variable.

Step 3: Write your app

Create a file called main.py with the following code:

import runpod

app = runpod.App("quickstart")

@app.queue(gpu="H100", workers=(0, 3), dependencies=["torch"])
def matrix_multiply(size: int):
    # Import packages inside the function so they load on the worker
    import torch

    a = torch.rand(size, size, device="cuda")
    b = torch.rand(size, size, device="cuda")
    c = a @ b

    return {
        "size": size,
        "mean": c.mean().item(),
        "gpu": torch.cuda.get_device_name(0),
    }

@runpod.local_entrypoint
def main():
    result = matrix_multiply.remote(2000)
    print(f"Matrix size: {result['size']}x{result['size']}")
    print(f"Result mean: {result['mean']:.4f}")
    print(f"GPU used: {result['gpu']}")

This code does three things:

runpod.App("quickstart") creates an app to hold your resources.
@app.queue(...) turns matrix_multiply into an autoscaling queue endpoint on H100 GPUs that scales between 0 and 3 workers and installs torch on each worker.
@runpod.local_entrypoint marks main as the code that runs on your machine. Calling matrix_multiply.remote(2000) sends the work to Runpod and waits for the result.

Import installed packages such as torch inside the remote function body, not at the top of the file. Top-level imports run on your machine, where the package may not be installed.

Step 4: Run a live development session

Start a development session:

rp flash dev main.py

The CLI creates a temporary endpoint, runs main, and streams worker logs back to your terminal. The first run takes longer while Runpod provisions a worker and installs dependencies:

Matrix size: 2000x2000
Result mean: 500.0412
GPU used: NVIDIA H100 80GB HBM3

Edit main.py, for example by changing 2000 to 4000, and run it again. Later runs reuse the warm worker, so they return much faster. When you’re done, press Ctrl-C. The session waits for the running entrypoint to finish and then deletes the temporary endpoints.

Step 5: Deploy your app

When your app works the way you want, deploy it:

rp flash deploy

This creates production endpoints that stay up after you close your terminal. List your deployed apps with:

rp flash app list

Clean up

To delete the endpoints you deployed, run:

rp flash undeploy --app quickstart

Next steps

Learn what’s in the SDK, including tasks, HTTP services, and storage.
Review the CLI commands.
Call an existing endpoint with runpod.Endpoint.

Get started

Runpod SDK

Flash

Serverless

Pods

Storage

Public Endpoints

Instant Clusters

Accounts and billing

Organizations (Beta)

Integrations

Hub

Reference

Requirements

Step 1: Install the SDK

Step 2: Log in

Step 3: Write your app

Step 4: Run a live development session

Step 5: Deploy your app

Clean up

Next steps

​Requirements

​Step 1: Install the SDK

​Step 2: Log in

​Step 3: Write your app

​Step 4: Run a live development session

​Step 5: Deploy your app

​Clean up

​Next steps

Requirements

Step 1: Install the SDK

Step 2: Log in

Step 3: Write your app

Step 4: Run a live development session

Step 5: Deploy your app

Clean up

Next steps