> For the complete documentation index, see [llms.txt](https://docs.crynux.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.crynux.io/application-development/how-to-run-llm-using-crynux-network.md).

# How to Run LLM using Crynux Network

Running LLM tasks with various open-source models can be as simple as calling an OpenAI-compatible API via the Crynux Network. The example below demonstrates how to send an LLM chat completion task to [Crynux AS](https://crynux-as.xyz) using the official OpenAI SDK:

{% hint style="info" %}
Replace `<endpoint_token>` and `your-api-key` with your project endpoint token and API key from [Crynux AS](https://crynux-as.xyz).
{% endhint %}

{% tabs %}
{% tab title="Python" %}

```python
from openai import OpenAI

client = OpenAI(
    base_url="https://api.crynux-as.xyz/api/<endpoint_token>/v1",
    api_key="your-api-key",
    timeout=60,
    max_retries=1,
)

res = client.chat.completions.create(
    model="Qwen/Qwen3.5-9B",
    messages=[
        {
            "role": "user",
            "content": "What is the capital of France?",
        },
    ],
    stream=False,
    extra_body={
        "vram_limit": 24,
    }
)

print(res)
```

{% endtab %}

{% tab title="JavaScript" %}

```javascript
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.crynux-as.xyz/api/<endpoint_token>/v1",
  apiKey: "your-api-key",
  timeout: 60000,
  maxRetries: 1,
});

async function main() {
  try {
    const chatCompletion = await client.chat.completions.create({
      model: "Qwen/Qwen3.5-9B",
      messages: [
        {
          role: "user",
          content: "What is the capital of France?",
        },
      ],
      stream: false,
      vram_limit: 24,
    });
    console.log("Chat completion response:", chatCompletion);

    return chatCompletion;
  } catch (error) {
    console.error("Error:", error);
  }
}

main();
```

{% endtab %}
{% endtabs %}

This code is standard for invoking OpenAI models through their API. The only modification is the `base_url`, which points at your [Crynux AS](https://crynux-as.xyz) project endpoint. Create a project, obtain the Base URL and API key, and manage Credits at [Crynux AS](https://crynux-as.xyz).

The API supports OpenAI-compatible `/completions`, `/chat/completions`, and `/responses` for the documented fields below. Support differs by endpoint:

| Capability                              | `/completions` | `/chat/completions` | `/responses`                   |
| --------------------------------------- | -------------- | ------------------- | ------------------------------ |
| Streaming                               | Yes            | Yes                 | No (`stream=true` is rejected) |
| Function tools (`tools`, `tool_choice`) | No             | Yes                 | Yes                            |
| Structured output                       | No             | `response_format`   | `text.format`                  |
| Background create-and-poll              | No             | No                  | Yes (`background=true`)        |

Crynux supports function tools only. Built-in tools and `parallel_tool_calls` are not supported on Crynux models. For forced tool calls and structured JSON generation, see [Tool Use](https://docs.crynux.io/application-development/how-to-run-llm-using-crynux-network/tool-use.md) and [Structured Output](https://docs.crynux.io/application-development/how-to-run-llm-using-crynux-network/structured-ouput.md).

## Crynux Extended Parameters

In addition to the documented OpenAI-compatible fields (`model`, `messages` / `input`, `temperature`, `top_p`, `max_tokens` / `max_completion_tokens` / `max_output_tokens`, `stop`, `seed`, `tools`, `tool_choice`, `response_format` / `text.format`, and `stream` where supported), Crynux extends `/v1/chat/completions`, `/v1/completions`, and `/v1/responses` with the fields below.

In the Python OpenAI SDK, pass them through `extra_body`. In the JavaScript OpenAI SDK, pass them as top-level request fields.

| Parameter                              | Notes                                                                                                                                                                                                                   |
| -------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `vram_limit`                           | Minimum GPU VRAM in GB. You can also set it in the URL path as `/v1/<vram_limit>/chat/completions`. When omitted, Crynux selects `min_vram` from the loaded-model cache (unknown models default to `24` GB)             |
| `queue_timeout`                        | Queue wait budget in seconds. See Flexible Pricing below                                                                                                                                                                |
| `dtype`                                | Model load precision: `"float16"`, `"bfloat16"`, `"float32"`, or `"auto"`. Used to match a loaded-model row and write into the underlying task args                                                                     |
| `quantization_config`                  | Optional object or `null` for task-time quantization. See [Model Quantization](https://docs.crynux.io/application-development/quantization.md)                                                                          |
| `chat_template_kwargs`                 | Object passed to the Hugging Face chat template on `/chat/completions` and `/responses`, for example `{"enable_thinking": false}`                                                                                       |
| `reasoning_effort`                     | On `/chat/completions`, controls thinking mode. Value `"none"` injects `enable_thinking=false`; any other non-empty value injects `enable_thinking=true`. On `/responses`, use `reasoning.effort` with the same mapping |
| `top_k`, `min_p`, `repetition_penalty` | Extra sampling controls on `/chat/completions` and `/completions`. `top_k` is an integer; `min_p` is a float in `0.0`–`1.0`; `repetition_penalty` is a float                                                            |

### Flexible Pricing with `queue_timeout`

Crynux schedules queued tasks by task fee. That is Flexible Pricing: a higher Cost Level / task fee ranks higher in the queue, and a lower Cost Level / task fee ranks lower.

You can combine a low project Cost Level with a long `queue_timeout` (for example several hours). The task stays in the queue longer and can run when the network is quieter, which is cheaper for large batches that do not need an immediate result.

Synchronous LLM APIs (`/v1/chat/completions`, `/v1/completions`, and `/v1/responses` with `background=false`) hold one HTTP request until the job finishes. That hold is limited (about 10 minutes by default) and is not suitable for long queue waits. For low Cost Level + long `queue_timeout`, do not use those synchronous APIs. Use asynchronous APIs instead:

* [Responses API](https://docs.crynux.io/application-development/how-to-run-llm-using-crynux-network/responses-api.md) with `background=true`, or
* [Raw Task API](https://docs.crynux.io/application-development/execute-tasks/text-to-text-task.md) create-and-poll

### Example with extended parameters

{% tabs %}
{% tab title="Python" %}

```python
from openai import OpenAI

client = OpenAI(
    base_url="https://api.crynux-as.xyz/api/<endpoint_token>/v1",
    api_key="your-api-key",
    timeout=60,
    max_retries=1,
)

res = client.chat.completions.create(
    model="Qwen/Qwen3.8-27B",
    messages=[
        {
            "role": "user",
            "content": "What is the capital of France?",
        },
    ],
    stream=False,
    extra_body={
        "vram_limit": 24,
        "queue_timeout": 3600,
        "dtype": "bfloat16",
        "chat_template_kwargs": {
            "enable_thinking": False,
        },
        "quantization_config": {
            "quant_method": "bitsandbytes",
            "quant_kwargs": {
                "load_in_4bit": True,
                "bnb_4bit_quant_type": "nf4",
                "bnb_4bit_compute_dtype": "bfloat16",
            },
        },
        "top_k": 40,
        "min_p": 0.05,
        "repetition_penalty": 1.05,
    },
)

print(res)
```

{% endtab %}

{% tab title="JavaScript" %}

```javascript
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.crynux-as.xyz/api/<endpoint_token>/v1",
  apiKey: "your-api-key",
  timeout: 60000,
  maxRetries: 1,
});

async function main() {
  const chatCompletion = await client.chat.completions.create({
    model: "Qwen/Qwen3.8-27B",
    messages: [
      {
        role: "user",
        content: "What is the capital of France?",
      },
    ],
    stream: false,
    vram_limit: 24,
    queue_timeout: 3600,
    dtype: "bfloat16",
    chat_template_kwargs: {
      enable_thinking: false,
    },
    quantization_config: {
      quant_method: "bitsandbytes",
      quant_kwargs: {
        load_in_4bit: true,
        bnb_4bit_quant_type: "nf4",
        bnb_4bit_compute_dtype: "bfloat16",
      },
    },
    top_k: 40,
    min_p: 0.05,
    repetition_penalty: 1.05,
  });
  console.log("Chat completion response:", chatCompletion);
}

main();
```

{% endtab %}
{% endtabs %}

## Need More Complex Features? Use the Raw Task API

Use the OpenAI-compatible LLM APIs when you want chat completions, tool calling, structured output, or Responses. Use the [Crynux AS](https://crynux-as.xyz) Raw Task API when you need finer control, for example:

* batch create up to 100 tasks with `POST /inference_tasks/batch` and batch status polling
* a specific Hugging Face `revision`
* full control over `GPTTaskArgs`, including `template_args` and task-level create-and-poll flow

{% content-ref url="/pages/wbQGyrKGcOdpNuWNNgqU" %}
[Text-to-Text Task](https://docs.crynux.io/application-development/execute-tasks/text-to-text-task.md)
{% endcontent-ref %}

If you need still more direct control over the wallet, Relay `task_fee`, Bridge-side repeat tasks, or the task lifecycle without Crynux AS, deploy your own [Crynux Bridge](https://docs.crynux.io/application-development/crynux-bridge.md).

## Advanced Usage

For more advanced use cases like Tool Calling, Structured Output, Responses API, and integrations with LangChain/LangGraph, please refer to the following guides:

{% content-ref url="/pages/YWBZG1xi78hEF57FbIrU" %}
[Tool Use/Function Calling](https://docs.crynux.io/application-development/how-to-run-llm-using-crynux-network/tool-use.md)
{% endcontent-ref %}

{% content-ref url="/pages/X4h8fSteUIuCspbHdMV1" %}
[Structured Output](https://docs.crynux.io/application-development/how-to-run-llm-using-crynux-network/structured-ouput.md)
{% endcontent-ref %}

{% content-ref url="/pages/d3Y2YDAR411um8SH0b5e" %}
[Vision Language Models (VLM)](https://docs.crynux.io/application-development/how-to-run-llm-using-crynux-network/vision-language-models.md)
{% endcontent-ref %}

{% content-ref url="/pages/oxLKYcKhZMbwTJfXlIyl" %}
[Responses API](https://docs.crynux.io/application-development/how-to-run-llm-using-crynux-network/responses-api.md)
{% endcontent-ref %}

{% content-ref url="/pages/tlRR30Y1MoplaOYmaQwF" %}
[Integration with LangChain & LangGraph](https://docs.crynux.io/application-development/how-to-run-llm-using-crynux-network/langchain.md)
{% endcontent-ref %}

{% content-ref url="/pages/aEuPw0R8uRvksFPmFXzp" %}
[Hermes Agent Integration](https://docs.crynux.io/application-development/how-to-run-llm-using-crynux-network/hermes-agent-integration.md)
{% endcontent-ref %}

## Get Started with Crynux AS

To use the Crynux Network in production, create a project on [Crynux AS](https://crynux-as.xyz), obtain your project Base URL and API key, deposit Credits, and call the OpenAI-compatible APIs as shown above.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.crynux.io/application-development/how-to-run-llm-using-crynux-network.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
