# How to Run LLM using Crynux Network

Running LLM tasks with various open-source models can be as simple as calling an OpenAI-compliant API via the Crynux Network. The example below demonstrates how to send an LLM chat completion task to the Crynux Network using the official OpenAI SDK:

{% tabs %}
{% tab title="Python" %}

```python
from openai import OpenAI

client = OpenAI(
    base_url="https://bridge.crynux.io/v1/llm",
    api_key="q3hXHA_8O0LuGJ1_tou4_KamMlQqAo-aYwyAIDttdmI=", # For public demonstration only, strict rate limit applied.
    timeout=60,
    max_retries=1,
)

res = client.chat.completions.create(
    model="Qwen/Qwen2.5-7B-Instruct",
    messages=[
        {
            "role": "user",
            "content": "What is the capital of France?",
        },
    ],
    stream=False,
    extra_body={
        "vram_limit": 24,
    }
)

print(res)
```

{% endtab %}

{% tab title="JavaScript" %}

```javascript
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://bridge.crynux.io/v1/llm",
  apiKey: "q3hXHA_8O0LuGJ1_tou4_KamMlQqAo-aYwyAIDttdmI=", // For public demonstration only, strict rate limit applied.
  timeout: 60000,
  maxRetries: 1,
});

async function main() {
  try {
    const chatCompletion = await client.chat.completions.create({
      model: "Qwen/Qwen2.5-7B-Instruct",
      messages: [
        {
          role: "user",
          content: "What is the capital of France?",
        },
      ],
      stream: false,
      vram_limit: 24,
    });
    console.log("Chat completion response:", chatCompletion);

    return chatCompletion;
  } catch (error) {
    console.error("Error:", error);
  }
}

main();
```

{% endtab %}
{% endtabs %}

This code is standard for invoking OpenAI models through their API. The only modification is the `base_url`, which is changed from the OpenAI URL to the official Crynux Bridge. A live version of this JavaScript code, embedded in a CodePen webpage, allows you to input arbitrary text and receive a response:

{% embed url="<https://codepen.io/Luke-Weber/pen/RNWaOgL>" %}

The API, provided by the official Crynux Bridge, supports both OpenAI-compliant `/completions` and `/chat/completions` endpoints. Features like streaming, tool-calling, and numerous other configuration options are also supported. For a comprehensive list of supported features, please refer to the[ Crynux Bridge documentation](/application-development/crynux-bridge.md).

## GPU VRAM Requirement

The `vram_limit` parameter specifies the minimum VRAM required to execute the task. Crynux Network uses this value to route the task to a node with sufficient GPU memory. This requirement is directly tied to the model size; for example, the 8B model used in the example runs comfortably on a 24GB card.

If this parameter is omitted, the Crynux Bridge defaults to `24` (GB). Therefore, when using a model larger than 8B that requires more memory, you must explicitly set `vram_limit` to a higher value. Failure to do so may result in the task being assigned to an insufficient node, causing a timeout or failure.

## Advanced Usage

For more advanced use cases like Tool Calling, Structured Output, and integrations with LangChain/LangGraph, please refer to the following guides:

{% content-ref url="/pages/YWBZG1xi78hEF57FbIrU" %}
[Tool Use/Function Calling](/application-development/how-to-run-llm-using-crynux-network/tool-use.md)
{% endcontent-ref %}

{% content-ref url="/pages/X4h8fSteUIuCspbHdMV1" %}
[Structured Output](/application-development/how-to-run-llm-using-crynux-network/structured-ouput.md)
{% endcontent-ref %}

{% content-ref url="/pages/d3Y2YDAR411um8SH0b5e" %}
[Vision Language Models (VLM)](/application-development/how-to-run-llm-using-crynux-network/vision-language-models.md)
{% endcontent-ref %}

{% content-ref url="/pages/tlRR30Y1MoplaOYmaQwF" %}
[Integration with LangChain & LangGraph](/application-development/how-to-run-llm-using-crynux-network/langchain.md)
{% endcontent-ref %}

{% content-ref url="/pages/aEuPw0R8uRvksFPmFXzp" %}
[Hermes Agent Integration](/application-development/how-to-run-llm-using-crynux-network/hermes-agent-integration.md)
{% endcontent-ref %}

The API Key in the example code is for public demonstration purposes only and has a strict rate limit, making it unsuitable for production environments. To use the Crynux Network in production, choose one of the following methods:

## Method 1: Using the Official Crynux Bridge

You can request a separate API Key with a higher quota from the Crynux Discord server. Join the server and request new keys from an admin in the "applications" channel.

{% embed url="<https://discord.gg/y8YKxb7uZk>" %}

## Method 2: Hosting Your Own Crynux Bridge

You can host your own instance of the Crynux Bridge to provide private APIs for your application. This approach gives you greater control over various system aspects, including reliability and speed-related configurations.

Starting a Crynux Bridge is as straightforward as running a Docker container. An additional requirement is a wallet funded with sufficient (test) CNX to cover the tasks you run on the network. And at this moment, you can get test CNXs for free in the [Crynux Discord](https://discord.gg/y8YKxb7uZk) as well.

Crynux Bridge is fully open-sourced on [GitHub](https://github.com/crynux-network/crynux-bridge). A step-by-step guide for starting a Crynux Bridge instance is available in the following document:

{% content-ref url="/pages/kiPKEEQwV77hmCOGd58B" %}
[Crynux Bridge](/application-development/crynux-bridge.md)
{% endcontent-ref %}

## Method 3: Sending Tasks Directly to the Blockchain

You can bypass the Crynux Bridge entirely and interact directly with the blockchain and Crynux Relay to send tasks. Crynux SDKs are available in various languages and can be embedded directly into your code to run LLM tasks. Please consult the Crynux SDK documentation for detailed usage instructions:

{% content-ref url="/pages/T7IKwH1gpqgUPdW6UYq0" %}
[Crynux SDK](/application-development/crynux-sdk.md)
{% endcontent-ref %}


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.crynux.io/application-development/how-to-run-llm-using-crynux-network.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.