> For the complete documentation index, see [llms.txt](https://developer.gaianet.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://developer.gaianet.ai/getting-started-as-a-developer/setting-up-your-node/customizing-your-node.md).
# Customizing your node
## Pre-set configurations[](https://docs.gaianet.ai/node-guide/customize#pre-set-configurations)
The `gaianet/config.json` file hold all node configuration options, like the LLM settings, vector collection for the knowledge base and prompts. You can either edit this file directly to use your models and vector collections, or you can select a different `config.json` when you initialize the node.
Simply pass in a URL to the `config.json` file in your `gaianet init` command, the URL to the `config.json` must point to the actual text file.
We have several pre-set `config.json` files to choose from [in this repo](https://github.com/GaiaNet-AI/node-configs). For example, the following command initializes a GaiaNet node with a Llama 3 8B model:
```
gaianet init --config https://raw.githubusercontent.com/GaiaNet-AI/node-configs/main/llama-3-8b-instruct/config.json
```
## The config subcommand[](https://docs.gaianet.ai/node-guide/customize#the-config-subcommand)
Even after initializing the node, you are able to make changes to its configuration by editing the `config.json` file. However, it is a best practice to use the `gaianet` CLI in order to make the changes as it is safer and easier.
After any changes have been made to the node configuration, the`gaianet init` command MUST be run.
`gaianet config list` command shows the `config.json` fields that are able to be changed.
## Selecting an LLM[](https://docs.gaianet.ai/node-guide/customize#select-an-llm)
Huggingface has over 10,000 fine-tuned, open-source LLMs to choose from. Each of them have different sizes (the larger models are more capable yet more expensive to run), unique capabilities (for example, some can support large context length, are uncensored, or excel in math), domain expertise (like coding or medicine), and/or styles (some respond in code, can speak like a pirate, etc).
Making changes to the model file, prompt template, and model context length parameters will allow you to replace your node's default LLM with an alternative fine-tuned model. The parameters vary depending on the model, but they can be found on the [gaianet Huggingface organization's](https://huggingface.co/gaianet) model cards.
For example, the following command changes the LLM to a Llama 3 8B model:
```
gaianet config \
--chat-url https://huggingface.co/gaianet/Llama-3-8B-Instruct-GGUF/resolve/main/Meta-Llama-3-8B-Instruct-Q5_K_M.gguf \
--chat-ctx-size 4096 \
--prompt-template llama-3-chat
```
> The llama 3 8B model requires at least 16GB of RAM.
If none of the published fine-tuned models are perfect for your use case, you can also fine-tune your own LLM by following [these guides](https://docs.gaianet.ai/creator-guide/finetune/intro). Your GaiaNet node can run your own fine-tuned models.
> The `--chat-url` argument could point to a local file under `$HOME/gaianet` instead of a public URL. That allows you to use a privately trained or fine-tuned LLM model file.
## Selecting a knowledge base[](https://docs.gaianet.ai/node-guide/customize#select-a-knowledge-base)
A key feature of GaiaNet is that users can create and deploy proprietary knowledge base on the node to supplement the LLM. Each knowledge base is a snapshot file for a vector collection. You can use ready-made knowledge bases, but we encourage you to [create your own knowledge base](https://docs.gaianet.ai/creator-guide/knowledge/concepts) . You will need to do the following:
1. specify the URL to the vector collection (i.e., the `snapshot` or `snapshot.tar.gz` file) in the `snapshot` option.
2. use the same embedding model that generated this vector collection.
3. modify the `system_prompt` to give the model background knowledge.
4. modify the `rag_prompt` to instruct the model to answer the question when context is retrieved from the vector collection.
The following example changes the knowledge base in the node from "Paris guidebook" to "London guidebook":
```
gaianet config \
--snapshot https://huggingface.co/datasets/gaianet/london/resolve/main/london_768_nomic-embed-text-v1.5-f16.snapshot.tar.gz \
--embedding-url https://huggingface.co/gaianet/Nomic-embed-text-v1.5-Embedding-GGUF/resolve/main/nomic-embed-text-v1.5.f16.gguf \
--embedding-ctx-size 8192 \
--system-prompt "You are a tour guide in London, UK. Please answer the question from a London visitor accurately." \
--rag-prompt "The following text is the context for the user question.\n----------------\n"
```
> The `--snapshot` could point to a local file under `$HOME/gaianet` instead of a public URL. That allows you to use a private vector collection snapshot.
Depending on the quality and size of the vectors, you might also need to change the `qdrant-` options to customize the retrieval behavior.
* `qdrant-limit` sets the max number of relevant context to add to the prompt. If your knowledge base consists of large sections of text (i.e., each book chapter is a vector), you should probably make this 1 or 2 to limit the prompt length to a reasonable size.
* `qdrant-score-threshold` is the min match "score" the knowledge content must meet in order to be considerred "relevant". This depends on the quality of the knowledge text and the embedding model. In general, this score should be over 0.5 to reduce irrelevant context in the prompt.
> The embedding model encodes and transforms text into vectors so that the can be stored, searched and retrieved. For different context material, you might need a different embedding model to achieve the optimal performance. The [MTEB leaderboard](https://huggingface.co/spaces/mteb/leaderboard) is a good place to see the performance benchmarks of embedding models. You can find many of them in the [gaianet organization on Huggingface](https://huggingface.co/gaianet).
## Customizing prompts[](https://docs.gaianet.ai/node-guide/customize#customize-prompts)
In `config.json`, you can also customize the prompts. Prompts are often tailored for the fine-tuned LLM or the knowledge base to generate optimal responses from the node.
The `--system-prompt` option sets a system prompt. It provides the background and "personality" of the node. Each API request can set its own system prompt.
The `--rag-prompt` is the prompt to be appended after the system prompt (or user query). It introduces the RAG context retrieved from the vector database, which follows it.
The `--rag-policy` option specifies where the `rag-prompt` and context should go. By default, its value is `system-message` and it puts the context in the system prompt. But you could also set it to `last-user-message`, which puts the `rag-prompt` and context in front of the latest message from the user.
### Next steps[](https://docs.gaianet.ai/node-guide/customize#next-steps)
Remember to re-initialize and re-start the node after you make configuration changes.
```
# If the node is running:
gaianet stop
gaianet init
gaianet start
```
---
# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.
## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.
Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:
```
GET https://developer.gaianet.ai/getting-started-as-a-developer/setting-up-your-node/customizing-your-node.md?ask=&goal=
```
`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.