***
Welcome to KoboldCpp - Version 1.124.2
Loading Chat Completions Adapter: C:\Users\<user>\AppData\Local\Temp\_MEI42162\kcpp_adapters\AutoGuess.json
Chat Completions Adapter Loaded
No GPU or CPU backend was selected. Trying to assign one for you automatically...
Unable to detect VRAM.
Auto Selected Default Backend (flag=0)

Unable to detect VRAM.
No GPU backend found, or could not automatically determine GPU layers. You may prefer to set layers manually.
System: Windows 10.0.26100 AMD64 AMD64 Family 25 Model 1 Stepping 1, AuthenticAMD
Unable to determine GPU Memory
Detected Available RAM: 13679 MB
Initializing dynamic library: koboldcpp_default.dll
Loaded shared common library: koboldcpp_common.dll
==========
Namespace(admin=False, admindir='', adminpassword=<redacted> adminunloadtimeout=0, agent=False, agent_api_key=<redacted> agent_base_url=None, agentload='', allow_config_onready=False, analyze='', autofit=False, autofitpadding=1024, autoswapmode=False, autoswapthreshold=256, baseconfig='', batchsize=512, benchmark=None, blasthreads=0, chatcompletionsadapter='AutoGuess', cli=False, config=None, contextsize=4096, debugmode=0, defaultgenamt=2048, device='', downloaddir='', draftamount=4, draftgpulayers=999, draftgpusplit=None, draftmodel='', embeddingsgpu=False, embeddingsmaxctx=4096, embeddingsmodel='', enableguidance=False, exportconfig='', exporttemplate='', failsafe=False, ffncpu=0, flashattention=False, forceversion=False, foreground=False, gendefaults='', gendefaultsoverwrite=False, genlimit=0, gpulayers=0, highpriority=False, hordeconfig=None, hordegenlen=0, hordekey='', hordemaxctx=0, hordemodelname='', hordeworkername='', host='', ignoremissing=False, jinja=False, jinja_kwargs='', jinjatemplate='', jinjathink='default', launch=False, lazy_mode='auto', lora=None, loramult=1.0, lowvram=False, maingpu=-1, maxrequestsize=32, mcpfile='', mmproj='', mmprojcpu=False, model=['models\\qwen2.5-0.5b-instruct-q4_k_m.gguf'], model_param='models\\qwen2.5-0.5b-instruct-q4_k_m.gguf', moe_cache_mib=0, moecpu=0, moeexperts=-1, multiplayer=False, multiuser=None, musicdiffusion='', musicembeddings='', musicllm='', musiclowvram=False, musicvae='', noavx2=False, noblas=False, nobostoken=<redacted> nocertify=False, nofastforward=False, noflashattention=False, nommap=False, nommq=False, nomodel=False, nopipelineparallel=False, noshift=False, noswa=False, onready='', overridekv='', overridenativecontext=0, overridetensors='', parallelrequests=1, paralleltotalcontext=0, password=<redacted> pipelineparallel=False, port=52139, port_param=52139, preloadstory='', prompt='', proxy_port=None, quantkv='f16', queuelimit=10, quiet=False, ratelimit=0, reasoningeffort='default', remotetunnel=False, reqtimeout=600, ropeconfig=[0.0, 10000.0], routermode=False, rpccache='', rpcdevice='', rpchost='0.0.0.0', rpcmode='disabled', rpcport=5551, rpctargets='', run_bundled_agent=False, savedatafile='', sdaudiovae='', sdclamped=0, sdclampedsoft=0, sdclip1='', sdclip2='', sdclipdevice=-2, sdclipgpu=False, sdconfig=None, sdconvdirect='off', sdflashattention=False, sdgendefaults=False, sdllm='', sdlora=[], sdloramult=[1.0], sdmaingpu=None, sdmodel='', sdnotile=False, sdoffloadcpu=False, sdphotomaker='', sdquant=0, sdt5xxl=None, sdthreads=0, sdtiledvae=512, sdupscaler='', sdvae='', sdvaeauto=False, sdvaecpu=False, sdvaedevice=-1, sdvramlimit=0, showgui=False, singleinstance=False, skiplauncher=True, smartcache=0, smartcontext=False, splitmode='layer', ssl=None, swapadding=0, tensor_split=None, testmemory=False, threads=2, ttsdir='', ttsgpu=False, ttsmaxlen=4096, ttsmodel='', ttsthreads=0, ttswavtokenizer='', ubatchsize=-1, unpack='', usecpu=False, usecuda=None, usedirectio=False, usemlock=False, usemmap=False, usemtp=False, useswa=False, usevulkan=None, version=False, visionmaxres=1024, visionmaxtokens=-1, visionmintokens=-1, websearch=False, whispermodel='')
==========
Loading Text Model: D:\a\test-KoboldCpp\test-KoboldCpp\models\qwen2.5-0.5b-instruct-q4_k_m.gguf
llama_model_loader: loaded meta data with 26 key-value pairs and 291 tensors from D:\a\test-KoboldCpp\test-KoboldCpp\models\qwen2.5-0.5b-instruct-q4_k_m.gguf (version GGUF V3 (latest))
print_info: file format = GGUF V3 (latest)
print_info: file type   = unknown, may not work
print_info: file size   = 462.96 MiB (6.16 BPW) 
init_tokenizer: initializing tokenizer for type 2
load: 0 unused tokens
load: control-looking token: <redacted> '</s>' was not control-type; this is probably a bug in the model. its type will be overridden
load: printing all EOG tokens:
load:   - 128247 ('</s>')
load:   - 151643 ('<|endoftext|>')
load:   - 151645 ('<|im_end|>')
load:   - 151662 ('<|fim_pad|>')
load:   - 151663 ('<|repo_name|>')
load:   - 151664 ('<|file_sep|>')
load: special tokens cache size = 23
load: token to piece cache size = 0.9310 MB
print_info: arch                  = qwen2
print_info: vocab_only            = 0
print_info: no_alloc              = 0
print_info: n_ctx_train           = 32768
print_info: n_embd_inp            = 896
print_info: n_embd                = 896
print_info: n_embd_out            = 896
print_info: n_layer               = 24
print_info: n_layer_all           = 24
print_info: n_head                = 14
print_info: n_head_kv             = 2
print_info: n_rot                 = 64
print_info: n_swa                 = 0
print_info: is_swa_any            = 0
print_info: non_causal_type       = 0
print_info: n_embd_head_k         = 64
print_info: n_embd_head_v         = 64
print_info: n_gqa                 = 7
print_info: n_embd_k_gqa          = 128
print_info: n_embd_v_gqa          = 128
print_info: f_norm_eps            = 0.0e+00
print_info: f_norm_rms_eps        = 1.0e-06
print_info: f_clamp_kqv           = 0.0e+00
print_info: f_max_alibi_bias      = 0.0e+00
print_info: f_logit_scale         = 0.0e+00
print_info: f_attn_scale          = 0.0e+00
print_info: f_attn_value_scale    = 0.0000
print_info: n_ff                  = 4864
print_info: n_expert              = 0
print_info: n_expert_used         = 0
print_info: n_expert_groups       = 0
print_info: n_group_used          = 0
print_info: causal attn           = 1
print_info: pooling type          = -1
print_info: rope type             = 2
print_info: rope scaling          = linear
print_info: freq_base_train       = 1000000.0
print_info: freq_scale_train      = 1
print_info: n_ctx_orig_yarn       = 32768
print_info: rope_yarn_log_mul     = 0.0000
print_info: rope_finetuned        = unknown
print_info: model type            = 1B
print_info: model params          = 630.17 M
print_info: general.name          = qwen2.5-0.5b-instruct
print_info: vocab type            = BPE
print_info: n_vocab               = 151936
print_info: n_merges              = 151387
print_info: BOS token             = <redacted> '<|endoftext|>'
print_info: EOS token             = <redacted> '<|im_end|>'
print_info: EOT token             = <redacted> '<|im_end|>'
print_info: PAD token             = <redacted> '<|endoftext|>'
print_info: LF token              = <redacted> 'Ċ'
print_info: FIM PRE token         = <redacted> '<|fim_prefix|>'
print_info: FIM SUF token         = <redacted> '<|fim_suffix|>'
print_info: FIM MID token         = <redacted> '<|fim_middle|>'
print_info: FIM PAD token         = <redacted> '<|fim_pad|>'
print_info: FIM REP token         = <redacted> '<|repo_name|>'
print_info: FIM SEP token         = <redacted> '<|file_sep|>'
print_info: EOG token             = <redacted> '</s>'
print_info: EOG token             = <redacted> '<|endoftext|>'
print_info: EOG token             = <redacted> '<|im_end|>'
print_info: EOG token             = <redacted> '<|fim_pad|>'
print_info: EOG token             = <redacted> '<|repo_name|>'
print_info: EOG token             = <redacted> '<|file_sep|>'
print_info: max token length      = 256
load_tensors: loading model tensors, this can take a while... (load_mode = none)
done_getting_tensors: tensor 'token_embd.weight' (q5_0) (and 278 others) moved from CPU_REPACK, using CPU instead
load_tensors: offloading 0 repeating layers to GPU
load_tensors: offloaded 0/25 layers to GPU
load_tensors:          CPU model buffer size =   434.91 MiB
load_tensors:   CPU_REPACK model buffer size =    28.05 MiB
.....................................................
llama_context: constructing llama_context
llama_context: n_seq_max             = 1
llama_context: n_ctx                 = 4352
llama_context: n_ctx_seq             = 4352
llama_context: n_batch               = 512
llama_context: n_ubatch              = 512
llama_context: causal_attn           = 1
llama_context: flash_attn            = enabled
llama_context: kv_unified            = true
llama_context: freq_base             = 1000000.0
llama_context: freq_scale            = 1
llama_context: n_rs_seq              = 0
llama_context: n_outputs_max         = 512
llama_context: n_outputs_max_per_seq = 1
llama_context: n_ctx_seq (4352) < n_ctx_train (32768) -- the full capacity of the model will not be utilized
set_abort_callback: call
llama_context:        CPU  output buffer size =     0.58 MiB
llama_kv_cache:        CPU KV buffer size =    51.00 MiB
llama_kv_cache: size =   51.00 MiB (  4352 cells,  24 layers,  1/1 seqs), K (f16):   25.50 MiB, V (f16):   25.50 MiB
llama_kv_cache: n_rot_k = 0, n_embd_head_k_all = 64
llama_kv_cache: n_rot_v = 0, n_embd_head_k_all = 64
llama_context: enumerating backends
llama_context: backend_ptrs.size() = 1
sched_reserve: reserving ...
sched_reserve: max_nodes = 2328
resolve_fused_ops: resolving fused DeepSeek V4 HC support:
resolve_fused_ops: fused DeepSeek V4 HC pre enabled
resolve_fused_ops: fused DeepSeek V4 HC comb enabled
resolve_fused_ops: fused DeepSeek V4 HC post enabled
sched_reserve:        CPU compute buffer size =   309.77 MiB
sched_reserve: graph: nodes = 825, splits = 1, input objects = 4, input tensors = 9
sched_reserve: reserve took 5.23 ms, sched copies = 1
attach_threadpool: call
Load Text Model OK: True
Chat completion heuristic: ChatML (Qwen 2.5 based)
Embedded KoboldAI Lite loaded.
Embedded API docs loaded.
Llama.cpp UI loaded.
======
Active Modules: TextGeneration
Inactive Modules: ImageGeneration VoiceRecognition MultimodalVision MultimodalAudio NetworkMultiplayer ApiKeyPassword WebSearchProxy TextToSpeech VectorEmbeddings AdminControl MCPBridge MusicGen RouterMode
Enabled APIs: KoboldCppApi OpenAiApi OllamaApi AnthropicApi
Note: For third party Ollama API Emulation, you should set the port to 11434.
Starting Kobold API on port 52139 at http://localhost:52139/api/
Starting OpenAI Compatible API on port 52139 at http://localhost:52139/v1/
Starting llama.cpp secondary WebUI at http://localhost:52139/lcpp/
======
Please connect to custom endpoint at http://localhost:52139

The reported GGUF Arch is: qwen2
Arch Category: 24

---
Identified as GGUF model.
Attempting to Load...
---
Using automatic RoPE scaling for GGUF. If the model has custom RoPE settings, they'll be used directly instead!
System Info: AVX = 1 | AVX_VNNI = 0 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | AVX512_BF16 = 0 | AMX_INT8 = 0 | FMA = 1 | NEON = 0 | SVE = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | RISCV_VECT = 0 | WASM_SIMD = 0 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 1 | 
Number of Backends: 2
Backend 0: RPC
Backend 1: CPU

This model does not use SWA
Automatic RoPE Scaling: Using model internal value.
Threadpool set to 2 threads and 2 blasthreads...

This architecture has explicitly disabled the BOS token - if you need it, you must add it manually.
Starting model warm up, please wait a moment...


Processing Prompt (0 / 4 tokens)
Processing Prompt (4 / 4 tokens)

Generating (1 / 24 tokens)
Generating (2 / 24 tokens)
Generating (3 / 24 tokens)
Generating (4 / 24 tokens)
Generating (5 / 24 tokens)
Generating (6 / 24 tokens)
Generating (7 / 24 tokens)
Generating (8 / 24 tokens)
Generating (9 / 24 tokens)
Generating (10 / 24 tokens)
Generating (11 / 24 tokens)
Generating (12 / 24 tokens)
Generating (13 / 24 tokens)
Generating (14 / 24 tokens)
Generating (15 / 24 tokens)
Generating (16 / 24 tokens)
Generating (17 / 24 tokens)
Generating (18 / 24 tokens)
Generating (19 / 24 tokens)
Generating (20 / 24 tokens)
Generating (21 / 24 tokens)
Generating (22 / 24 tokens)
Generating (23 / 24 tokens)
Generating (24 / 24 tokens)
[12:47:44] CtxLimit:28/4096, Init:0.00s, Processed:4 in 0.05s (83.33T/s), Generated:24/24 in 0.51s (47.52T/s), Total:0.56s

Processing Prompt (0 / 4 tokens)
Processing Prompt (4 / 4 tokens)

Generating (1 / 24 tokens)
Generating (2 / 24 tokens)
Generating (3 / 24 tokens)
Generating (4 / 24 tokens)
Generating (5 / 24 tokens)
Generating (6 / 24 tokens)
Generating (7 / 24 tokens)
Generating (8 / 24 tokens)
Generating (9 / 24 tokens)
Generating (10 / 24 tokens)
Generating (11 / 24 tokens)
Generating (12 / 24 tokens)
Generating (13 / 24 tokens)
Generating (14 / 24 tokens)
Generating (15 / 24 tokens)
Generating (16 / 24 tokens)
Generating (17 / 24 tokens)
Generating (18 / 24 tokens)
Generating (19 / 24 tokens)
Generating (20 / 24 tokens)
Generating (21 / 24 tokens)
Generating (22 / 24 tokens)
Generating (23 / 24 tokens)
Generating (24 / 24 tokens)
[12:47:45] CtxLimit:28/4096, Init:0.00s, Processed:4 in 0.05s (86.96T/s), Generated:24/24 in 0.49s (48.98T/s), Total:0.54s

Processing Prompt (0 / 14 tokens)
Processing Prompt (14 / 14 tokens)

Generating (1 / 16 tokens)
Generating (2 / 16 tokens)
(EOS token triggered! ID:151645)
[12:47:45] CtxLimit:16/4096, Init:0.00s, Processed:14 in 0.14s (101.45T/s), Generated:2/16 in 0.02s (95.24T/s), Total:0.16s

Processing Prompt (0 / 29 tokens)
Processing Prompt (29 / 29 tokens)

Generating (1 / 8 tokens)
Generating (2 / 8 tokens)
(EOS token triggered! ID:151645)
[12:47:45] CtxLimit:31/4096, Init:0.00s, Processed:29 in 0.26s (111.54T/s), Generated:2/8 in 0.02s (90.91T/s), Total:0.28s
Warning: KCPP Embeddings Model not initialized!


Processing Prompt (0 / 9 tokens)
Processing Prompt (9 / 9 tokens)

Generating (1 / 8 tokens)
Generating (2 / 8 tokens)
(EOS token triggered! ID:151645)
[12:47:46] CtxLimit:11/4096, Init:0.00s, Processed:9 in 0.09s (100.00T/s), Generated:2/8 in 0.02s (90.91T/s), Total:0.11s

Processing Prompt (0 / 5 tokens)
Processing Prompt (5 / 5 tokens)

Generating (1 / 2048 tokens)
Generating (2 / 2048 tokens)
Generating (3 / 2048 tokens)
(EOS token triggered! ID:151645)
[12:47:46] CtxLimit:14/4096, Init:0.00s, Processed:5 in 0.06s (86.21T/s), Generated:3/2048 in 0.05s (56.60T/s), Total:0.11s

Processing Prompt (0 / 6 tokens)
Processing Prompt (6 / 6 tokens)

Generating (1 / 16 tokens)
Generating (2 / 16 tokens)
Generating (3 / 16 tokens)
Generating (4 / 16 tokens)
Generating (5 / 16 tokens)
Generating (6 / 16 tokens)
Generating (7 / 16 tokens)
Generating (8 / 16 tokens)
Generating (9 / 16 tokens)
Generating (10 / 16 tokens)
Generating (11 / 16 tokens)
Generating (12 / 16 tokens)
Generating (13 / 16 tokens)
Generating (14 / 16 tokens)
Generating (15 / 16 tokens)
Generating (16 / 16 tokens)
[12:47:46] CtxLimit:22/4096, Init:0.00s, Processed:6 in 0.07s (89.55T/s), Generated:16/16 in 0.33s (48.48T/s), Total:0.40s

Processing Prompt (0 / 12 tokens)
Processing Prompt (12 / 12 tokens)

Generating (1 / 24 tokens)
Generating (2 / 24 tokens)
Generating (3 / 24 tokens)
Generating (4 / 24 tokens)
Generating (5 / 24 tokens)
Generating (6 / 24 tokens)
Generating (7 / 24 tokens)
Generating (8 / 24 tokens)
Generating (9 / 24 tokens)
Generating (10 / 24 tokens)
Generating (11 / 24 tokens)
Generating (12 / 24 tokens)
Generating (13 / 24 tokens)
Generating (14 / 24 tokens)
Generating (15 / 24 tokens)
Generating (16 / 24 tokens)
Generating (17 / 24 tokens)
Generating (18 / 24 tokens)
Generating (19 / 24 tokens)
Generating (20 / 24 tokens)
Generating (21 / 24 tokens)
(EOS token triggered! ID:151645)
[12:47:47] CtxLimit:33/4096, Init:0.00s, Processed:12 in 0.10s (116.50T/s), Generated:21/24 in 0.44s (47.73T/s), Total:0.54s