***
Welcome to KoboldCpp - Version 1.124.2
Loading Chat Completions Adapter: /var/folders/2s/vty41l455yv698g4xkgfv7z00000gn/T/_MEI000036beF5o36X/kcpp_adapters/AutoGuess.json
Chat Completions Adapter Loaded
MacOS detected: Auto GPU layers set to maximum
System: Darwin Darwin Kernel Version 24.6.0: Sun Oct 11 23:44:11 PDT 2026; root:xnu-11417.140.69.x.x~2/RELEASE_ARM64_VMAPPLE arm64 arm
Unable to determine GPU Memory
Unable to determine available RAM
Initializing dynamic library: koboldcpp_default.so
==========
Namespace(usecuda=None, usevulkan=None, usecpu=False, model=['models/qwen2.5-0.5b-instruct-q8_0.gguf'], model_param='models/qwen2.5-0.5b-instruct-q8_0.gguf', config=None, contextsize=4096, gpulayers=200, host='', launch=False, tensor_split=None, threads=1, port=5001, port_param=5001, version=False, agent=False, agentload='', analyze='', autofit=False, autofitpadding=1024, batchsize=512, ubatchsize=-1, benchmark='results/ci-2/macos-15-arm64/bench-q8_0/bench.csv', blasthreads=0, chatcompletionsadapter='AutoGuess', cli=False, debugmode=0, defaultgenamt=2048, device='', downloaddir='', draftamount=4, draftgpulayers=999, draftgpusplit=None, draftmodel='', enableguidance=False, exportconfig='', exporttemplate='', failsafe=False, foreground=False, gendefaults='', gendefaultsoverwrite=False, genlimit=0, highpriority=False, ignoremissing=False, jinja=False, jinja_kwargs='', jinjatemplate='', jinjathink='default', lora=None, loramult=1.0, lowvram=False, maingpu=-1, maxrequestsize=32, mcpfile='', mmproj='', mmprojcpu=False, lazy_mode='auto', moe_cache_mib=0, moecpu=0, ffncpu=0, moeexperts=-1, multiplayer=False, nocertify=False, nofastforward=False, noflashattention=False, noavx2=False, nobostoken=<redacted> nommq=False, nomodel=False, noshift=False, noswa=False, onready='', allow_config_onready=False, overridekv='', overridenativecontext=0, overridetensors='', parallelrequests=1, paralleltotalcontext=0, password=<redacted> preloadstory='', prompt='', quantkv='f16', queuelimit=10, quiet=False, ratelimit=0, reasoningeffort='default', remotetunnel=False, ropeconfig=[0.0, 10000.0], savedatafile='', singleinstance=False, smartcache=0, smartcontext=False, splitmode='layer', ssl=None, swapadding=0, unpack='', usemtp=False, usemlock=False, usemmap=False, usedirectio=False, visionmaxres=1024, visionmaxtokens=-1, visionmintokens=-1, websearch=False, showgui=False, skiplauncher=True, hordemodelname='', hordeworkername='', hordekey='', hordemaxctx=0, hordegenlen=0, sdaudiovae='', sdclamped=0, sdclampedsoft=0, sdclip1='', sdclip2='', sdclipdevice=-2, sdconvdirect='off', sdflashattention=False, sdlora=[], sdloramult=[1.0], sdmaingpu=None, sdmodel='', sdoffloadcpu=False, sdphotomaker='', sdquant=0, sdllm='', sdthreads=0, sdtiledvae=512, sdupscaler='', sdvae='', sdvaeauto=False, sdvaedevice=-1, sdvramlimit=0, whispermodel='', ttsmodel='', ttswavtokenizer='', ttsgpu=False, ttsmaxlen=4096, ttsthreads=0, ttsdir='', musicllm='', musicembeddings='', musicdiffusion='', musicvae='', musiclowvram=False, embeddingsmodel='', embeddingsmaxctx=4096, embeddingsgpu=False, rpcmode='disabled', rpcport=5551, rpchost='0.0.0.0', rpcdevice='', rpccache='', rpctargets='', admin=False, adminpassword=<redacted> admindir='', adminunloadtimeout=0, routermode=False, reqtimeout=600, autoswapmode=False, autoswapthreshold=256, baseconfig='', hordeconfig=None, sdconfig=None, noblas=False, nommap=False, nopipelineparallel=False, pipelineparallel=False, sdnotile=False, sdt5xxl=None, sdvaecpu=False, sdclipgpu=False, forceversion=False, sdgendefaults=False, flashattention=False, useswa=False, multiuser=None, testmemory=False, run_bundled_agent=False, agent_base_url=None, agent_api_key=<redacted> proxy_port=None)
==========
Loading Text Model: /Users/<user>/work/test-KoboldCpp/test-KoboldCpp/models/qwen2.5-0.5b-instruct-q8_0.gguf

The reported GGUF Arch is: qwen2
Arch Category: 24

---
Identified as GGUF model.
Attempting to Load...
---
Using automatic RoPE scaling for GGUF. If the model has custom RoPE settings, they'll be used directly instead!
System Info: AVX = 0 | AVX_VNNI = 0 | AVX2 = 0 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | AVX512_BF16 = 0 | AMX_INT8 = 0 | FMA = 0 | NEON = 1 | SVE = 0 | ARM_FMA = 1 | F16C = 0 | FP16_VA = 1 | RISCV_VECT = 0 | WASM_SIMD = 0 | SSE3 = 0 | SSSE3 = 0 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 1 | 
Number of Backends: 4
Backend 0: MTL
Backend 1: BLAS
Backend 2: RPC
Backend 3: CPU

This model does not use SWA
Automatic RoPE Scaling: Using model internal value.
Threadpool set to 1 threads and 1 blasthreads...

This architecture has explicitly disabled the BOS token - if you need it, you must add it manually.
Starting model warm up, please wait a moment...


Processing Prompt (0 / 3996 tokens)
Processing Prompt (512 / 3996 tokens)
Processing Prompt (1024 / 3996 tokens)
Processing Prompt (1536 / 3996 tokens)
Processing Prompt (2048 / 3996 tokens)
Processing Prompt (2560 / 3996 tokens)
Processing Prompt (3072 / 3996 tokens)
Processing Prompt (3584 / 3996 tokens)
Processing Prompt (3996 / 3996 tokens)

Generating (1 / 100 tokens)
Generating (2 / 100 tokens)
Generating (3 / 100 tokens)
Generating (4 / 100 tokens)
Generating (5 / 100 tokens)
Generating (6 / 100 tokens)
Generating (7 / 100 tokens)
Generating (8 / 100 tokens)
Generating (9 / 100 tokens)
Generating (10 / 100 tokens)
Generating (11 / 100 tokens)
Generating (12 / 100 tokens)
Generating (13 / 100 tokens)
Generating (14 / 100 tokens)
Generating (15 / 100 tokens)
Generating (16 / 100 tokens)
Generating (17 / 100 tokens)
Generating (18 / 100 tokens)
Generating (19 / 100 tokens)
Generating (20 / 100 tokens)
Generating (21 / 100 tokens)
Generating (22 / 100 tokens)
Generating (23 / 100 tokens)
Generating (24 / 100 tokens)
Generating (25 / 100 tokens)
Generating (26 / 100 tokens)
Generating (27 / 100 tokens)
Generating (28 / 100 tokens)
Generating (29 / 100 tokens)
Generating (30 / 100 tokens)
Generating (31 / 100 tokens)
Generating (32 / 100 tokens)
Generating (33 / 100 tokens)
Generating (34 / 100 tokens)
Generating (35 / 100 tokens)
Generating (36 / 100 tokens)
Generating (37 / 100 tokens)
Generating (38 / 100 tokens)
Generating (39 / 100 tokens)
Generating (40 / 100 tokens)
Generating (41 / 100 tokens)
Generating (42 / 100 tokens)
Generating (43 / 100 tokens)
Generating (44 / 100 tokens)
Generating (45 / 100 tokens)
Generating (46 / 100 tokens)
Generating (47 / 100 tokens)
Generating (48 / 100 tokens)
Generating (49 / 100 tokens)
Generating (50 / 100 tokens)
Generating (51 / 100 tokens)
Generating (52 / 100 tokens)
Generating (53 / 100 tokens)
Generating (54 / 100 tokens)
Generating (55 / 100 tokens)
Generating (56 / 100 tokens)
Generating (57 / 100 tokens)
Generating (58 / 100 tokens)
Generating (59 / 100 tokens)
Generating (60 / 100 tokens)
Generating (61 / 100 tokens)
Generating (62 / 100 tokens)
Generating (63 / 100 tokens)
Generating (64 / 100 tokens)
Generating (65 / 100 tokens)
Generating (66 / 100 tokens)
Generating (67 / 100 tokens)
Generating (68 / 100 tokens)
Generating (69 / 100 tokens)
Generating (70 / 100 tokens)
Generating (71 / 100 tokens)
Generating (72 / 100 tokens)
Generating (73 / 100 tokens)
Generating (74 / 100 tokens)
Generating (75 / 100 tokens)
Generating (76 / 100 tokens)
Generating (77 / 100 tokens)
Generating (78 / 100 tokens)
Generating (79 / 100 tokens)
Generating (80 / 100 tokens)
Generating (81 / 100 tokens)
Generating (82 / 100 tokens)
Generating (83 / 100 tokens)
Generating (84 / 100 tokens)
Generating (85 / 100 tokens)
Generating (86 / 100 tokens)
Generating (87 / 100 tokens)
Generating (88 / 100 tokens)
Generating (89 / 100 tokens)
Generating (90 / 100 tokens)
Generating (91 / 100 tokens)
Generating (92 / 100 tokens)
Generating (93 / 100 tokens)
Generating (94 / 100 tokens)
Generating (95 / 100 tokens)
Generating (96 / 100 tokens)
Generating (97 / 100 tokens)
Generating (98 / 100 tokens)
Generating (99 / 100 tokens)
Generating (100 / 100 tokens)
[13:17:01] CtxLimit:4096/4096, Init:0.10s, Processed:3996 in 31.60s (126.47T/s), Generated:100/100 in 20.34s (4.92T/s), Total:52.03sLoad Text Model OK: True
Chat completion heuristic: ChatML (Qwen 2.5 based)
Embedded KoboldAI Lite loaded.
Embedded API docs loaded.
Llama.cpp UI loaded.
======
Active Modules: TextGeneration
Inactive Modules: ImageGeneration VoiceRecognition MultimodalVision MultimodalAudio NetworkMultiplayer ApiKeyPassword WebSearchProxy TextToSpeech VectorEmbeddings AdminControl MCPBridge MusicGen RouterMode
Enabled APIs: KoboldCppApi OpenAiApi OllamaApi AnthropicApi

Running benchmark (Save to File: results/ci-2/macos-15-arm64/bench-q8_0/bench.csv)...

Benchmark Completed - v1.124.2 Results:
======
Flags: {'usecuda': None, 'usevulkan': None, 'usecpu': False, 'model': ['models/qwen2.5-0.5b-instruct-q8_0.gguf'], 'model_param': 'models/qwen2.5-0.5b-instruct-q8_0.gguf', 'config': None, 'contextsize': 4096, 'gpulayers': 200, 'host': '', 'launch': False, 'tensor_split': None, 'threads': 1, 'port': 5001, 'port_param': 5001, 'version': False, 'agent': False, 'agentload': '', 'analyze': '', 'autofit': False, 'autofitpadding': 1024, 'batchsize': 512, 'ubatchsize': -1, 'benchmark': 'results/ci-2/macos-15-arm64/bench-q8_0/bench.csv', 'blasthreads': 1, 'chatcompletionsadapter': 'AutoGuess', 'cli': False, 'debugmode': 0, 'defaultgenamt': 2048, 'device': '', 'downloaddir': '', 'draftamount': 4, 'draftgpulayers': 999, 'draftgpusplit': None, 'draftmodel': '', 'enableguidance': False, 'exportconfig': '', 'exporttemplate': '', 'failsafe': False, 'foreground': False, 'gendefaults': '', 'gendefaultsoverwrite': False, 'genlimit': 0, 'highpriority': False, 'ignoremissing': False, 'jinja': False, 'jinja_kwargs': '', 'jinjatemplate': '', 'jinjathink': 'default', 'lora': None, 'loramult': 1.0, 'lowvram': False, 'maingpu': -1, 'maxrequestsize': 32, 'mcpfile': '', 'mmproj': '', 'mmprojcpu': False, 'lazy_mode': 'auto', 'moe_cache_mib': 0, 'moecpu': 0, 'ffncpu': 0, 'moeexperts': -1, 'multiplayer': False, 'nocertify': False, 'nofastforward': False, 'noflashattention': False, 'noavx2': False, 'nobostoken': False, 'nommq': False, 'nomodel': False, 'noshift': False, 'noswa': False, 'onready': '', 'allow_config_onready': False, 'overridekv': '', 'overridenativecontext': 0, 'overridetensors': '', 'parallelrequests': 1, 'paralleltotalcontext': 0, 'password': None, 'preloadstory': '', 'prompt': '', 'quantkv': 'f16', 'queuelimit': 10, 'quiet': False, 'ratelimit': 0, 'reasoningeffort': 'default', 'remotetunnel': False, 'ropeconfig': [0.0, 10000.0], 'savedatafile': '', 'singleinstance': False, 'smartcache': 0, 'smartcontext': False, 'splitmode': 'layer', 'ssl': None, 'swapadding': 0, 'unpack': '', 'usemtp': False, 'usemlock': False, 'usemmap': False, 'usedirectio': False, 'visionmaxres': 1024, 'visionmaxtokens': -1, 'visionmintokens': -1, 'websearch': False, 'showgui': False, 'skiplauncher': True, 'hordemodelname': '', 'hordeworkername': '', 'hordekey': '', 'hordemaxctx': 0, 'hordegenlen': 0, 'sdaudiovae': '', 'sdclamped': 0, 'sdclampedsoft': 0, 'sdclip1': '', 'sdclip2': '', 'sdclipdevice': -2, 'sdconvdirect': 'off', 'sdflashattention': False, 'sdlora': [], 'sdloramult': [1.0], 'sdmaingpu': None, 'sdmodel': '', 'sdoffloadcpu': False, 'sdphotomaker': '', 'sdquant': 0, 'sdllm': '', 'sdthreads': 0, 'sdtiledvae': 512, 'sdupscaler': '', 'sdvae': '', 'sdvaeauto': False, 'sdvaedevice': -1, 'sdvramlimit': 0, 'whispermodel': '', 'ttsmodel': '', 'ttswavtokenizer': '', 'ttsgpu': False, 'ttsmaxlen': 4096, 'ttsthreads': 0, 'ttsdir': '', 'musicllm': '', 'musicembeddings': '', 'musicdiffusion': '', 'musicvae': '', 'musiclowvram': False, 'embeddingsmodel': '', 'embeddingsmaxctx': 4096, 'embeddingsgpu': False, 'rpcmode': 'disabled', 'rpcport': 5551, 'rpchost': '0.0.0.0', 'rpcdevice': '', 'rpccache': '', 'rpctargets': '', 'admin': False, 'adminpassword': None, 'admindir': '', 'adminunloadtimeout': 0, 'routermode': False, 'reqtimeout': 600, 'autoswapmode': False, 'autoswapthreshold': 256, 'baseconfig': '', 'hordeconfig': None, 'sdconfig': None, 'noblas': False, 'nommap': False, 'nopipelineparallel': False, 'pipelineparallel': False, 'sdnotile': False, 'sdt5xxl': None, 'sdvaecpu': False, 'sdclipgpu': False, 'forceversion': False, 'sdgendefaults': False, 'flashattention': False, 'useswa': False, 'multiuser': None, 'testmemory': False, 'run_bundled_agent': False, 'agent_base_url': None, 'agent_api_key': None, 'proxy_port': None}
Timestamp: 2026-11-13 13:17:01.971785+00:00
Backend: koboldcpp_default.so
Layers: 200
Model: qwen2.5-0.5b-instruct-q8_0
MaxCtx: 4096
GenAmount: 100
-----
ProcessingTime: 31.597s
ProcessingSpeed: 126.47T/s
GenerationTime: 20.341s
GenerationSpeed: 4.92T/s
TotalTime: 51.938s
Output:  1 1 1 1
-----
Server was not started, main function complete. Idling.
ggml_metal_device_init: tensor API disabled for pre-M5 and pre-A19 devices
ggml_metal_library_init: default.metallib not found, loading kernel sources
ggml_metal_library_init: GGML_METAL_PATH_RESOURCES = /private/var/folders/2s/vty41l455yv698g4xkgfv7z00000gn/T/_MEI000036beF5o36X
ggml_metal_library_init: loading '/private/var/folders/2s/vty41l455yv698g4xkgfv7z00000gn/T/_MEI000036beF5o36X/kernels/fa_aux.metal'
ggml_metal_library_init: loading '/private/var/folders/2s/vty41l455yv698g4xkgfv7z00000gn/T/_MEI000036beF5o36X/kernels/fa_f16.metal'
ggml_metal_library_init: loading '/private/var/folders/2s/vty41l455yv698g4xkgfv7z00000gn/T/_MEI000036beF5o36X/kernels/fa_f32.metal'
ggml_metal_library_init: loading '/private/var/folders/2s/vty41l455yv698g4xkgfv7z00000gn/T/_MEI000036beF5o36X/kernels/fa_q4_0.metal'
ggml_metal_library_init: loading '/private/var/folders/2s/vty41l455yv698g4xkgfv7z00000gn/T/_MEI000036beF5o36X/kernels/fa_q4_1.metal'
ggml_metal_library_init: loading '/private/var/folders/2s/vty41l455yv698g4xkgfv7z00000gn/T/_MEI000036beF5o36X/kernels/fa_q5_0.metal'
ggml_metal_library_init: loading '/private/var/folders/2s/vty41l455yv698g4xkgfv7z00000gn/T/_MEI000036beF5o36X/kernels/fa_q5_1.metal'
ggml_metal_library_init: loading '/private/var/folders/2s/vty41l455yv698g4xkgfv7z00000gn/T/_MEI000036beF5o36X/kernels/fa_q8_0.metal'
ggml_metal_library_init: loading '/private/var/folders/2s/vty41l455yv698g4xkgfv7z00000gn/T/_MEI000036beF5o36X/kernels/fa_tensor.metal'
ggml_metal_library_init: loading '/private/var/folders/2s/vty41l455yv698g4xkgfv7z00000gn/T/_MEI000036beF5o36X/kernels/fa_vec_f16.metal'
ggml_metal_library_init: loading '/private/var/folders/2s/vty41l455yv698g4xkgfv7z00000gn/T/_MEI000036beF5o36X/kernels/fa_vec_f32.metal'
ggml_metal_library_init: loading '/private/var/folders/2s/vty41l455yv698g4xkgfv7z00000gn/T/_MEI000036beF5o36X/kernels/fa_vec_q4_0.metal'
ggml_metal_library_init: loading '/private/var/folders/2s/vty41l455yv698g4xkgfv7z00000gn/T/_MEI000036beF5o36X/kernels/fa_vec_q4_1.metal'
ggml_metal_library_init: loading '/private/var/folders/2s/vty41l455yv698g4xkgfv7z00000gn/T/_MEI000036beF5o36X/kernels/fa_vec_q5_0.metal'
ggml_metal_library_init: loading '/private/var/folders/2s/vty41l455yv698g4xkgfv7z00000gn/T/_MEI000036beF5o36X/kernels/fa_vec_q5_1.metal'
ggml_metal_library_init: loading '/private/var/folders/2s/vty41l455yv698g4xkgfv7z00000gn/T/_MEI000036beF5o36X/kernels/fa_vec_q8_0.metal'
ggml_metal_library_init: loading '/private/var/folders/2s/vty41l455yv698g4xkgfv7z00000gn/T/_MEI000036beF5o36X/kernels/mul_mv.metal'
ggml_metal_library_init: loading '/private/var/folders/2s/vty41l455yv698g4xkgfv7z00000gn/T/_MEI000036beF5o36X/kernels/mul_mv_mma.metal'
ggml_metal_library_init: loading '/private/var/folders/2s/vty41l455yv698g4xkgfv7z00000gn/T/_MEI000036beF5o36X/kernels/mul_mm.metal'
ggml_metal_library_init: loading '/private/var/folders/2s/vty41l455yv698g4xkgfv7z00000gn/T/_MEI000036beF5o36X/kernels/quantize.metal'
ggml_metal_library_init: loading '/private/var/folders/2s/vty41l455yv698g4xkgfv7z00000gn/T/_MEI000036beF5o36X/kernels/softmax.metal'
ggml_metal_library_init: loading '/private/var/folders/2s/vty41l455yv698g4xkgfv7z00000gn/T/_MEI000036beF5o36X/kernels/norm.metal'
ggml_metal_library_init: loading '/private/var/folders/2s/vty41l455yv698g4xkgfv7z00000gn/T/_MEI000036beF5o36X/kernels/unary.metal'
ggml_metal_library_init: loading '/private/var/folders/2s/vty41l455yv698g4xkgfv7z00000gn/T/_MEI000036beF5o36X/kernels/binbcast.metal'
ggml_metal_library_init: loading '/private/var/folders/2s/vty41l455yv698g4xkgfv7z00000gn/T/_MEI000036beF5o36X/kernels/reduce.metal'
ggml_metal_library_init: loading '/private/var/folders/2s/vty41l455yv698g4xkgfv7z00000gn/T/_MEI000036beF5o36X/kernels/tri.metal'
ggml_metal_library_init: loading '/private/var/folders/2s/vty41l455yv698g4xkgfv7z00000gn/T/_MEI000036beF5o36X/kernels/ssm.metal'
ggml_metal_library_init: loading '/private/var/folders/2s/vty41l455yv698g4xkgfv7z00000gn/T/_MEI000036beF5o36X/kernels/wkv.metal'
ggml_metal_library_init: loading '/private/var/folders/2s/vty41l455yv698g4xkgfv7z00000gn/T/_MEI000036beF5o36X/kernels/gated_delta_net.metal'
ggml_metal_library_init: loading '/private/var/folders/2s/vty41l455yv698g4xkgfv7z00000gn/T/_MEI000036beF5o36X/kernels/solve_tri.metal'
ggml_metal_library_init: loading '/private/var/folders/2s/vty41l455yv698g4xkgfv7z00000gn/T/_MEI000036beF5o36X/kernels/rope.metal'
ggml_metal_library_init: loading '/private/var/folders/2s/vty41l455yv698g4xkgfv7z00000gn/T/_MEI000036beF5o36X/kernels/conv.metal'
ggml_metal_library_init: loading '/private/var/folders/2s/vty41l455yv698g4xkgfv7z00000gn/T/_MEI000036beF5o36X/kernels/upscale.metal'
ggml_metal_library_init: loading '/private/var/folders/2s/vty41l455yv698g4xkgfv7z00000gn/T/_MEI000036beF5o36X/kernels/argsort.metal'
ggml_metal_library_init: loading '/private/var/folders/2s/vty41l455yv698g4xkgfv7z00000gn/T/_MEI000036beF5o36X/kernels/pool.metal'
ggml_metal_library_init: loading '/private/var/folders/2s/vty41l455yv698g4xkgfv7z00000gn/T/_MEI000036beF5o36X/kernels/misc.metal'
ggml_metal_library_compile_all: compiled 'fa_aux' library in 0.025 sec
ggml_metal_library_compile_all: compiled 'fa_f16' library in 0.019 sec
ggml_metal_library_compile_all: compiled 'fa_f32' library in 0.023 sec
ggml_metal_library_compile_all: compiled 'fa_q4_0' library in 0.025 sec
ggml_metal_library_compile_all: compiled 'fa_q4_1' library in 0.027 sec
ggml_metal_library_compile_all: compiled 'fa_q5_0' library in 0.020 sec
ggml_metal_library_compile_all: compiled 'fa_q5_1' library in 0.021 sec
ggml_metal_library_compile_all: compiled 'fa_q8_0' library in 0.009 sec
ggml_metal_library_compile_all: compiled 'fa_tensor' library in 0.002 sec
ggml_metal_library_compile_all: compiled 'fa_vec_f16' library in 0.009 sec
ggml_metal_library_compile_all: compiled 'fa_vec_f32' library in 0.013 sec
ggml_metal_library_compile_all: compiled 'fa_vec_q4_0' library in 0.016 sec
ggml_metal_library_compile_all: compiled 'fa_vec_q4_1' library in 0.017 sec
ggml_metal_library_compile_all: compiled 'fa_vec_q5_0' library in 0.044 sec
ggml_metal_library_compile_all: compiled 'fa_vec_q5_1' library in 0.015 sec
ggml_metal_library_compile_all: compiled 'fa_vec_q8_0' library in 0.042 sec
ggml_metal_library_compile_all: compiled 'mul_mv' library in 0.052 sec
ggml_metal_library_compile_all: compiled 'mul_mv_mma' library in 0.038 sec
ggml_metal_library_compile_all: compiled 'mul_mm' library in 0.025 sec
ggml_metal_library_compile_all: compiled 'quantize' library in 0.021 sec
ggml_metal_library_compile_all: compiled 'softmax' library in 0.010 sec
ggml_metal_library_compile_all: compiled 'norm' library in 0.012 sec
ggml_metal_library_compile_all: compiled 'unary' library in 0.011 sec
ggml_metal_library_compile_all: compiled 'binbcast' library in 0.005 sec
ggml_metal_library_compile_all: compiled 'reduce' library in 0.003 sec
ggml_metal_library_compile_all: compiled 'tri' library in 0.005 sec
ggml_metal_library_compile_all: compiled 'ssm' library in 0.003 sec
ggml_metal_library_compile_all: compiled 'wkv' library in 0.003 sec
ggml_metal_library_compile_all: compiled 'gated_delta_net' library in 0.005 sec
ggml_metal_library_compile_all: compiled 'solve_tri' library in 0.003 sec
ggml_metal_library_compile_all: compiled 'rope' library in 0.004 sec
ggml_metal_library_compile_all: compiled 'conv' library in 0.008 sec
ggml_metal_library_compile_all: compiled 'upscale' library in 0.003 sec
ggml_metal_library_compile_all: compiled 'argsort' library in 0.003 sec
ggml_metal_library_compile_all: compiled 'pool' library in 0.003 sec
ggml_metal_library_compile_all: compiled 'misc' library in 0.004 sec
ggml_metal_library_compile_all: loaded 36 libraries from source in 0.179 sec (max single = 0.052 sec)
ggml_metal_rsets_init: creating a residency set collection (keep_alive = 180 s)
ggml_metal_device_init: GPU name:   MTL0 (Apple Paravirtual device)
ggml_metal_device_init: GPU family: MTLGPUFamilyApple5  (1005)
ggml_metal_device_init: simdgroup reduction   = false
ggml_metal_device_init: simdgroup matrix mul. = false
ggml_metal_device_init: has unified memory    = true
ggml_metal_device_init: has bfloat            = false
ggml_metal_device_init: has tensor            = false
ggml_metal_device_init: use residency sets    = true
ggml_metal_device_init: use shared buffers    = true
ggml_metal_device_init: recommendedMaxWorkingSetSize  =  5010.80 MB
llama_model_loader: loaded meta data with 26 key-value pairs and 291 tensors from /Users/<user>/work/test-KoboldCpp/test-KoboldCpp/models/qwen2.5-0.5b-instruct-q8_0.gguf (version GGUF V3 (latest))
print_info: file format = GGUF V3 (latest)
print_info: file type   = unknown, may not work
print_info: file size   = 638.74 MiB (8.50 BPW) 
llama_prepare_model_devices: using device MTL0 (Apple Paravirtual device) (unknown id) - 4778 MiB free
init_tokenizer: initializing tokenizer for type 2
load: 0 unused tokens
load: control-looking token: <redacted> '</s>' was not control-type; this is probably a bug in the model. its type will be overridden
load: printing all EOG tokens:
load:   - 128247 ('</s>')
load:   - 151643 ('<|endoftext|>')
load:   - 151645 ('<|im_end|>')
load:   - 151662 ('<|fim_pad|>')
load:   - 151663 ('<|repo_name|>')
load:   - 151664 ('<|file_sep|>')
load: special tokens cache size = 23
load: token to piece cache size = 0.9310 MB
print_info: arch                  = qwen2
print_info: vocab_only            = 0
print_info: no_alloc              = 0
print_info: n_ctx_train           = 32768
print_info: n_embd_inp            = 896
print_info: n_embd                = 896
print_info: n_embd_out            = 896
print_info: n_layer               = 24
print_info: n_layer_all           = 24
print_info: n_head                = 14
print_info: n_head_kv             = 2
print_info: n_rot                 = 64
print_info: n_swa                 = 0
print_info: is_swa_any            = 0
print_info: non_causal_type       = 0
print_info: n_embd_head_k         = 64
print_info: n_embd_head_v         = 64
print_info: n_gqa                 = 7
print_info: n_embd_k_gqa          = 128
print_info: n_embd_v_gqa          = 128
print_info: f_norm_eps            = 0.0e+00
print_info: f_norm_rms_eps        = 1.0e-06
print_info: f_clamp_kqv           = 0.0e+00
print_info: f_max_alibi_bias      = 0.0e+00
print_info: f_logit_scale         = 0.0e+00
print_info: f_attn_scale          = 0.0e+00
print_info: f_attn_value_scale    = 0.0000
print_info: n_ff                  = 4864
print_info: n_expert              = 0
print_info: n_expert_used         = 0
print_info: n_expert_groups       = 0
print_info: n_group_used          = 0
print_info: causal attn           = 1
print_info: pooling type          = -1
print_info: rope type             = 2
print_info: rope scaling          = linear
print_info: freq_base_train       = 1000000.0
print_info: freq_scale_train      = 1
print_info: n_ctx_orig_yarn       = 32768
print_info: rope_yarn_log_mul     = 0.0000
print_info: rope_finetuned        = unknown
print_info: model type            = 1B
print_info: model params          = 630.17 M
print_info: general.name          = qwen2.5-0.5b-instruct
print_info: vocab type            = BPE
print_info: n_vocab               = 151936
print_info: n_merges              = 151387
print_info: BOS token             = <redacted> '<|endoftext|>'
print_info: EOS token             = <redacted> '<|im_end|>'
print_info: EOT token             = <redacted> '<|im_end|>'
print_info: PAD token             = <redacted> '<|endoftext|>'
print_info: LF token              = <redacted> 'Ċ'
print_info: FIM PRE token         = <redacted> '<|fim_prefix|>'
print_info: FIM SUF token         = <redacted> '<|fim_suffix|>'
print_info: FIM MID token         = <redacted> '<|fim_middle|>'
print_info: FIM PAD token         = <redacted> '<|fim_pad|>'
print_info: FIM REP token         = <redacted> '<|repo_name|>'
print_info: FIM SEP token         = <redacted> '<|file_sep|>'
print_info: EOG token             = <redacted> '</s>'
print_info: EOG token             = <redacted> '<|endoftext|>'
print_info: EOG token             = <redacted> '<|im_end|>'
print_info: EOG token             = <redacted> '<|fim_pad|>'
print_info: EOG token             = <redacted> '<|repo_name|>'
print_info: EOG token             = <redacted> '<|file_sep|>'
print_info: max token length      = 256
load_tensors: loading model tensors, this can take a while... (load_mode = none)
done_getting_tensors: tensor 'token_embd.weight' (q8_0) (and 169 others) moved from CPU_REPACK, using CPU instead
load_tensors: offloading output layer to GPU
load_tensors: offloading 23 repeating layers to GPU
load_tensors: offloaded 25/25 layers to GPU
load_tensors:          CPU model buffer size =   137.94 MiB
load_tensors:   CPU_REPACK model buffer size =   500.52 MiB
load_tensors:         MTL0 model buffer size =     0.27 MiB
...........................................................
llama_context: constructing llama_context
llama_context: n_seq_max             = 1
llama_context: n_ctx                 = 4352
llama_context: n_ctx_seq             = 4352
llama_context: n_batch               = 512
llama_context: n_ubatch              = 512
llama_context: causal_attn           = 1
llama_context: flash_attn            = enabled
llama_context: kv_unified            = true
llama_context: freq_base             = 1000000.0
llama_context: freq_scale            = 1
llama_context: n_rs_seq              = 0
llama_context: n_outputs_max         = 512
llama_context: n_outputs_max_per_seq = 1
llama_context: n_ctx_seq (4352) < n_ctx_train (32768) -- the full capacity of the model will not be utilized
ggml_metal_init: allocating
ggml_metal_init: picking default device: Apple Paravirtual device
ggml_metal_init: use fusion         = true
ggml_metal_init: use concurrency    = true
ggml_metal_init: use graph optimize = true
set_abort_callback: call
llama_context:        CPU  output buffer size =     0.58 MiB
llama_kv_cache:       MTL0 KV buffer size =    51.00 MiB
llama_kv_cache: size =   51.00 MiB (  4352 cells,  24 layers,  1/1 seqs), K (f16):   25.50 MiB, V (f16):   25.50 MiB
llama_kv_cache: n_rot_k = 0, n_embd_head_k_all = 64
llama_kv_cache: n_rot_v = 0, n_embd_head_k_all = 64
llama_context: enumerating backends
llama_context: backend_ptrs.size() = 3
sched_reserve: reserving ...
sched_reserve: max_nodes = 2328
resolve_fused_ops: resolving fused DeepSeek V4 HC support:
resolve_fused_ops: fused DeepSeek V4 HC pre enabled
resolve_fused_ops: fused DeepSeek V4 HC comb enabled
resolve_fused_ops: fused DeepSeek V4 HC post enabled
sched_reserve:       MTL0 compute buffer size =     8.75 MiB
sched_reserve:        CPU compute buffer size =   300.25 MiB
sched_reserve: graph: nodes = 825, splits = 341, input objects = 4, input tensors = 9
sched_reserve: reserve took 16.06 ms, sched copies = 1
attach_threadpool: call
ggml_metal_library_compile_pipeline: compiling pipeline: base = 'kernel_bin_fuse_f32_f32_f32_4', name = 'kernel_bin_fuse_f32_f32_f32_4_op=2_nf=1_rb=1_cb=0'
ggml_metal_library_compile_pipeline: loaded kernel_bin_fuse_f32_f32_f32_4_op=2_nf=1_rb=1_cb=0   0x6000038dc500 | th_max = 1024 | th_width =   32
ggml_metal_library_compile_pipeline: compiling pipeline: base = 'kernel_bin_fuse_f32_f32_f32_4', name = 'kernel_bin_fuse_f32_f32_f32_4_op=0_nf=1_rb=1_cb=0'
ggml_metal_library_compile_pipeline: loaded kernel_bin_fuse_f32_f32_f32_4_op=0_nf=1_rb=1_cb=0   0x6000038dc5a0 | th_max = 1024 | th_width =   32
ggml_metal_library_compile_pipeline: compiling pipeline: base = 'kernel_rope_neox_f32', name = 'kernel_rope_neox_f32_imrope=0_is_back=0'
ggml_metal_library_compile_pipeline: loaded kernel_rope_neox_f32_imrope=0_is_back=0    0x6000038dc640 | th_max = 1024 | th_width =   32
ggml_metal_library_compile_pipeline: compiling pipeline: base = 'kernel_set_rows_f32_i64_f16', name = 'kernel_set_rows_f32_i64_f16'
ggml_metal_library_compile_pipeline: loaded kernel_set_rows_f32_i64_f16                0x6000038c00a0 | th_max = 1024 | th_width =   32
ggml_metal_library_compile_pipeline: compiling pipeline: base = 'kernel_bin_fuse_f32_f32_f32_4', name = 'kernel_bin_fuse_f32_f32_f32_4_op=0_nf=1_rb=0_cb=0'
ggml_metal_library_compile_pipeline: loaded kernel_bin_fuse_f32_f32_f32_4_op=0_nf=1_rb=0_cb=0   0x6000038dc6e0 | th_max = 1024 | th_width =   32
ggml_metal_library_compile_pipeline: compiling pipeline: base = 'kernel_get_rows_f32', name = 'kernel_get_rows_f32'
ggml_metal_library_compile_pipeline: loaded kernel_get_rows_f32                        0x6000038d4640 | th_max = 1024 | th_width =   32
ggml_metal_library_compile_pipeline: compiling pipeline: base = 'kernel_bin_fuse_f32_f32_f32_4', name = 'kernel_bin_fuse_f32_f32_f32_4_op=2_nf=1_rb=0_cb=0'
ggml_metal_library_compile_pipeline: loaded kernel_bin_fuse_f32_f32_f32_4_op=2_nf=1_rb=0_cb=0   0x6000038c01e0 | th_max = 1024 | th_width =   32