***
Welcome to KoboldCpp - Version 1.124.2
Loading Chat Completions Adapter: /tmp/_MEIDBFRun/kcpp_adapters/AutoGuess.json
Chat Completions Adapter Loaded
No GPU or CPU backend was selected. Trying to assign one for you automatically...
Unable to detect VRAM.
Auto Selected Default Backend (flag=0)

Unable to detect VRAM.
No GPU backend found, or could not automatically determine GPU layers. You may prefer to set layers manually.
System: Linux #17~24.04.1-Ubuntu SMP PREEMPT Thu Sep 24 18:46:57 UTC 2026 x86_64 x86_64
Unable to determine GPU Memory
Detected Available RAM: 14906 MB
Initializing dynamic library: koboldcpp_default.so
Loaded shared common library: koboldcpp_common.so
==========
Namespace(admin=False, admindir='', adminpassword=<redacted> adminunloadtimeout=0, agent=False, agent_api_key=<redacted> agent_base_url=None, agentload='', allow_config_onready=False, analyze='', autofit=False, autofitpadding=1024, autoswapmode=False, autoswapthreshold=256, baseconfig='', batchsize=512, benchmark=None, blasthreads=0, chatcompletionsadapter='AutoGuess', cli=False, config=None, contextsize=4096, debugmode=0, defaultgenamt=2048, device='', downloaddir='', draftamount=4, draftgpulayers=999, draftgpusplit=None, draftmodel='', embeddingsgpu=False, embeddingsmaxctx=4096, embeddingsmodel='', enableguidance=False, exportconfig='', exporttemplate='', failsafe=False, ffncpu=0, flashattention=False, forceversion=False, foreground=False, gendefaults='', gendefaultsoverwrite=False, genlimit=0, gpulayers=0, highpriority=False, hordeconfig=None, hordegenlen=0, hordekey='', hordemaxctx=0, hordemodelname='', hordeworkername='', host='', ignoremissing=False, jinja=False, jinja_kwargs='', jinjatemplate='', jinjathink='default', launch=False, lazy_mode='auto', lora=None, loramult=1.0, lowvram=False, maingpu=-1, maxrequestsize=32, mcpfile='', mmproj='', mmprojcpu=False, model=['models/qwen2.5-0.5b-instruct-q2_k.gguf'], model_param='models/qwen2.5-0.5b-instruct-q2_k.gguf', moe_cache_mib=0, moecpu=0, moeexperts=-1, multiplayer=False, multiuser=None, musicdiffusion='', musicembeddings='', musicllm='', musiclowvram=False, musicvae='', noavx2=False, noblas=False, nobostoken=<redacted> nocertify=False, nofastforward=False, noflashattention=False, nommap=False, nommq=False, nomodel=False, nopipelineparallel=False, noshift=False, noswa=False, onready='', overridekv='', overridenativecontext=0, overridetensors='', parallelrequests=1, paralleltotalcontext=0, password=<redacted> pipelineparallel=False, port=39325, port_param=39325, preloadstory='', prompt='', proxy_port=None, quantkv='f16', queuelimit=10, quiet=False, ratelimit=0, reasoningeffort='default', remotetunnel=False, reqtimeout=600, ropeconfig=[0.0, 10000.0], routermode=False, rpccache='', rpcdevice='', rpchost='0.0.0.0', rpcmode='disabled', rpcport=5551, rpctargets='', run_bundled_agent=False, savedatafile='', sdaudiovae='', sdclamped=0, sdclampedsoft=0, sdclip1='', sdclip2='', sdclipdevice=-2, sdclipgpu=False, sdconfig=None, sdconvdirect='off', sdflashattention=False, sdgendefaults=False, sdllm='', sdlora=[], sdloramult=[1.0], sdmaingpu=None, sdmodel='', sdnotile=False, sdoffloadcpu=False, sdphotomaker='', sdquant=0, sdt5xxl=None, sdthreads=0, sdtiledvae=512, sdupscaler='', sdvae='', sdvaeauto=False, sdvaecpu=False, sdvaedevice=-1, sdvramlimit=0, showgui=False, singleinstance=False, skiplauncher=True, smartcache=0, smartcontext=False, splitmode='layer', ssl=None, swapadding=0, tensor_split=None, testmemory=False, threads=2, ttsdir='', ttsgpu=False, ttsmaxlen=4096, ttsmodel='', ttsthreads=0, ttswavtokenizer='', ubatchsize=-1, unpack='', usecpu=False, usecuda=None, usedirectio=False, usemlock=False, usemmap=False, usemtp=False, useswa=False, usevulkan=None, version=False, visionmaxres=1024, visionmaxtokens=-1, visionmintokens=-1, websearch=False, whispermodel='')
==========
Loading Text Model: /home/<user>/work/test-KoboldCpp/test-KoboldCpp/models/qwen2.5-0.5b-instruct-q2_k.gguf
llama_model_loader: loaded meta data with 26 key-value pairs and 291 tensors from /home/<user>/work/test-KoboldCpp/test-KoboldCpp/models/qwen2.5-0.5b-instruct-q2_k.gguf (version GGUF V3 (latest))
print_info: file format = GGUF V3 (latest)
print_info: file type   = (guessed) unknown, may not work
print_info: file size   = 390.28 MiB (5.20 BPW) 
init_tokenizer: initializing tokenizer for type 2
load: 0 unused tokens
load: control-looking token: <redacted> '</s>' was not control-type; this is probably a bug in the model. its type will be overridden
load: printing all EOG tokens:
load:   - 128247 ('</s>')
load:   - 151643 ('<|endoftext|>')
load:   - 151645 ('<|im_end|>')
load:   - 151662 ('<|fim_pad|>')
load:   - 151663 ('<|repo_name|>')
load:   - 151664 ('<|file_sep|>')
load: special tokens cache size = 23
load: token to piece cache size = 0.9310 MB
print_info: arch                  = qwen2
print_info: vocab_only            = 0
print_info: no_alloc              = 0
print_info: n_ctx_train           = 32768
print_info: n_embd_inp            = 896
print_info: n_embd                = 896
print_info: n_embd_out            = 896
print_info: n_layer               = 24
print_info: n_layer_all           = 24
print_info: n_head                = 14
print_info: n_head_kv             = 2
print_info: n_rot                 = 64
print_info: n_swa                 = 0
print_info: is_swa_any            = 0
print_info: non_causal_type       = 0
print_info: n_embd_head_k         = 64
print_info: n_embd_head_v         = 64
print_info: n_gqa                 = 7
print_info: n_embd_k_gqa          = 128
print_info: n_embd_v_gqa          = 128
print_info: f_norm_eps            = 0.0e+00
print_info: f_norm_rms_eps        = 1.0e-06
print_info: f_clamp_kqv           = 0.0e+00
print_info: f_max_alibi_bias      = 0.0e+00
print_info: f_logit_scale         = 0.0e+00
print_info: f_attn_scale          = 0.0e+00
print_info: f_attn_value_scale    = 0.0000
print_info: n_ff                  = 4864
print_info: n_expert              = 0
print_info: n_expert_used         = 0
print_info: n_expert_groups       = 0
print_info: n_group_used          = 0
print_info: causal attn           = 1
print_info: pooling type          = -1
print_info: rope type             = 2
print_info: rope scaling          = linear
print_info: freq_base_train       = 1000000.0
print_info: freq_scale_train      = 1
print_info: n_ctx_orig_yarn       = 32768
print_info: rope_yarn_log_mul     = 0.0000
print_info: rope_finetuned        = unknown
print_info: model type            = 1B
print_info: model params          = 630.17 M
print_info: general.name          = qwen2.5-0.5b-instruct
print_info: vocab type            = BPE
print_info: n_vocab               = 151936
print_info: n_merges              = 151387
print_info: BOS token             = <redacted> '<|endoftext|>'
print_info: EOS token             = <redacted> '<|im_end|>'
print_info: EOT token             = <redacted> '<|im_end|>'
print_info: PAD token             = <redacted> '<|endoftext|>'
print_info: LF token              = <redacted> 'Ċ'
print_info: FIM PRE token         = <redacted> '<|fim_prefix|>'
print_info: FIM SUF token         = <redacted> '<|fim_suffix|>'
print_info: FIM MID token         = <redacted> '<|fim_middle|>'
print_info: FIM PAD token         = <redacted> '<|fim_pad|>'
print_info: FIM REP token         = <redacted> '<|repo_name|>'
print_info: FIM SEP token         = <redacted> '<|file_sep|>'
print_info: EOG token             = <redacted> '</s>'
print_info: EOG token             = <redacted> '<|endoftext|>'
print_info: EOG token             = <redacted> '<|im_end|>'
print_info: EOG token             = <redacted> '<|fim_pad|>'
print_info: EOG token             = <redacted> '<|repo_name|>'
print_info: EOG token             = <redacted> '<|file_sep|>'
print_info: max token length      = 256
load_tensors: loading model tensors, this can take a while... (load_mode = none)
done_getting_tensors: tensor 'token_embd.weight' (iq4_nl) (and 290 others) moved from CPU_REPACK, using CPU instead
load_tensors: offloading 0 repeating layers to GPU
load_tensors: offloaded 0/25 layers to GPU
load_tensors:          CPU model buffer size =   390.28 MiB
................................................
llama_context: constructing llama_context
llama_context: n_seq_max             = 1
llama_context: n_ctx                 = 4352
llama_context: n_ctx_seq             = 4352
llama_context: n_batch               = 512
llama_context: n_ubatch              = 512
llama_context: causal_attn           = 1
llama_context: flash_attn            = enabled
llama_context: kv_unified            = true
llama_context: freq_base             = 1000000.0
llama_context: freq_scale            = 1
llama_context: n_rs_seq              = 0
llama_context: n_outputs_max         = 512
llama_context: n_outputs_max_per_seq = 1
llama_context: n_ctx_seq (4352) < n_ctx_train (32768) -- the full capacity of the model will not be utilized
set_abort_callback: call
llama_context:        CPU  output buffer size =     0.58 MiB
llama_kv_cache:        CPU KV buffer size =    51.00 MiB
llama_kv_cache: size =   51.00 MiB (  4352 cells,  24 layers,  1/1 seqs), K (f16):   25.50 MiB, V (f16):   25.50 MiB
llama_kv_cache: n_rot_k = 0, n_embd_head_k_all = 64
llama_kv_cache: n_rot_v = 0, n_embd_head_k_all = 64
llama_context: enumerating backends
llama_context: backend_ptrs.size() = 1
sched_reserve: reserving ...
sched_reserve: max_nodes = 2328
resolve_fused_ops: resolving fused DeepSeek V4 HC support:
resolve_fused_ops: fused DeepSeek V4 HC pre enabled
resolve_fused_ops: fused DeepSeek V4 HC comb enabled
resolve_fused_ops: fused DeepSeek V4 HC post enabled
sched_reserve:        CPU compute buffer size =   309.77 MiB
sched_reserve: graph: nodes = 825, splits = 1, input objects = 4, input tensors = 9
sched_reserve: reserve took 2.42 ms, sched copies = 1
attach_threadpool: call
Load Text Model OK: True
Chat completion heuristic: ChatML (Qwen 2.5 based)
Embedded KoboldAI Lite loaded.
Embedded API docs loaded.
Llama.cpp UI loaded.
======
Active Modules: TextGeneration
Inactive Modules: ImageGeneration VoiceRecognition MultimodalVision MultimodalAudio NetworkMultiplayer ApiKeyPassword WebSearchProxy TextToSpeech VectorEmbeddings AdminControl MCPBridge MusicGen RouterMode
Enabled APIs: KoboldCppApi OpenAiApi OllamaApi AnthropicApi
Note: For third party Ollama API Emulation, you should set the port to 11434.
Starting Kobold API on port 39325 at http://localhost:39325/api/
Starting OpenAI Compatible API on port 39325 at http://localhost:39325/v1/
Starting llama.cpp secondary WebUI at http://localhost:39325/lcpp/
======
Please connect to custom endpoint at http://localhost:39325

The reported GGUF Arch is: qwen2
Arch Category: 24

---
Identified as GGUF model.
Attempting to Load...
---
Using automatic RoPE scaling for GGUF. If the model has custom RoPE settings, they'll be used directly instead!
System Info: AVX = 1 | AVX_VNNI = 0 | AVX2 = 1 | AVX512 = 0 | AVX512_VBMI = 0 | AVX512_VNNI = 0 | AVX512_BF16 = 0 | AMX_INT8 = 0 | FMA = 1 | NEON = 0 | SVE = 0 | ARM_FMA = 0 | F16C = 1 | FP16_VA = 0 | RISCV_VECT = 0 | WASM_SIMD = 0 | SSE3 = 1 | SSSE3 = 1 | VSX = 0 | MATMUL_INT8 = 0 | LLAMAFILE = 1 | 
Number of Backends: 2
Backend 0: RPC
Backend 1: CPU

This model does not use SWA
Automatic RoPE Scaling: Using model internal value.
Threadpool set to 2 threads and 2 blasthreads...

This architecture has explicitly disabled the BOS token - if you need it, you must add it manually.
Starting model warm up, please wait a moment...


Processing Prompt (0 / 38 tokens)
Processing Prompt (38 / 38 tokens)

Generating (1 / 128 tokens)
Generating (2 / 128 tokens)
Generating (3 / 128 tokens)
Generating (4 / 128 tokens)
Generating (5 / 128 tokens)
Generating (6 / 128 tokens)
Generating (7 / 128 tokens)
Generating (8 / 128 tokens)
Generating (9 / 128 tokens)
Generating (10 / 128 tokens)
Generating (11 / 128 tokens)
Generating (12 / 128 tokens)
Generating (13 / 128 tokens)
Generating (14 / 128 tokens)
Generating (15 / 128 tokens)
Generating (16 / 128 tokens)
Generating (17 / 128 tokens)
Generating (18 / 128 tokens)
Generating (19 / 128 tokens)
Generating (20 / 128 tokens)
Generating (21 / 128 tokens)
Generating (22 / 128 tokens)
Generating (23 / 128 tokens)
Generating (24 / 128 tokens)
Generating (25 / 128 tokens)
Generating (26 / 128 tokens)
Generating (27 / 128 tokens)
Generating (28 / 128 tokens)
Generating (29 / 128 tokens)
Generating (30 / 128 tokens)
Generating (31 / 128 tokens)
Generating (32 / 128 tokens)
Generating (33 / 128 tokens)
Generating (34 / 128 tokens)
Generating (35 / 128 tokens)
Generating (36 / 128 tokens)
Generating (37 / 128 tokens)
Generating (38 / 128 tokens)
Generating (39 / 128 tokens)
Generating (40 / 128 tokens)
Generating (41 / 128 tokens)
Generating (42 / 128 tokens)
Generating (43 / 128 tokens)
Generating (44 / 128 tokens)
Generating (45 / 128 tokens)
Generating (46 / 128 tokens)
Generating (47 / 128 tokens)
Generating (48 / 128 tokens)
Generating (49 / 128 tokens)
Generating (50 / 128 tokens)
Generating (51 / 128 tokens)
Generating (52 / 128 tokens)
Generating (53 / 128 tokens)
Generating (54 / 128 tokens)
Generating (55 / 128 tokens)
Generating (56 / 128 tokens)
Generating (57 / 128 tokens)
Generating (58 / 128 tokens)
Generating (59 / 128 tokens)
Generating (60 / 128 tokens)
Generating (61 / 128 tokens)
Generating (62 / 128 tokens)
Generating (63 / 128 tokens)
Generating (64 / 128 tokens)
Generating (65 / 128 tokens)
Generating (66 / 128 tokens)
Generating (67 / 128 tokens)
Generating (68 / 128 tokens)
Generating (69 / 128 tokens)
Generating (70 / 128 tokens)
Generating (71 / 128 tokens)
Generating (72 / 128 tokens)
Generating (73 / 128 tokens)
Generating (74 / 128 tokens)
Generating (75 / 128 tokens)
Generating (76 / 128 tokens)
Generating (77 / 128 tokens)
Generating (78 / 128 tokens)
Generating (79 / 128 tokens)
Generating (80 / 128 tokens)
Generating (81 / 128 tokens)
Generating (82 / 128 tokens)
Generating (83 / 128 tokens)
Generating (84 / 128 tokens)
Generating (85 / 128 tokens)
Generating (86 / 128 tokens)
Generating (87 / 128 tokens)
Generating (88 / 128 tokens)
Generating (89 / 128 tokens)
Generating (90 / 128 tokens)
Generating (91 / 128 tokens)
Generating (92 / 128 tokens)
Generating (93 / 128 tokens)
Generating (94 / 128 tokens)
Generating (95 / 128 tokens)
Generating (96 / 128 tokens)
Generating (97 / 128 tokens)
Generating (98 / 128 tokens)
Generating (99 / 128 tokens)
Generating (100 / 128 tokens)
Generating (101 / 128 tokens)
Generating (102 / 128 tokens)
Generating (103 / 128 tokens)
Generating (104 / 128 tokens)
Generating (105 / 128 tokens)
Generating (106 / 128 tokens)
Generating (107 / 128 tokens)
Generating (108 / 128 tokens)
Generating (109 / 128 tokens)
Generating (110 / 128 tokens)
Generating (111 / 128 tokens)
Generating (112 / 128 tokens)
Generating (113 / 128 tokens)
Generating (114 / 128 tokens)
Generating (115 / 128 tokens)
Generating (116 / 128 tokens)
Generating (117 / 128 tokens)
Generating (118 / 128 tokens)
Generating (119 / 128 tokens)
Generating (120 / 128 tokens)
Generating (121 / 128 tokens)
Generating (122 / 128 tokens)
Generating (123 / 128 tokens)
Generating (124 / 128 tokens)
Generating (125 / 128 tokens)
Generating (126 / 128 tokens)
Generating (127 / 128 tokens)
Generating (128 / 128 tokens)
[12:39:59] CtxLimit:166/4096, Init:0.00s, Processed:38 in 0.31s (124.18T/s), Generated:128/128 in 2.66s (48.12T/s), Total:2.97s

Processing Prompt (0 / 1 tokens)
Processing Prompt (1 / 1 tokens)

Generating (1 / 128 tokens)
Generating (2 / 128 tokens)
Generating (3 / 128 tokens)
Generating (4 / 128 tokens)
Generating (5 / 128 tokens)
Generating (6 / 128 tokens)
Generating (7 / 128 tokens)
Generating (8 / 128 tokens)
Generating (9 / 128 tokens)
Generating (10 / 128 tokens)
Generating (11 / 128 tokens)
Generating (12 / 128 tokens)
Generating (13 / 128 tokens)
Generating (14 / 128 tokens)
Generating (15 / 128 tokens)
Generating (16 / 128 tokens)
Generating (17 / 128 tokens)
Generating (18 / 128 tokens)
Generating (19 / 128 tokens)
Generating (20 / 128 tokens)
Generating (21 / 128 tokens)
Generating (22 / 128 tokens)
Generating (23 / 128 tokens)
Generating (24 / 128 tokens)
Generating (25 / 128 tokens)
Generating (26 / 128 tokens)
Generating (27 / 128 tokens)
Generating (28 / 128 tokens)
Generating (29 / 128 tokens)
Generating (30 / 128 tokens)
Generating (31 / 128 tokens)
Generating (32 / 128 tokens)
Generating (33 / 128 tokens)
Generating (34 / 128 tokens)
Generating (35 / 128 tokens)
Generating (36 / 128 tokens)
Generating (37 / 128 tokens)
Generating (38 / 128 tokens)
Generating (39 / 128 tokens)
Generating (40 / 128 tokens)
Generating (41 / 128 tokens)
Generating (42 / 128 tokens)
Generating (43 / 128 tokens)
Generating (44 / 128 tokens)
Generating (45 / 128 tokens)
Generating (46 / 128 tokens)
Generating (47 / 128 tokens)
Generating (48 / 128 tokens)
Generating (49 / 128 tokens)
Generating (50 / 128 tokens)
Generating (51 / 128 tokens)
Generating (52 / 128 tokens)
Generating (53 / 128 tokens)
Generating (54 / 128 tokens)
Generating (55 / 128 tokens)
Generating (56 / 128 tokens)
Generating (57 / 128 tokens)
Generating (58 / 128 tokens)
Generating (59 / 128 tokens)
Generating (60 / 128 tokens)
Generating (61 / 128 tokens)
Generating (62 / 128 tokens)
Generating (63 / 128 tokens)
Generating (64 / 128 tokens)
Generating (65 / 128 tokens)
Generating (66 / 128 tokens)
Generating (67 / 128 tokens)
Generating (68 / 128 tokens)
Generating (69 / 128 tokens)
Generating (70 / 128 tokens)
Generating (71 / 128 tokens)
Generating (72 / 128 tokens)
Generating (73 / 128 tokens)
Generating (74 / 128 tokens)
Generating (75 / 128 tokens)
Generating (76 / 128 tokens)
Generating (77 / 128 tokens)
Generating (78 / 128 tokens)
Generating (79 / 128 tokens)
Generating (80 / 128 tokens)
Generating (81 / 128 tokens)
Generating (82 / 128 tokens)
Generating (83 / 128 tokens)
Generating (84 / 128 tokens)
Generating (85 / 128 tokens)
Generating (86 / 128 tokens)
Generating (87 / 128 tokens)
Generating (88 / 128 tokens)
Generating (89 / 128 tokens)
Generating (90 / 128 tokens)
Generating (91 / 128 tokens)
Generating (92 / 128 tokens)
Generating (93 / 128 tokens)
Generating (94 / 128 tokens)
Generating (95 / 128 tokens)
Generating (96 / 128 tokens)
Generating (97 / 128 tokens)
Generating (98 / 128 tokens)
Generating (99 / 128 tokens)
Generating (100 / 128 tokens)
Generating (101 / 128 tokens)
Generating (102 / 128 tokens)
Generating (103 / 128 tokens)
Generating (104 / 128 tokens)
Generating (105 / 128 tokens)
Generating (106 / 128 tokens)
Generating (107 / 128 tokens)
Generating (108 / 128 tokens)
Generating (109 / 128 tokens)
Generating (110 / 128 tokens)
Generating (111 / 128 tokens)
Generating (112 / 128 tokens)
Generating (113 / 128 tokens)
Generating (114 / 128 tokens)
Generating (115 / 128 tokens)
Generating (116 / 128 tokens)
Generating (117 / 128 tokens)
Generating (118 / 128 tokens)
Generating (119 / 128 tokens)
Generating (120 / 128 tokens)
Generating (121 / 128 tokens)
Generating (122 / 128 tokens)
Generating (123 / 128 tokens)
Generating (124 / 128 tokens)
Generating (125 / 128 tokens)
Generating (126 / 128 tokens)
Generating (127 / 128 tokens)
Generating (128 / 128 tokens)
[12:40:02] CtxLimit:166/4096, Init:0.00s, Processed:1 in 0.02s (52.63T/s), Generated:128/128 in 2.76s (46.33T/s), Total:2.78s

Processing Prompt (0 / 1 tokens)
Processing Prompt (1 / 1 tokens)

Generating (1 / 128 tokens)
Generating (2 / 128 tokens)
Generating (3 / 128 tokens)
Generating (4 / 128 tokens)
Generating (5 / 128 tokens)
Generating (6 / 128 tokens)
Generating (7 / 128 tokens)
Generating (8 / 128 tokens)
Generating (9 / 128 tokens)
Generating (10 / 128 tokens)
Generating (11 / 128 tokens)
Generating (12 / 128 tokens)
Generating (13 / 128 tokens)
Generating (14 / 128 tokens)
Generating (15 / 128 tokens)
Generating (16 / 128 tokens)
Generating (17 / 128 tokens)
Generating (18 / 128 tokens)
Generating (19 / 128 tokens)
Generating (20 / 128 tokens)
Generating (21 / 128 tokens)
Generating (22 / 128 tokens)
Generating (23 / 128 tokens)
Generating (24 / 128 tokens)
Generating (25 / 128 tokens)
Generating (26 / 128 tokens)
Generating (27 / 128 tokens)
Generating (28 / 128 tokens)
Generating (29 / 128 tokens)
Generating (30 / 128 tokens)
Generating (31 / 128 tokens)
Generating (32 / 128 tokens)
Generating (33 / 128 tokens)
Generating (34 / 128 tokens)
Generating (35 / 128 tokens)
Generating (36 / 128 tokens)
Generating (37 / 128 tokens)
Generating (38 / 128 tokens)
Generating (39 / 128 tokens)
Generating (40 / 128 tokens)
Generating (41 / 128 tokens)
Generating (42 / 128 tokens)
Generating (43 / 128 tokens)
Generating (44 / 128 tokens)
Generating (45 / 128 tokens)
Generating (46 / 128 tokens)
Generating (47 / 128 tokens)
Generating (48 / 128 tokens)
Generating (49 / 128 tokens)
Generating (50 / 128 tokens)
Generating (51 / 128 tokens)
Generating (52 / 128 tokens)
Generating (53 / 128 tokens)
Generating (54 / 128 tokens)
Generating (55 / 128 tokens)
Generating (56 / 128 tokens)
Generating (57 / 128 tokens)
Generating (58 / 128 tokens)
Generating (59 / 128 tokens)
Generating (60 / 128 tokens)
Generating (61 / 128 tokens)
Generating (62 / 128 tokens)
Generating (63 / 128 tokens)
Generating (64 / 128 tokens)
Generating (65 / 128 tokens)
Generating (66 / 128 tokens)
Generating (67 / 128 tokens)
Generating (68 / 128 tokens)
Generating (69 / 128 tokens)
Generating (70 / 128 tokens)
Generating (71 / 128 tokens)
Generating (72 / 128 tokens)
Generating (73 / 128 tokens)
Generating (74 / 128 tokens)
Generating (75 / 128 tokens)
Generating (76 / 128 tokens)
Generating (77 / 128 tokens)
Generating (78 / 128 tokens)
Generating (79 / 128 tokens)
Generating (80 / 128 tokens)
Generating (81 / 128 tokens)
Generating (82 / 128 tokens)
Generating (83 / 128 tokens)
Generating (84 / 128 tokens)
Generating (85 / 128 tokens)
Generating (86 / 128 tokens)
Generating (87 / 128 tokens)
Generating (88 / 128 tokens)
Generating (89 / 128 tokens)
Generating (90 / 128 tokens)
Generating (91 / 128 tokens)
Generating (92 / 128 tokens)
Generating (93 / 128 tokens)
Generating (94 / 128 tokens)
Generating (95 / 128 tokens)
Generating (96 / 128 tokens)
Generating (97 / 128 tokens)
Generating (98 / 128 tokens)
Generating (99 / 128 tokens)
Generating (100 / 128 tokens)
Generating (101 / 128 tokens)
Generating (102 / 128 tokens)
Generating (103 / 128 tokens)
Generating (104 / 128 tokens)
Generating (105 / 128 tokens)
Generating (106 / 128 tokens)
Generating (107 / 128 tokens)
Generating (108 / 128 tokens)
Generating (109 / 128 tokens)
Generating (110 / 128 tokens)
Generating (111 / 128 tokens)
Generating (112 / 128 tokens)
Generating (113 / 128 tokens)
Generating (114 / 128 tokens)
Generating (115 / 128 tokens)
Generating (116 / 128 tokens)
Generating (117 / 128 tokens)
Generating (118 / 128 tokens)
Generating (119 / 128 tokens)
Generating (120 / 128 tokens)
Generating (121 / 128 tokens)
Generating (122 / 128 tokens)
Generating (123 / 128 tokens)
Generating (124 / 128 tokens)
Generating (125 / 128 tokens)
Generating (126 / 128 tokens)
Generating (127 / 128 tokens)
Generating (128 / 128 tokens)
[12:40:06] CtxLimit:166/4096, Init:0.00s, Processed:1 in 0.03s (31.25T/s), Generated:128/128 in 3.64s (35.19T/s), Total:3.67s

Processing Prompt (0 / 1650 tokens)
Processing Prompt (512 / 1650 tokens)
Processing Prompt (1024 / 1650 tokens)
Processing Prompt (1536 / 1650 tokens)
Processing Prompt (1650 / 1650 tokens)

Generating (1 / 32 tokens)
Generating (2 / 32 tokens)
Generating (3 / 32 tokens)
Generating (4 / 32 tokens)
Generating (5 / 32 tokens)
Generating (6 / 32 tokens)
Generating (7 / 32 tokens)
Generating (8 / 32 tokens)
Generating (9 / 32 tokens)
Generating (10 / 32 tokens)
Generating (11 / 32 tokens)
Generating (12 / 32 tokens)
Generating (13 / 32 tokens)
Generating (14 / 32 tokens)
Generating (15 / 32 tokens)
Generating (16 / 32 tokens)
Generating (17 / 32 tokens)
Generating (18 / 32 tokens)
Generating (19 / 32 tokens)
Generating (20 / 32 tokens)
Generating (21 / 32 tokens)
Generating (22 / 32 tokens)
Generating (23 / 32 tokens)
Generating (24 / 32 tokens)
Generating (25 / 32 tokens)
Generating (26 / 32 tokens)
Generating (27 / 32 tokens)
Generating (28 / 32 tokens)
Generating (29 / 32 tokens)
Generating (30 / 32 tokens)
Generating (31 / 32 tokens)
Generating (32 / 32 tokens)
[12:40:22] CtxLimit:1682/4096, Init:0.01s, Processed:1650 in 14.82s (111.31T/s), Generated:32/32 in 0.94s (34.15T/s), Total:15.77s
Input: {"max_length": 128, "temperature": 0.01, "top_k": 1, "rep_pen": 1.0, "seed": 42, "prompt": "Below is a short story about a lighthouse keeper who finds a strange machine on the beach. Write the next paragraph.\n\nThe storm had passed by morning. Mara walked down to the shore and"}

Output:  looked at the lighthouse. The lighthouse was still lit, but the light was dim. She had been there for a long time, and it was time to check the machine. She walked to the lighthouse and looked at the machine. It was a strange machine, but it was also very useful. It was a device that could help people find their way around the island. She had never seen anything like it before, and she was excited to see it in action.

The lighthouse keeper had been working for many years, and she had never seen anything like this before. She had been working for many years, and she had

Input: {"max_length": 128, "temperature": 0.01, "top_k": 1, "rep_pen": 1.0, "seed": 42, "prompt": "Below is a short story about a lighthouse keeper who finds a strange machine on the beach. Write the next paragraph.\n\nThe storm had passed by morning. Mara walked down to the shore and"}

Output:  looked at the lighthouse. She had never seen anything like it before. The lighthouse was a beacon of hope, a beacon of light that guided ships to safety. She had been a lighthouse keeper for many years, and she had never seen anything like it before. She had never seen anything like it before.

The lighthouse was a beacon of hope, a beacon of light that guided ships to safety. She had been a lighthouse keeper for many years, and she had never seen anything like it before. She had never seen anything like it before.

The storm had passed by morning. Mara walked down to the shore and looked

Input: {"max_length": 128, "temperature": 0.01, "top_k": 1, "rep_pen": 1.0, "seed": 42, "prompt": "Below is a short story about a lighthouse keeper who finds a strange machine on the beach. Write the next paragraph.\n\nThe storm had passed by morning. Mara walked down to the shore and"}

Output:  looked at the lighthouse. She had never seen anything like it before. The lighthouse was a beacon of hope, a beacon of light that guided ships to safety. She had been a lighthouse keeper for many years, and she had never seen anything like it before. She had never seen anything like it before.

The lighthouse was a beacon of hope, a beacon of light that guided ships to safety. She had been a lighthouse keeper for many years, and she had never seen anything like it before. She had never seen anything like it before.

The storm had passed by morning. Mara walked down to the shore and looked

Input: {"max_length": 32, "temperature": 0.01, "top_k": 1, "rep_pen": 1.0, "seed": 42, "prompt": "The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge."}

Output:  The quick brown fox jumps over the lazy dog near the old stone bridge. The quick brown fox jumps over the lazy dog near the old stone bridge. The quick


Processing Prompt (0 / 1 tokens)
Processing Prompt (1 / 1 tokens)

Generating (1 / 32 tokens)
Generating (2 / 32 tokens)
Generating (3 / 32 tokens)
Generating (4 / 32 tokens)
Generating (5 / 32 tokens)
Generating (6 / 32 tokens)
Generating (7 / 32 tokens)
Generating (8 / 32 tokens)
Generating (9 / 32 tokens)
Generating (10 / 32 tokens)
Generating (11 / 32 tokens)
Generating (12 / 32 tokens)
Generating (13 / 32 tokens)
Generating (14 / 32 tokens)
Generating (15 / 32 tokens)
Generating (16 / 32 tokens)
Generating (17 / 32 tokens)
Generating (18 / 32 tokens)
Generating (19 / 32 tokens)
Generating (20 / 32 tokens)
Generating (21 / 32 tokens)
Generating (22 / 32 tokens)
Generating (23 / 32 tokens)
Generating (24 / 32 tokens)
Generating (25 / 32 tokens)
Generating (26 / 32 tokens)
Generating (27 / 32 tokens)
Generating (28 / 32 tokens)
Generating (29 / 32 tokens)
Generating (30 / 32 tokens)
Generating (31 / 32 tokens)
Generating (32 / 32 tokens)
[12:40:23] CtxLimit:1682/4096, Init:0.00s, Processed:1 in 0.03s (34.48T/s), Generated:32/32 in 0.93s (34.26T/s), Total:0.97s