📝 Archived :https://ojitha.blogspot.com.au

Multi-Token Prediction on OJAI

May 15, 2026
Overview:
The provided source outlines the implementation of Multi-Token Prediction (MTP) to accelerate AI inference on the OJAI hardware platform, which features an AMD Ryzen AI 9 processor. This technical guide explains how speculative decoding bypasses memory bandwidth bottlenecks by using an auxiliary head to propose multiple tokens simultaneously for verification by the main model. It details necessary system configurations, such as BIOS updates and memory allocator adjustments, required to optimise the Radeon 890M iGPU and XDNA 2 NPU. The text further distinguishes between different software stacks, noting that while llama.cpp handles MTP tasks on the graphics processor, FastFlowLM targets the NPU for energy-efficient processing. Finally, it describes how the Lemonade orchestration server manages these various backends to provide a unified, high-performance local AI environment.
More…

Google Gemma 4 MoE (26B) on AMD Ryzen AI

May 10, 2026
Overview:
This technical log documents the installation and optimisation of the Google Gemma 4 Mixture-of-Experts (MoE) model on the MINISFORUM AI X1 Pro, a mini PC featuring the AMD Ryzen AI 9 HX 470 processor. The report details the challenges of running a large 26-billion-parameter model on a consumer-grade Unified Memory Architecture, focusing on critical RAM allocation and BIOS UMA adjustments. It explains how to resolve memory-mapping failures and hardware-specific OOM errors by bypassing standard Linux kernel overcommit limits and fine-tuning the vLLM and ROCm software stack. Performance comparisons highlight that while Ollama offers higher speeds for individual users, the vLLM backend provides superior efficiency for multi-user API environments. Ultimately, the guide provides a comprehensive resolution matrix and a definitive Docker configuration to achieve stable inference on this specific RDNA 3.5 hardware.
More…

A Learning Guide to the blog-post-excerpt Claude Skill

May 9, 2026
SKILL.mdSVGJekyllYAMLTLDR · excerpt

A practical, end-to-end walkthrough of the project-scoped blog-post-excerpt Claude skill that lives in this repository under .claude/skills/. The guide follows the chapter structure of Anthropic's Complete Guide to Building Skills for Claude and applies each principle to the working skill that automates Jekyll front-matter excerpt generation. Learn how progressive disclosure splits across SKILL.md, its body, and the references folder; how to craft a triggering description; how to inject an inline SVG illustration plus a TLDR into a single line of YAML; and how a final verification step keeps the front matter from breaking on every run.

More…

Running Gemma 4 E4B on the AMD ROCm

May 5, 2026
Gemma 4vLLM · ROCmRyzen AI · NPU

This deep-dive shows how to run the Google DeepMind Gemma 4 E4B model — a 4.5B-effective dense network with Per-Layer Embeddings — on a Minisforum AI X1 Pro driven by the AMD Ryzen AI 9 HX 470, Radeon 890M iGPU and XDNA 2 NPU. It walks the verified vLLM Docker recipe on ROCm 7.2, decomposes the hybrid sliding-window plus global attention that makes a 128K context fit on a 16GB-class memory budget, and shows where MIGraphX can offload an ONNX sidecar to the NPU. The result is a layered guide from architecture math through tuning, quantisation, and benchmarking.

More…

LLM Wiki

April 8, 2026
Scala Functors
Andrej Karpathy Obsidian method refers to his publicly shared system for using LLMs to build and maintain personal knowledge bases as interlinked markdown wikis, all viewed and navigated through Obsidian. The system, which he calls LLM Wiki or LLM Knowledge Bases.
More…