📝 Research :https://ojitha.blogspot.com.au for my lengthy articles.
Ontology Evals for LLMs
My previous work on ontologies defined in RDF Structured Data mining1 and Apache Jena2 provide formal, structured representations of domain knowledge that can be harnessed as evaluation frameworks (Evals) for assessing outputs of LLMs. Using the well-known Pizza ontology3 as a running example, illustrate how domain-specific ontologies can guide the evaluation of LLM-generated content such as recipe emails. The framework emphasises mapping unstructured LLM outputs into ontology-aligned structured data, applying reasoning engines to verify factual and logical coherence, and deriving quantitative and qualitative evaluation metrics.
LlamaIndex RAG for Science Community
LlamaIndex is a comprehensive data orchestration framework designed to connect large language models with private data sources through Retrieval-Augmented Generation (RAG) architecture. The framework operates through two main stages: indexing, where vector embeddings are created from documents, and querying, where relevant information is retrieved and synthesized. LlamaIndex supports multiple chat engines including condense question, context, and condense-plus-context modes for enhanced conversational AI applications. Quality evaluation is crucial for RAG performance, utilizing metrics like Mean Reciprocal Rank (MRR) and Hit Rate to assess retrieval accuracy. The framework includes faithfulness and relevance evaluators to measure response quality, making it essential for building reliable AI applications that require domain-specific knowledge integration with seamless LLM integration capabilities.
Structured Model for AI
Python type annotations are essential for AI agent development with frameworks like LangGraph and AutoGen, enabling automatic schema generation, runtime validation, and seamless LLM integration. This comprehensive guide explores modern Python typing features including TypedDict for state management, PyDantic models for data validation, and parameterized generics. It covers practical implementations using Python 3.12+ syntax, structured output with ChatGPT, and real-world examples demonstrating tool registration and function calling. The tutorial showcases how type hints transform from optional documentation to architectural foundations in AI development, providing automatic error handling, JSON schema generation, and reliable data flow between agents for robust conversational AI systems.
AI Agent testing with LangGraph
The comprehensive testing framework demonstrates local validation strategies using detailed conversation history analysis, where each message type, tool call, and state transition is meticulously tracked and displayed for debugging purposes. The implementation showcases systematic test coverage across multiple agent capabilities including weather retrieval, mathematical calculations, web search, and state management, ensuring robust validation of tool selection logic and parameter handling. This testing methodology ensures that AI agents maintain transparency, reliability, and predictable behavior while scaling from development prototypes to production-ready systems with confidence in their decision-making processes.
Steup LangGraph Studio for AI Agent
Complete LangGraph Studio setup guide for AI agent development featuring visual debugging, real-time execution monitoring, and hot reloading capabilities. Step-by-step tutorial includes working code examples for building production-ready AI agents with tool integration, state management, conditional routing, and seamless LangSmith tracing for optimized multi-step reasoning processes.
✅ Complete UV setup - from installation to project configuration
✅ Working agent example - with tools, state management, and proper routing
✅ Multiple testing methods - Studio UI, Python scripts, and direct API calls