📝 Research :https://ojitha.blogspot.com.au
for my lengthy articles.
Lua filters for Pandoc
Lua filter used in Pandoc 3.6.3. This blog has solutions for:
- Creating Glossary for ePub ver 3 book
- GitHub style alerts
AWS PITR Explained
PITR stands for Point-in-Time Recovery, which is a feature offered by several AWS services to provide continuous data protection and the ability to restore data to a specific point in time.
Spark - create database and table
This is a short note to create a Hive meta store using Spark 3.3.1.
Semantic search with ELSER in Elasticsearch
Elastic Learned Sparse EncodeR(ELSER) is a retrieval model trained by Elastic that enables you to perform semantic search to retrieve more relevant search results.
- install ELSER v2: Only once (DevOPs will do for your)
- Create source index where you can insert all your documents
- Create target index
- Create ingestion pipeline
- Reindex process to create embeddings
- Ready to do semantic search using text expansion queries
I created this blog post on docker to demonstrate Linux-optimised ELSER v2. The Elasticsearch version is 8.11.1.
Kafka PySpark streaming example
The diagram shows that the Kafka producer reads from Wikimedia and writes to the Kafka topic. Then Kafka Spark consumer pulls the data from the Kafka topic and writes the steam batches to disk.