📝 Research :https://ojitha.blogspot.com.au for my lengthy articles.
Encrypting in Dockerfile and Decrypting in Python
This approach allows you to encrypt sensitive data (like a database password) during Docker build and decrypt it safely at runtime in your Python application.
Spark - create database and table
This is a short note to create a Hive meta store using Spark 3.3.1.
Semantic search with ELSER in Elasticsearch
Elastic Learned Sparse EncodeR(ELSER) is a retrieval model trained by Elastic that enables you to perform semantic search to retrieve more relevant search results.
👉 I created this blog post on docker to demonstrate Linux-optimised ELSER v2. The Elasticsearch version is 8.11.1.
Elastic Search Introduction
Learn Elasticsearch from zero to hero with this comprehensive guide covering installation, CRUD operations, mapping, and advanced search techniques.
Kafka PySpark streaming example
The diagram shows that the Kafka producer reads from Wikimedia and writes to the Kafka topic. Then Kafka Spark consumer pulls the data from the Kafka topic and writes the steam batches to disk.