📝 Research :https://ojitha.blogspot.com.au for my lengthy articles.
Elastic Search Introduction
Learn Elasticsearch from zero to hero with this comprehensive guide covering installation, CRUD operations, mapping, and advanced search techniques.
Kafka PySpark streaming example
The diagram shows that the Kafka producer reads from Wikimedia and writes to the Kafka topic. Then Kafka Spark consumer pulls the data from the Kafka topic and writes the steam batches to disk.
Terraform For each iteration
This is to explain Terraform for each looping technique. In this example, 3 buckets are created to demonstrate the looping idea.
Spark to create a table in AWS Redshift
In this post, Spark reads the data from a CSV file to a DateFrame and saves that DataFrame as a Redshift table.
Spark Kafka Docker Configuration
This is the continuation of the [Spark Streaming Basics](/apache%20spark/2023/06/09/Spark-Streaming-part-1.html). I explained the basic stream example, which runs only on one AWS Glue container. The stream producer was Netcat, and the sink was a text file. In this post, the stream producer is still Netcat, but the sink is Kafka. Both Kafka and Spark running on Docker containers.