📝 Research :https://ojitha.blogspot.com.au for my lengthy articles.
Spark to create a table in AWS Redshift
In this post, Spark reads the data from a CSV file to a DateFrame and saves that DataFrame as a Redshift table.
Spark Kafka Docker Configuration
This is the continuation of the [Spark Streaming Basics](/apache%20spark/2023/06/09/Spark-Streaming-part-1.html). I explained the basic stream example, which runs only on one AWS Glue container. The stream producer was Netcat, and the sink was a text file. In this post, the stream producer is still Netcat, but the sink is Kafka. Both Kafka and Spark running on Docker containers.
Spark Streaming Basics
This is a very basic example created to explain Spark streaming. Spark run on the AWS Glue container locally.
Introduction to Lambda Calculus
This is a short description of lambda calculus. Lambda calculus is the smallest programming language that is capable of variable substitution and a single function definition scheme. Haskell is the functional programming language based on lambda calculus, which I will explore. I already explained how to use VSCode for Haskell Development to support the code listed here.
Python Parameter passing
Discuss the most possible ways of passing parameters in the python functions.