A simple PySpark example to show how to consume Kafka stream (given Kafka tutorial).

Let’s see how to play K8s in MacOs using MniKube. Some of the topics are very basic such as How to create a namespace and pod in it. Shelling to the pod and after delete pod and the namespace. However, this is written to address the concepts such as configMap, secrets, resource sharing and Helm charts.

As I understood, RegExs are very useful for general work. Most of the following regular expressions (RegEx)s can be run on the macOS terminal, where you can get the great value of command line tools that have no value without RegExs (grep, sed and so on). In addition, I’ve used some popular tools to explain complex operations later in the document, which have been referenced under the footnotes.

PySpark date in string to date type conversion example. How you can use python sql functions like datediff to calculate the differences in days.

Here python list, tulple basic operations are discussed.