Logging in PySpark

Logging while writing pyspark applications is a common issue. I’ve come across many questions on Stack overflow where beginner Spark programmers are worried that they have tried logging using some means and it didn’t work. This short post will help you configure your pyspark applications with log4j. Know that this is only one of the many…