Saturday, April 11, 2020

Set log level in spark-shell in spark REPL Environment

Follow the below steps .

1. Add the Logger 


import org.apache.log4j.Logger
2. Add the Level
import org.apache.log4j.Level
3.Set the Log Level (Here we are setting the Level to "org" package)
Logger.getLogger("org").setLevel(Level.INFO)  (We can add any Level Like  DEBUG,ERROR,TRACE,OFF )



No comments:

Post a Comment

Recent Post

Databricks Delta table merge Example

here's some sample code that demonstrates a merge operation on a Delta table using PySpark:   from pyspark.sql import SparkSession # cre...