Monday 26 March 2018

Check Query Execution Time in spark

def time[A](f: => A) = {
  val s = System.nanoTime
  val ret = f
  println("time: " + (System.nanoTime - s) / 1e9 + " seconds")
  ret
  }
val test = time('statement')


val test = time(spark.sql("SELECT * from mmc_push_event.push_devices"))

No comments:

Post a Comment

Revert last commit or second last and more....

 Git revert commit_id -m 1 this command willl revert last commit  Git revert commit_id -m 2 this command will revert second commit with same...