Spark dataframe stats mean

df.describe().rdd.map{ case r : Row => (r.getAs[String](“summary”),r) }.filter(_._1 == “mean”).map(_._2).first().toSeq.drop(1).map(x => x.toString().toDouble)

This entry was posted in spark. Bookmark the permalink.

Leave a comment