A simple and easy Spark standalone program for beginners.
A simple and easy Spark standalone program for beginners.... Courtsey: Hariprasad Bhaskaran Aim : To demonstrate writing a simple spark scala program use sbt to build it create dependencies for spark-core and spark-sql in build.sbt create a final jar that can be submitted using spark-submit... Program code below: MyAdd2.scala ================= import org.apache.spark.SparkContext import org.apache.spark.SparkConf import org.apache.spark.sql._ import org.apache.spark.sql.hive.HiveContext import org.apache.spark.sql.hive._ import org.apache.spark.sql.SQLContext import org.apache.spark.sql.SQLContext._ object MyAdd2 { def main(args: Array[String] ) { val conf = new SparkConf().setAppName("t1").setMaster("local") val sc2 = new SparkContext(conf) val sqlContext = new org.apache.spark.sql.hive.HiveContext(sc2) import sqlContext.implicits._ val mylist = sc2.parallelize(List( 1 ,2,3,4,5,6,7,8,9,100) ) val mylistDF = mylist.map( x =&