apache spark - Find out the partition no/id -


is there way(a method) in spark find out parition id/no

take example here

val input1 = sc.parallelize(list(8, 9, 10), 3)  val res = input1.reduce{ (x, y) => println("inside partiton " + ???)                                 x + y)} 

i put code in ??? print partition id / no

indeed, mapparitionswithindex give iterator & partition index. (this isn't same reduce of course, combine result of aggregate).


Comments

Popular posts from this blog

toolbar - How to add link to user registration inside toobar in admin joomla 3 custom component -

linux - disk space limitation when creating war file -

How to provide Authorization & Authentication using Asp.net, C#? -