图计算
作者: 时海
最大连通图ConnectedComponents

最大连通图ConnectedComponents会将输入的图作为无向图处理。

示例:

    val spark = SparkSession
      .builder()
      .appName("k6k4Test")
      .master("local[4]")
      .getOrCreate()

    val sqlContext = spark.sqlContext

    // 创建顶点DataFrame
    val v = sqlContext.createDataFrame(List(
      ("a", "Alice", 34),
      ("b", "Bob", 36),
      ("c", "Charlie", 30),
      ("d", "David", 29),
      ("e", "Esther", 32),
      ("f", "Fanny", 36),
      ("g", "Alice", 60)
    )).toDF("id", "name", "age")

    // 创建边DataFrame
    val e = sqlContext.createDataFrame(List(
      ("a", "b", "friend"),
      ("b", "c", "follow"),
      ("c", "b", "follow"),
      ("f", "c", "follow"),
      ("e", "f", "follow"),
      ("e", "d", "friend"),
      ("d", "a", "friend"),
      ("a", "e", "friend")
    )).toDF("src", "dst", "relationship")



    val g = GraphFrame(v, e)

    val result = g.connectedComponents.setCheckpointInterval(0).run();

    result.printSchema()

    result.show(false)

    //结果写入hive表
    //result.write.saveAsTable("hiveTable")

    spark.stop()


结果输出:

 |-- id: string (nullable = true)
 |-- name: string (nullable = true)
 |-- age: integer (nullable = false)
 |-- component: long (nullable = true)

+---+-------+---+------------+
|id |name   |age|component   |
+---+-------+---+------------+
|a  |Alice  |34 |412316860416|
|b  |Bob    |36 |412316860416|
|c  |Charlie|30 |412316860416|
|d  |David  |29 |412316860416|
|e  |Esther |32 |412316860416|
|f  |Fanny  |36 |412316860416|
|g  |Alice  |60 |146028888064|
+---+-------+---+------------+






标签: val、sqlcontext、friend、alice、follow
一个创业中的苦逼程序员
  • 回复
隐藏