Create featuregroup with hudi

Feature Store
#1

Hi, I’ve been trying to create hudi offline feature group for point-in-time query. But there is this error:

image
image1190×932 144 KB

Seem like there is some “ts” columns being added. Am I doing this wrong?

#2

Hi,

it is not obvious, but you either need to provide a partition key or turn off partitioning by setting the following property in the Hudi arguments to an empty string:

import org.apache.hudi.DataSourceWriteOptions

val hudiArgs = Map(DataSourceWriteOptions.PARTITIONPATH_FIELD_OPT_KEY -> "")
(Hops.createFeaturegroup("hudi_test")
    .setHudi(true)
    .setDataframe(someDf)
    .setHudiArgs(hudiArgs.asJava)
    .setPrimaryKey(List("number_f").asJava).write())

Or provide a partition key:

(Hops.createFeaturegroup("hudi_test")
    .setHudi(true)
    .setDataframe(someDf)
    .setPartitionBy(List("word_f").asJava)
    .setPrimaryKey(List("number_f").asJava).write())

Append:
You can find more info here on the Hudi documentation in the Datasource Writer section.

1 Like