Hopsworks as feature registry

#1

Can Hopsworks be integrated purely as feature registry component without altering the existing feature store (BigQuery and BigTable)? Does it have integration with BigTable at all? It’s not on the list of connectors but for example DataProc is mentioned here on the picture which makes me think that other integrations are possible?

Thank you!

#2

Hi @nastia,

You can use Hopsworks as a feature registry on top of BigQuery by leveraging external feature groups (External Feature Groups - Hopsworks Documentation)

Hopsworks does not support BigTable as a source for external feature groups, however you can define a feature pipeline in Pandas/Spark/Beam to read data from BigTable, compute features and register them with Hopsworks.

Regards,
Fabio

#3

Hi Fabio,

Thanks for your response!
Could you please provide more information on how to implement feature registry only without leveraging other components? Will it require some custom setup?

I’m looking for solution that would help unify transformations for training and inference steps. Is it possible to define names and transformations in a DSL format in Hopsworks?