Hi,
We are evaluating HopsWorks Feature Store for our own project. It is awesome and pretty much meets most of our requirements. One concern we have is seems a new table is created every time when user is adding a new feature set. We do see some perf issues when we created too many tables in our PostgreSQL cluster. Is there any performance concern with too many feature set tables in MySQL?
Thanks!
Hi. From the mysql cluster documentation:
" Number of database objects. The maximum number of all NDB database objects in a single NDB Cluster—including databases, tables, and indexes—is limited to 20320."
There shouldn’t be any performance degradation with an increased number of tables. You may need to configure/tune MySQL Cluster for your workload. Logical Clocks can help out with that.
Thanks Jim! I’ll discuss with my team and get back to you if we need help from hopsworks. We have other concerns but I’ll post in a separate thread.
We have a new API coming out very soon, FYI. Before the end of June.
That’s COOL! Will keep an eye on that. Is there a slack channel fro Hopsworks?