I read this article: Configuring Hops YARN quota system — Documentation 2.2 documentation. But I didn’t find where to configure these parameters.
I deployed the Hopsworks 2.0 on the local servers. Any comments will be much appreciated.
Hi Freeman. When we moved over to the cloud version of Hopsworks, we decided to turn off the YARN quota extension that we had added. It shouldn’t be in the docs anymore - we should remove it. The code is still there, if you think it is useful, you could build a fork of Hopsworks with the feature re-enabled.
Note, there are a few things we do to simplify YARN for Hopsworks Jobs. We assume a single YARN queue ‘default’. We also now manage on the cloud platform automated adding/decommissioning of nodemanagers, when you add/remove workers in the cloud platform. We also have auto-scaling coming soon on the cloud platform.
Hi Jim,
Thank you for your reply.
I’m sorry to hear that this feature has been turned off, but the good news is that the source code is still there. I’m trying to figure out how the Hopsworks manages the server resources such as CPU, GPU, memory and so on, I think it’s an important function for an AI platform. And I think that the Hopsworks community edition is a fairly good open source software, I hope the future releases will continue to retain the necessary features. At the same time, I hope the cloud platform can be successful.
hi man, I also want to install hopsworks in my vm using http_proxy. How to dal with the Invalid yaml file, since Troubleshooting — Documentation 2.0.0 documentation troubleshooting do not work.