Error when creating hopsworks cluster on AWS

Hopsworks.ai
#1

We’re trying to set up Hopsworks on our AWS account. We followed the documentation on this page link, but when we reach the point to create the cluster we get the following error:

We could not fetch the role arn for the provided instance profile. Make sure that the instance profile and the role are correctly attached. This can for example happen if the user creating the instance profile did not have the iam:AddRoleToInstanceProfile permission.
(error code: 400)

As far as we can tell from the AWS console, the role and policy we created when following the steps in the guide are correctly attached. What are we missing?

#2

Hello,

Can you check from using the following aws cli command if the instance profile you have created has an attached role to it.
aws iam get-instance-profile --instance-profile-name <YOUR_INSTANCE_PROFILE_NAME>

You should get a similar output. Also, regarding the documentation, the link you are sharing is obsolete here is a link to the latest documentation

{
    "InstanceProfile": {
        "Path": "/",
        "InstanceProfileName": "test",
        "InstanceProfileId": "xxx",
        "Arn": "arn:aws:iam::xxxxxxx:instance-profile/test",
        "CreateDate": "2020-05-12T08:44:41+00:00",
        "Roles": [
            {
                "Path": "/",
                "RoleName": "test",
                "RoleId": "xxxxx",
                "Arn": "arn:aws:iam::xxxxxx:role/test",
                "CreateDate": "2020-05-12T08:44:41+00:00",
                "AssumeRolePolicyDocument": {
                    "Version": "2012-10-17",
                    "Statement": [
                        {
                            "Effect": "Allow",
                            "Principal": {
                                "Service": "ec2.amazonaws.com"
                            },
                            "Action": "sts:AssumeRole"
                        }
                    ]
                }
            }
        ],
        "Tags": []
    }
}

Regards,
Mahmoud

#3

Hi Mahmoud,

It seems indeed the issue was that we were following the obsolete documentation. We started from scratch with the documentation you shared and were able to get a cluster up and running without further issue.

Thanks for the help!