Azure installation hangs after 'Running task: conda::default'

phil · July 1, 2020, 4:11pm

I am attempting to install Hopsworks on Microsoft Azure using two VMs, following the instructions shown here: https://hopsworks.readthedocs.io/en/latest/getting_started/installation_guide/platforms/cloud.html#quickstart-on-azure

When I run the install program, it runs several install tasks but hangs after ‘Running task: conda::default’. Could someone please help me get past this?

Full log:

Running command from /home/AzureUser/karamel-0.6:

setsid ./bin/karamel -headless -launch ../cluster-defns/hopsworks-installer-active.
yml  > ../installation.log 2>&1 &

***********************************************************************************
************************

Installation has started, but may take 1 hour or more..........

The Karamel installer UI will soon start at:  http://172.16.0.4:9090/index.html
Note: port 9090 must be open for external traffic and Karamel will shutdown when in
stallation finishes.

=====================================================================

You can view the installation logs with this command:

tail -f installation.log

***********************************************************************************
************************
AzureUser@hw1:~$ tail -f installation.log
    PUT     /api/occi/validateCredentials (se.kth.karamel.webservice.calls.occi.Val
idateOcciCredentials)

INFO  [2020-07-01 14:21:59,096] org.eclipse.jetty.server.handler.ContextHandler: St
arted i.d.j.MutableServletContextHandler@437e951d{/,null,AVAILABLE}
INFO  [2020-07-01 14:21:59,098] io.dropwizard.setup.AdminEnvironment: tasks =

    POST    /tasks/gc (io.dropwizard.servlets.tasks.GarbageCollectionTask)

INFO  [2020-07-01 14:21:59,115] org.eclipse.jetty.server.handler.ContextHandler: St
arted i.d.j.MutableServletContextHandler@3be8821f{/admin,null,AVAILABLE}
INFO  [2020-07-01 14:21:59,134] org.eclipse.jetty.server.ServerConnector: Started k
aramel-core@d8305c2{HTTP/1.1}{0.0.0.0:9090}
INFO  [2020-07-01 14:22:01,351] se.kth.karamel.client.api.CookbookCacheIml: 0-level
 cookbooks for Hops is 1
INFO  [2020-07-01 14:22:10,661] se.kth.karamel.client.api.CookbookCacheIml: 1-level
 cookbooks for Hops is 22
INFO  [2020-07-01 14:22:11,438] se.kth.karamel.client.api.CookbookCacheIml: 2-level cookbooks for Hops is 14
INFO  [2020-07-01 14:22:11,443] se.kth.karamel.client.api.CookbookCacheIml: 0-level cookbooks for Hops is 1
INFO  [2020-07-01 14:22:11,452] se.kth.karamel.client.api.CookbookCacheIml: 1-level cookbooks for Hops is 20
INFO  [2020-07-01 14:22:11,455] se.kth.karamel.client.api.CookbookCacheIml: 2-level cookbooks for Hops is 14
INFO  [2020-07-01 14:22:11,469] se.kth.karamel.client.api.CookbookCacheIml: 0-level cookbooks for Hops is 1
INFO  [2020-07-01 14:22:11,469] se.kth.karamel.client.api.CookbookCacheIml: 1-level cookbooks for Hops is 20
INFO  [2020-07-01 14:22:11,470] se.kth.karamel.client.api.CookbookCacheIml: 2-level cookbooks for Hops is 14
INFO  [2020-07-01 14:22:20,144] se.kth.karamel.client.api.CookbookCacheIml: 0-level cookbooks for Hops is 21
INFO  [2020-07-01 14:22:20,146] se.kth.karamel.client.api.CookbookCacheIml: 1-level cookbooks for Hops is 20
INFO  [2020-07-01 14:22:20,198] se.kth.karamel.client.api.CookbookCacheIml: 0-level cookbooks for Hops is 1
INFO  [2020-07-01 14:22:20,198] se.kth.karamel.client.api.CookbookCacheIml: 1-level cookbooks for Hops is 20
INFO  [2020-07-01 14:22:20,199] se.kth.karamel.client.api.CookbookCacheIml: 2-level cookbooks for Hops is 14
INFO  [2020-07-01 14:22:20,213] se.kth.karamel.client.api.CookbookCacheIml: 0-level cookbooks for Hops is 1
INFO  [2020-07-01 14:22:20,213] se.kth.karamel.client.api.CookbookCacheIml: 1-level cookbooks for Hops is 20
INFO  [2020-07-01 14:22:20,214] se.kth.karamel.client.api.CookbookCacheIml: 2-level cookbooks for Hops is 14
INFO  [2020-07-01 14:22:20,220] se.kth.karamel.client.api.CookbookCacheIml: 0-level cookbooks for Hops is 1
INFO  [2020-07-01 14:22:20,221] se.kth.karamel.client.api.CookbookCacheIml: 1-level cookbooks for Hops is 20
INFO  [2020-07-01 14:22:20,221] se.kth.karamel.client.api.CookbookCacheIml: 2-level cookbooks for Hops is 14
INFO  [2020-07-01 14:22:24,095] se.kth.karamel.client.api.CookbookCacheIml: 0-level cookbooks for Hops is 21
INFO  [2020-07-01 14:22:24,097] se.kth.karamel.client.api.CookbookCacheIml: 1-level cookbooks for Hops is 20
INFO  [2020-07-01 14:22:24,131] se.kth.karamel.client.api.CookbookCacheIml: 0-level cookbooks for Hops is 1
INFO  [2020-07-01 14:22:24,132] se.kth.karamel.client.api.CookbookCacheIml: 1-level cookbooks for Hops is 20
INFO  [2020-07-01 14:22:24,132] se.kth.karamel.client.api.CookbookCacheIml: 2-level cookbooks for Hops is 14
INFO  [2020-07-01 14:22:24,134] se.kth.karamel.client.api.CookbookCacheIml: 0-level cookbooks for Hops is 1
INFO  [2020-07-01 14:22:24,144] se.kth.karamel.client.api.CookbookCacheIml: 1-level cookbooks for Hops is 20
INFO  [2020-07-01 14:22:24,148] se.kth.karamel.client.api.CookbookCacheIml: 2-level cookbooks for Hops is 14
INFO  [2020-07-01 14:22:24,151] se.kth.karamel.client.api.CookbookCacheIml: 0-level cookbooks for Hops is 1
INFO  [2020-07-01 14:22:24,151] se.kth.karamel.client.api.CookbookCacheIml: 1-level cookbooks for Hops is 20
INFO  [2020-07-01 14:22:24,155] se.kth.karamel.client.api.CookbookCacheIml: 2-level cookbooks for Hops is 14
INFO  [2020-07-01 14:22:24,186] se.kth.karamel.client.api.CookbookCacheIml: 0-level cookbooks for Hops is 21
INFO  [2020-07-01 14:22:24,198] se.kth.karamel.client.api.CookbookCacheIml: 1-level cookbooks for Hops is 20
INFO  [2020-07-01 14:22:24,225] se.kth.karamel.backend.launcher.baremetal.BaremetalLauncher: Public-key='/home/AzureUser/.ssh/id_rsa.pub'
INFO  [2020-07-01 14:22:24,226] se.kth.karamel.backend.launcher.baremetal.BaremetalLauncher: Private-key='/home/AzureUser/.ssh/id_rsa'
INFO  [2020-07-01 14:22:24,226] se.kth.karamel.backend.ClusterManager: Cluster-Manager started for 'Hops' d'-'
INFO  [2020-07-01 14:22:24,233] se.kth.karamel.backend.machines.MachinesMonitor: Machines-Monitor started for 'Hops' d'-'
INFO  [2020-07-01 14:22:24,238] se.kth.karamel.backend.ClusterManager: Going to serve 'LAUNCH_CLUSTER'
INFO  [2020-07-01 14:22:24,252] se.kth.karamel.backend.ClusterManager: Prelaunch Cleaning 'Hops' ...
INFO  [2020-07-01 14:22:24,252] se.kth.karamel.backend.ClusterManager: \o/\o/\o/\o/\o/'Hops' PRECLEANED \o/\o/\o/\o/\o/
INFO  [2020-07-01 14:22:24,256] se.kth.karamel.backend.ClusterManager: Forking groups 'Hops' ...
INFO  [2020-07-01 14:22:24,257] se.kth.karamel.backend.ClusterManager: \o/\o/\o/\o/\o/'Hops' GROUPS_FORKED \o/\o/\o/\o/\o/
INFO  [2020-07-01 14:22:24,257] se.kth.karamel.backend.ClusterManager: Launching 'Hops' ...
INFO  [2020-07-01 14:22:24,257] se.kth.karamel.backend.ClusterManager: groups '[se.kth.karamel.backend.running.model.GroupRuntime@fcfafc4, se.kth.karamel.backend.running.model.GroupRuntime@598c7aac]'
INFO  [2020-07-01 14:22:24,257] se.kth.karamel.backend.ClusterManager: Gogo
INFO  [2020-07-01 14:22:24,257] se.kth.karamel.backend.ClusterManager: Using provider 'se.kth.karamel.common.clusterdef.Baremetal@357afc96'
INFO  [2020-07-01 14:22:24,257] se.kth.karamel.backend.ClusterManager: Using launcher 'se.kth.karamel.backend.launcher.baremetal.BaremetalLauncher@3fd8478b'
INFO  [2020-07-01 14:22:24,257] se.kth.karamel.backend.ClusterManager: Using launcher 'se.kth.karamel.backend.launcher.baremetal.BaremetalLauncher@3fd8478b'
INFO  [2020-07-01 14:22:24,693] se.kth.karamel.backend.ClusterManager: Gogo
INFO  [2020-07-01 14:22:24,693] se.kth.karamel.backend.ClusterManager: Using provider 'se.kth.karamel.common.clusterdef.Baremetal@4c029135'
INFO  [2020-07-01 14:22:24,693] se.kth.karamel.backend.ClusterManager: Using launcher 'se.kth.karamel.backend.launcher.baremetal.BaremetalLauncher@3fd8478b'
INFO  [2020-07-01 14:22:24,694] se.kth.karamel.backend.ClusterManager: Using launcher 'se.kth.karamel.backend.launcher.baremetal.BaremetalLauncher@3fd8478b'
INFO  [2020-07-01 14:22:24,694] se.kth.karamel.backend.ClusterManager: \o/\o/\o/\o/\o/'Hops' MACHINES_FORKED \o/\o/\o/\o/\o/
INFO  [2020-07-01 14:22:24,694] se.kth.karamel.backend.ClusterManager: Going to serve 'SUBMIT_INSTALL_DAG'
INFO  [2020-07-01 14:22:24,694] se.kth.karamel.backend.ClusterManager: Running theDAG for 'Hops' ...
INFO  [2020-07-01 14:22:24,818] se.kth.karamel.client.api.CookbookCacheIml: 0-level cookbooks for Hops is 21
INFO  [2020-07-01 14:22:24,825] se.kth.karamel.client.api.CookbookCacheIml: 1-level cookbooks for Hops is 20
INFO  [2020-07-01 14:22:34,299] net.schmizz.sshj.transport.random.BouncyCastleRandom: Generating random seed from SecureRandom.
INFO  [2020-07-01 14:22:34,464] se.kth.karamel.backend.machines.SshMachine: 172.16.0.4: connecting ...
INFO  [2020-07-01 14:22:34,465] net.schmizz.sshj.transport.TransportImpl: Client identity string: SSH-2.0-SSHJ_0.20.0
INFO  [2020-07-01 14:22:34,489] net.schmizz.sshj.transport.TransportImpl: Server identity string: SSH-2.0-OpenSSH_7.6p1 Ubuntu-4ubuntu0.3
INFO  [2020-07-01 14:22:34,760] se.kth.karamel.backend.machines.SshMachine: 172.16.0.4: Yey!! connected ^-^
INFO  [2020-07-01 14:22:34,858] net.schmizz.sshj.transport.random.BouncyCastleRandom: Generating random seed from SecureRandom.
INFO  [2020-07-01 14:22:34,876] se.kth.karamel.backend.machines.SshMachine: 172.16.0.5: connecting ...
INFO  [2020-07-01 14:22:34,878] net.schmizz.sshj.transport.TransportImpl: Client identity string: SSH-2.0-SSHJ_0.20.0
INFO  [2020-07-01 14:22:34,884] net.schmizz.sshj.transport.TransportImpl: Server identity string: SSH-2.0-OpenSSH_7.6p1 Ubuntu-4ubuntu0.3
INFO  [2020-07-01 14:22:34,920] se.kth.karamel.backend.machines.SshMachine: 172.16.0.5: Yey!! connected ^-^
INFO  [2020-07-01 14:22:35,575] se.kth.karamel.backend.machines.SshMachine: 172.16.0.5: Running task: find os-type
INFO  [2020-07-01 14:22:35,776] se.kth.karamel.backend.machines.SshMachine: 172.16.0.5: Running task: apt-get essentials
INFO  [2020-07-01 14:22:36,172] se.kth.karamel.backend.machines.SshMachine: 172.16.0.4: Running task: find os-type
INFO  [2020-07-01 14:22:36,248] se.kth.karamel.backend.machines.SshMachine: 172.16.0.4: Running task: apt-get essentials
INFO  [2020-07-01 14:22:47,305] se.kth.karamel.backend.machines.SshMachine: 172.16.0.5: Running task: install chefdk
INFO  [2020-07-01 14:22:47,331] se.kth.karamel.backend.machines.SshMachine: 172.16.0.5: Running task: make solo.rb
INFO  [2020-07-01 14:22:47,397] se.kth.karamel.backend.machines.SshMachine: 172.16.0.5: Running task: clone and vendor hopsworks-chef
INFO  [2020-07-01 14:22:52,514] se.kth.karamel.backend.machines.SshMachine: 172.16.0.4: Running task: install chefdk
INFO  [2020-07-01 14:22:52,583] se.kth.karamel.backend.machines.SshMachine: 172.16.0.4: Running task: make solo.rb
INFO  [2020-07-01 14:22:52,604] se.kth.karamel.backend.machines.SshMachine: 172.16.0.4: Running task: clone and vendor hopsworks-chef
INFO  [2020-07-01 14:23:06,702] se.kth.karamel.backend.machines.SshMachine: 172.16.0.5: Running task: livy::install
INFO  [2020-07-01 14:23:13,254] se.kth.karamel.backend.machines.SshMachine: 172.16.0.4: Running task: kagent::install
WARN  [2020-07-01 14:23:25,047] net.schmizz.sshj.xfer.scp.SCPEngine: SCP exit status: 1
INFO  [2020-07-01 14:23:25,048] se.kth.karamel.backend.machines.SshMachine: 172.16.0.5: Running task: hopslog::install
WARN  [2020-07-01 14:23:27,408] net.schmizz.sshj.xfer.scp.SCPEngine: SCP exit status: 1
INFO  [2020-07-01 14:23:27,409] se.kth.karamel.backend.machines.SshMachine: 172.16.0.4: Running task: kzookeeper::install
WARN  [2020-07-01 14:23:35,485] net.schmizz.sshj.xfer.scp.SCPEngine: SCP exit status: 1
INFO  [2020-07-01 14:23:35,486] se.kth.karamel.backend.machines.SshMachine: 172.16.0.4: Running task: hopslog::install
WARN  [2020-07-01 14:24:07,806] net.schmizz.sshj.xfer.scp.SCPEngine: SCP exit status: 1
INFO  [2020-07-01 14:24:07,861] se.kth.karamel.backend.machines.SshMachine: 172.16.0.5: Running task: kagent::install
WARN  [2020-07-01 14:24:18,666] net.schmizz.sshj.xfer.scp.SCPEngine: SCP exit status: 1
INFO  [2020-07-01 14:24:18,669] se.kth.karamel.backend.machines.SshMachine: 172.16.0.5: Running task: hopsmonitor::install
WARN  [2020-07-01 14:24:30,967] net.schmizz.sshj.xfer.scp.SCPEngine: SCP exit status: 1
INFO  [2020-07-01 14:24:30,970] se.kth.karamel.backend.machines.SshMachine: 172.16.0.5: Running task: flink::install
WARN  [2020-07-01 14:24:45,918] net.schmizz.sshj.xfer.scp.SCPEngine: SCP exit status: 1
INFO  [2020-07-01 14:24:45,925] se.kth.karamel.backend.machines.SshMachine: 172.16.0.4: Running task: consul::install
WARN  [2020-07-01 14:25:03,331] net.schmizz.sshj.xfer.scp.SCPEngine: SCP exit status: 1
INFO  [2020-07-01 14:25:03,335] se.kth.karamel.backend.machines.SshMachine: 172.16.0.4: Running task: hopsmonitor::install
WARN  [2020-07-01 14:25:16,975] net.schmizz.sshj.xfer.scp.SCPEngine: SCP exit status: 1
INFO  [2020-07-01 14:25:16,976] se.kth.karamel.backend.machines.SshMachine: 172.16.0.4: Running task: flink::install
WARN  [2020-07-01 14:25:21,947] net.schmizz.sshj.xfer.scp.SCPEngine: SCP exit status: 1
INFO  [2020-07-01 14:25:21,948] se.kth.karamel.backend.machines.SshMachine: 172.16.0.5: Running task: consul::install
WARN  [2020-07-01 14:25:35,274] net.schmizz.sshj.xfer.scp.SCPEngine: SCP exit status: 1
INFO  [2020-07-01 14:25:35,275] se.kth.karamel.backend.machines.SshMachine: 172.16.0.5: Running task: hops::install
WARN  [2020-07-01 14:25:50,009] net.schmizz.sshj.xfer.scp.SCPEngine: SCP exit status: 1
INFO  [2020-07-01 14:25:50,059] se.kth.karamel.backend.machines.SshMachine: 172.16.0.5: Running task: conda::install
WARN  [2020-07-01 14:26:22,469] net.schmizz.sshj.xfer.scp.SCPEngine: SCP exit status: 1
INFO  [2020-07-01 14:26:22,475] se.kth.karamel.backend.machines.SshMachine: 172.16.0.5: Running task: hadoop_spark::install
WARN  [2020-07-01 14:26:31,117] net.schmizz.sshj.xfer.scp.SCPEngine: SCP exit status: 1
INFO  [2020-07-01 14:26:31,118] se.kth.karamel.backend.machines.SshMachine: 172.16.0.4: Running task: hops::install
WARN  [2020-07-01 14:26:54,624] net.schmizz.sshj.xfer.scp.SCPEngine: SCP exit status: 1
INFO  [2020-07-01 14:26:54,627] se.kth.karamel.backend.machines.SshMachine: 172.16.0.5: Running task: tensorflow::install
WARN  [2020-07-01 14:27:09,767] net.schmizz.sshj.xfer.scp.SCPEngine: SCP exit status: 1
INFO  [2020-07-01 14:27:09,770] se.kth.karamel.backend.machines.SshMachine: 172.16.0.4: Running task: hadoop_spark::install
WARN  [2020-07-01 14:27:10,891] net.schmizz.sshj.xfer.scp.SCPEngine: SCP exit status: 1
INFO  [2020-07-01 14:27:10,892] se.kth.karamel.backend.machines.SshMachine: 172.16.0.5: Running task: hopsmonitor::purge_telegraf
WARN  [2020-07-01 14:27:22,688] net.schmizz.sshj.xfer.scp.SCPEngine: SCP exit status: 1
INFO  [2020-07-01 14:27:22,690] se.kth.karamel.backend.machines.SshMachine: 172.16.0.5: Running task: hopsmonitor::node_exporter
WARN  [2020-07-01 14:27:33,148] net.schmizz.sshj.xfer.scp.SCPEngine: SCP exit status: 1
INFO  [2020-07-01 14:27:33,150] se.kth.karamel.backend.machines.SshMachine: 172.16.0.5: Running task: conda::default
WARN  [2020-07-01 14:28:03,569] net.schmizz.sshj.xfer.scp.SCPEngine: SCP exit status: 1
INFO  [2020-07-01 14:28:03,575] se.kth.karamel.backend.machines.SshMachine: 172.16.0.4: Running task: elastic::install
WARN  [2020-07-01 14:28:07,590] net.schmizz.sshj.xfer.scp.SCPEngine: SCP exit status: 1
WARN  [2020-07-01 14:28:14,210] net.schmizz.sshj.xfer.scp.SCPEngine: SCP exit status: 1
INFO  [2020-07-01 14:28:14,210] se.kth.karamel.backend.machines.SshMachine: 172.16.0.4: Running task: tensorflow::install
WARN  [2020-07-01 14:28:31,772] net.schmizz.sshj.xfer.scp.SCPEngine: SCP exit status: 1
INFO  [2020-07-01 14:28:31,773] se.kth.karamel.backend.machines.SshMachine: 172.16.0.4: Running task: kkafka::install
WARN  [2020-07-01 14:28:41,279] net.schmizz.sshj.xfer.scp.SCPEngine: SCP exit status: 1
INFO  [2020-07-01 14:28:41,280] se.kth.karamel.backend.machines.SshMachine: 172.16.0.4: Running task: hive2::install
WARN  [2020-07-01 14:29:26,411] net.schmizz.sshj.xfer.scp.SCPEngine: SCP exit status: 1
INFO  [2020-07-01 14:29:26,414] se.kth.karamel.backend.machines.SshMachine: 172.16.0.4: Running task: hops_airflow::install
WARN  [2020-07-01 14:29:39,337] net.schmizz.sshj.xfer.scp.SCPEngine: SCP exit status: 1
INFO  [2020-07-01 14:29:39,338] se.kth.karamel.backend.machines.SshMachine: 172.16.0.4: Running task: livy::install
WARN  [2020-07-01 14:29:56,448] net.schmizz.sshj.xfer.scp.SCPEngine: SCP exit status: 1
INFO  [2020-07-01 14:29:56,449] se.kth.karamel.backend.machines.SshMachine: 172.16.0.4: Running task: conda::install
WARN  [2020-07-01 14:31:22,619] net.schmizz.sshj.xfer.scp.SCPEngine: SCP exit status: 1
INFO  [2020-07-01 14:31:22,623] se.kth.karamel.backend.machines.SshMachine: 172.16.0.4: Running task: ndb::install
WARN  [2020-07-01 14:33:19,929] net.schmizz.sshj.xfer.scp.SCPEngine: SCP exit status: 1
INFO  [2020-07-01 14:33:19,933] se.kth.karamel.backend.machines.SshMachine: 172.16.0.4: Running task: hopsmonitor::purge_telegraf
WARN  [2020-07-01 14:33:33,415] net.schmizz.sshj.xfer.scp.SCPEngine: SCP exit status: 1
INFO  [2020-07-01 14:33:33,417] se.kth.karamel.backend.machines.SshMachine: 172.16.0.4: Running task: hopsmonitor::prometheus
WARN  [2020-07-01 14:33:49,285] net.schmizz.sshj.xfer.scp.SCPEngine: SCP exit status: 1
INFO  [2020-07-01 14:33:49,287] se.kth.karamel.backend.machines.SshMachine: 172.16.0.4: Running task: hopsmonitor::alertmanager
WARN  [2020-07-01 14:34:04,454] net.schmizz.sshj.xfer.scp.SCPEngine: SCP exit status: 1
INFO  [2020-07-01 14:34:04,457] se.kth.karamel.backend.machines.SshMachine: 172.16.0.4: Running task: hopsmonitor::node_exporter
2020-07-01T14:34:15.442464Z INFO ExtHandler ExtHandler [HEARTBEAT] Agent WALinuxAgent-2.2.48.1 is running as the goal state agent
WARN  [2020-07-01 14:34:27,299] net.schmizz.sshj.xfer.scp.SCPEngine: SCP exit status: 1
INFO  [2020-07-01 14:34:27,303] se.kth.karamel.backend.machines.SshMachine: 172.16.0.4: Running task: hopsworks::install
WARN  [2020-07-01 14:35:05,255] net.schmizz.sshj.xfer.scp.SCPEngine: SCP exit status: 1
INFO  [2020-07-01 14:35:05,277] se.kth.karamel.backend.machines.SshMachine: 172.16.0.4: Running task: conda::default
2020-07-01T15:04:20.099367Z INFO ExtHandler ExtHandler [HEARTBEAT] Agent WALinuxAgent-2.2.48.1 is running as the goal state agent
2020-07-01T15:04:24.144914Z INFO ExtHandler ExtHandler Checking for agent updates (family: Prod)

Fabio · July 1, 2020, 8:54pm

Hi @Phil,

Welcome to Hopsworks!
conda::default is expected to take some time. It’s updating the base Anaconda installation, which is quite heavy.

Did it make any progress? Or is it still stuck?

By the way, you can find more logs about this specific recipe execution here: /home/AzureUser/.karamel/install/conda__default.log
If it’s stuck you can post it here as well, so we can help you out.

–
Fabio

Jim_Dowling · July 2, 2020, 9:16am

Hi @phil . Another issue with installing Hopsworks on Azure is that you need to add a private DNS Zone to your resource group and connect it to your virtual network. Hopsworks uses TLS, and it needs FQDNs, which you don’t get in VMs by default in Azure. You have to enable private DNS zones.

If you have azure tools and want to try out bleeding edge, you could try this:
git clone https://github.com/logicalclocks/karamel-chef.git
cd karamel-chef && git checkout 1.3
cd cloud/azure
. setup.sh
./install.sh cpu community

This will create a resource group, private dns zone, virtual network, hook them up, spin up a VM, and install hopsworks on it with Karamel. It is still pre-release, so any feedback on it is welcome.
You can edit the config.sh file to change the VM type, Zone/Region, etc.