Hello
I am working in ten prime environment.
KafkaException: KafkaError{code=_TRANSPORT,val=-195,str=“Failed to get metadata: Local: Broker transport failure”}
what is the reason for this error? How do I solve it?
Hello,
Can you briefly explain your setup? I am not aware of ten prime
Did you install Hopsworks yourself? Are you trying to write to feature store from outside the cluster network and you are getting this error? Is there any firewall in place that could be blocking traffic?
Kind regards
I have set up a cluster on GCP and am facing the same error when trying to load the first data into a new feature group. Attached are my cluster details. Going to try to isolate the error now.
RESOLVED see below
Successful GET calls, followed by the error uploading a basic test feature.
import numpy as np
import pandas as pd
test_fg = fs.get_or_create_feature_group(
name="test",
version=1,
description="test",
primary_key=["index"],
)
index = np.arange(50)
values = np.ones_like(index)
xdf = pd.DataFrame(index=index, data=values, columns=['values']).reset_index()
test_fg.insert(xdf, write_options=dict(wait_for_job=True))
KafkaException: KafkaError{code=_TRANSPORT,val=-195,str="Failed to get metadata: Local: Broker transport failure"}
RESOLVED
@antonios could you explain what Kafka is doing here and why this service needs access from the internet to perform the load.
Hello,
The ingested data are written to a Kafka topic that is then used by Hopsworks to synchronise the writes to the Online and Offline feature groups, please refer to the documentation for more details. You don’t need to open the port to the internet as long as you can ensure access to that port from your environment.
Regards,
Mahmoud
Hi, I still get the same error while adding data frames to feature group’s
cimpl.KafkaException: KafkaError{code=_TRANSPORT,val=-195,str=“Failed to get metadata: Local: Broker transport failure”}
Is there any solution for this, please. Kind of stuck here. Thanks
Hello @vdsanku ,
Can you elaborate on your setup? Are you trying to write to the feature store from outside the cluster network?
The error above essentially mean that you cannot connect to the Kafka brokers running on your cluster which suggests either a problem with the network/ports or with the advertised ips for the brokers.
Regards,
Mahmoud
Try using a different network, that worked for me