cancel
Showing results for 
Search instead for 
Did you mean: 
Community Platform Discussions
Connect with fellow community members to discuss general topics related to the Databricks platform, industry trends, and best practices. Share experiences, ask questions, and foster collaboration within the community.
cancel
Showing results for 
Search instead for 
Did you mean: 

I cannot curl a URL in a notebook

jq2024
New Contributor II

I tried to curl the following url in a notebook

 

%sh

curl https://staging-api.newrelic.com/graphql -v

 

 But I got the following error message

 

{ [5 bytes data]
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
} [5 bytes data]
* TLSv1.2 (IN), TLS header, Supplemental data (23):
{ [5 bytes data]
* TLSv1.2 (IN), TLS header, Supplemental data (23):
{ [5 bytes data]
< HTTP/2 418 
< date: Tue, 27 Aug 2024 09:16:43 GMT
< content-type: text/plain
< content-length: 23
< server: cloudflare
< cf-ray: 8b9af6e55b080580-IAD
< 
* TLSv1.2 (IN), TLS header, Supplemental data (23):
{ [5 bytes data]
* TLSv1.2 (IN), TLS header, Supplemental data (23):
{ [5 bytes data]
100    23  100    23    0     0    283      0 --:--:-- --:--:-- --:--:--   283
* Connection #0 to host staging-api.newrelic.com left intact
Unauthorized IP address

 

How can I fix this  Unauthorized IP address issue?

1 ACCEPTED SOLUTION

Accepted Solutions

szymon_dybczak
Contributor III

Hi @jq2024 ,

It looks like this API is behind some kind of firewall. You need to find IP address of your cluster and then add this IP address to whitelist.
To find your cluster ip addresses, you can use below code:

def get_external_ip(x):
    import requests
    import socket

    hostname = socket.gethostname()
    r = requests.get("https://api.ipify.org/")
    public_IP = r.content
    return(f"#{x} From {hostname} with publicIP {public_IP}.")

print('DRIVER:')
rdd1 = get_external_ip(0)
print(rdd1)
print('WORKERS:')   
rdd2 = sc.parallelize(range(1, 4)).map(get_external_ip)
datacoll2 = rdd2.collect()
for row in datacoll2:
    print(row)

View solution in original post

2 REPLIES 2

szymon_dybczak
Contributor III

Hi @jq2024 ,

It looks like this API is behind some kind of firewall. You need to find IP address of your cluster and then add this IP address to whitelist.
To find your cluster ip addresses, you can use below code:

def get_external_ip(x):
    import requests
    import socket

    hostname = socket.gethostname()
    r = requests.get("https://api.ipify.org/")
    public_IP = r.content
    return(f"#{x} From {hostname} with publicIP {public_IP}.")

print('DRIVER:')
rdd1 = get_external_ip(0)
print(rdd1)
print('WORKERS:')   
rdd2 = sc.parallelize(range(1, 4)).map(get_external_ip)
datacoll2 = rdd2.collect()
for row in datacoll2:
    print(row)

jq2024
New Contributor II

Thank you for the suggestion. It works

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group