I cannot curl a URL in a notebook

jq2024
New Contributor II

‎08-27-2024 02:24 AM

I tried to curl the following url in a notebook

 

%sh

curl https://staging-api.newrelic.com/graphql -v

 

 But I got the following error message

 

{ [5 bytes data]
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
} [5 bytes data]
* TLSv1.2 (IN), TLS header, Supplemental data (23):
{ [5 bytes data]
* TLSv1.2 (IN), TLS header, Supplemental data (23):
{ [5 bytes data]
< HTTP/2 418 
< date: Tue, 27 Aug 2024 09:16:43 GMT
< content-type: text/plain
< content-length: 23
< server: cloudflare
< cf-ray: 8b9af6e55b080580-IAD
< 
* TLSv1.2 (IN), TLS header, Supplemental data (23):
{ [5 bytes data]
* TLSv1.2 (IN), TLS header, Supplemental data (23):
{ [5 bytes data]
100    23  100    23    0     0    283      0 --:--:-- --:--:-- --:--:--   283
* Connection #0 to host staging-api.newrelic.com left intact
Unauthorized IP address

 

How can I fix this  Unauthorized IP address issue?

0 Kudos

szymon_dybczak
Esteemed Contributor III

‎08-27-2024 05:22 AM

Hi @jq2024 ,

It looks like this API is behind some kind of firewall. You need to find IP address of your cluster and then add this IP address to whitelist.
To find your cluster ip addresses, you can use below code:

def get_external_ip(x):
    import requests
    import socket

    hostname = socket.gethostname()
    r = requests.get("https://api.ipify.org/")
    public_IP = r.content
    return(f"#{x} From {hostname} with publicIP {public_IP}.")

print('DRIVER:')
rdd1 = get_external_ip(0)
print(rdd1)
print('WORKERS:')   
rdd2 = sc.parallelize(range(1, 4)).map(get_external_ip)
datacoll2 = rdd2.collect()
for row in datacoll2:
    print(row)

View solution in original post

jq2024
New Contributor II

‎09-04-2024 08:05 AM

Thank you for the suggestion. It works

0 Kudos