Can you use a proxy to connect to HDFS using the Kite SDK?

84 views Asked by At

I am attempting to create a proof of concept Hadoop instance, and have chosen to use the KiteSDK to interact with it (as well as to structure my data). When I set up a Hadoop instance on my local box, everything is working correctly and my code connects without problems. However, when I set up Hadoop on an AWS server, I am no longer able to connect to it. I believe that this is because of my network's security settings, and I suspect that I will need to use a proxy to connect. Is this possible? If so, can you provide an example of how to do it? The Kite CLI has a proxy-user parameter that can be passed in, but when I look at the SDK's code, it doesn't appear that setting the value of that does what I need.

1

There are 1 answers

0
Josh Edwards On

I never figured out a proxy setting for Kite SDK, but I was able to work around this issue by finding a port that was not behind a firewall and reconfigure Hadoop to have the nodename reside on that port.