Python: use xmlrpc module with a server requiring authentication

Question

I want to query an XML-RPC server using Python script.

First obstacle was that the server uses authentication, and I had to resort to requests.Session to overcome this. Now I did, and I can query the server over RPC, but I'm using rather ugly mechanism to do it - glueing strings. This, of course, produces just XML data, which I would again have to process manually to convert it into usable list or dictionary.

I found there's an xmlrpc module for Python which allows to cleanly work with a server; however, I can't find how to auth myself prior to using XML-RPC.

Here's the code that I already wrote, using requests:

from requests import Session

myserver_address='http://myserver.test.com/admin'
myserver_RPC=myserver_address+'?RPC2'
myserver_header={'Content-Type': "text/xml; charset=UTF-8"}
myserver_login={"srvAction":"LoginAdmin", "login":"testuser", "password":"testpassword", "Submit":"Login", "select_locale":"en"}
myserver_login2={"srvAction":"LoginOrg", "selectSection":"20", "submit":"Continue"}

method_header = '<methodCall xmlns:ex="http://ws.apache.org/xmlrpc/namespaces/extensions"><methodName>'
param_header = '</methodName><params>'
param_body_header = '<param><value><string>'
param_body_footer = '</string></value></param>'
method_footer = '</params></methodCall>'

def myserver_get_xml(myserver_method, myserver_method_param):
    param_body = ''

    for param in myserver_method_param:
        param_body = param_body + param_body_header + str(param) + param_body_footer

    myserver_post = method_header + myserver_method + param_header + param_body + method_footer
    page = s.post(myserver_RPC, myserver_post)
    return page.text

s = Session()
s.get(myserver_address)
s.post(myserver_address, myserver_login) 
s.post(myserver_address, myserver_login2)

s.headers.update(myserver_header)

result_xml = myserver_get_xml('myserverServer.getNetworkTree', ['31', ] )
print(result_xml)

So the question is:

• how to use xmlrpc module within an authenticated session?

• if that's not possible, is there something nice to convert XML into dictionary (preferrably) or list?

Example output:

<?xml version="1.0" encoding="UTF-8"?><methodResponse xmlns:ex="http://ws.apache.org/xmlrpc/namespaces/extensions"><params><param><value><array><data><value><array><data><value>10.0.0.0/16</value><value><i4>1</i4></value><value>31</value></data></array></value><value><array><data><value>10.1.0.0/16</value><value><i4>2</i4></value><value>31</value></data></array></value></data></array></value></param></params></methodResponse>

Answer 1

You would need to implement your own xmlrpclib.Transport class and specify that as your transport parameter when you instantiated the xmlrpclib.ServerProxy. See the example of a proxied transport implementation on https://docs.python.org/2/library/xmlrpclib.html#example-of-client-usage. You probably have to implement and modified Transport.make_connection only, which needs to return an httplib.HTTPConnection instance which has the Cookie header set from the http response headers from your login actions. You can't really use the Session class to make requests, as it is not compatible with the httplib.HTTPConnection class that is used by the rest of the Transport class methods. So to keep things simple, you might be able to perform the login with a Session instance, extract the headers (in particular the cookie header) and create an HTTPConnection with these headers.

def create_session_headers():
      s = requests.Session()
      # perform your request.Session creation with the login posts
      # return headers from the loggedin session
      return s.headers

class SessionTransport(xmlrpclib.Transport):
    def make_connection(self, host):
          # create the standard connection
          connection = super(SessionTransport, self).make_connection(host)
          session_headers = create_session_headers()
          for key, value in session_headers.items():
              connection.putheader(key, value)
          return connection

 service = xmlrpclib.ServerProxy(server_address, SessionTransport())

I'm not sure if it will work like this, I haven't tested it. I hope it gives you enough information to try this approach.

Answer 2

Found solution a while ago, works so far. It's very crude and not suitable to processing XML, but since I need just some data out of the app, it works.

#!/cygdrive/c/Python34/python.exe

from requests import Session
import sys
import json
import xmltodict

myserver_address='http://myserver.test.com/admin'
myserver_RPC=myserver_address+'?RPC2'
myserver_header={'Content-Type': "text/xml; charset=UTF-8"}
myserver_login={"srvAction":"LoginAdmin", "login":"testuser", "password":"testpassword", "Submit":"Login", "select_locale":"en"}
myserver_login2={"srvAction":"LoginOrg", "selectSection":"20", "submit":"Continue"}

s = Session()
s.get(myserver_address)
s.post(myserver_address, myserver_login) 
s.post(myserver_address, myserver_login2)

s.headers.update(myserver_header)

myserverMethod = '<methodCall xmlns:ex="http://ws.apache.org/xmlrpc/namespaces/extensions"><methodName>myserverMyViewUsage.getMyViewChildrenArray</methodName><params><param><value><int>602</int></value></param></params></methodCall>'
subnet_list = s.post(myserver_RPC, myserverMethod)
subnet_dictionary_coarse = xmltodict.parse(subnet_org_list.text)

#print( json.dumps(subnet_org_dictionary_coarse, indent=2))

for subnet_org_dictionary in subnet_org_dictionary_coarse['methodResponse']['params']['param']['value']['array']['data']['value']:
<put code here>

Make no mistake, this is basically raping XML to get the data I need; it's a "quick and very dirty" hack.