How can I parse an inbound message with MTOM attachments under Metro without pulling in all the attachment data?

Question

Using JAX-WS-RI or Metro I can write a WebService using the com.sun.xml.ws.api.server.AsyncProvider interface.

I can choose to get the whole message including the SOAP headers

import javax.xml.transform.Source;
import com.sun.xml.ws.api.server.AsyncProvider;
import com.sun.xml.ws.api.server.AsyncProviderCallback;

import javax.xml.ws.ServiceMode;
import javax.xml.ws.WebServiceContext;
import javax.xml.ws.WebServiceProvider;


@ServiceMode(value=Service.Mode.MESSAGE)
@WebServiceProvider()
public class AddNumbersAsynchImpl implements AsyncProvider<Source> {
...

I can then write something which parses the message and processes accordingly

public void invoke(Source source, AsyncProviderCallback<Source> cbak, WebServiceContext ctxt) {
        DOMResult dom = new DOMResult();
        Transformer trans = TransformerFactory.newInstance().newTransformer();
        trans.transform(source, dom);

        XPath xpath = XPathFactory.newInstance().newXPath();
        xpath.setNamespaceContext(new MyNamespaceContext());

        String opName = (String) xpath.evaluate("name(/S:Envelope/S:Body/X:*[1])", dom.getNode(), XPathConstants.STRING);

        if(knownOp(opName)) {
            doOp(opName, dom.getNode(), cbak);
        }
        else {
            doFault("Unknown operation " + opName, cbak);
        }

}

However part of the reason for doing this is to adapt an existing XML based application into the SOAP stack. The application defines a complete set of schemas for the messages that it takes and it easy to generate a WSDL to define the service.

For small XML messages this all works fine.

However if I wish to process the XML in a more stream orientated fashion in some operations where the messages are bulky I come across two problems using MTOM attachments. I change the set up of my Provider as follows;

import com.sun.xml.ws.api.message.Message;
import javax.xml.ws.soap.MTOM;
import com.sun.xml.ws.developer.StreamingAttachment;
import com.sun.xml.ws.developer.StreamingDataHandler;

ServiceMode(value=Service.Mode.MESSAGE)
@WebServiceProvider()
@StreamingAttachment(parseEagerly=true, memoryThreshold=1000L)
@MTOM(enabled=true, threshold=1000) 
public class AddNumbersAsynchImplMessage implements AsyncProvider<Message> {
...

I Add the appropriate MTOM policy to my WSDL;

<definitions
        name="AddNumbersAsynch"
        targetNamespace="http://asynch.duke.example.org"
        xmlns:tns="http://asynch.duke.example.org"
        xmlns="http://schemas.xmlsoap.org/wsdl/"
        xmlns:xsd="http://www.w3.org/2001/XMLSchema"
        xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
        xmlns:wsam="http://www.w3.org/2007/05/addressing/metadata"
        xmlns:wsrm="http://docs.oasis-open.org/ws-rx/wsrmp/200702"
        xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd"
        xmlns:wsoma="http://schemas.xmlsoap.org/ws/2004/09/policy/optimizedmimeserialization"
        xmlns:wsp="http://schemas.xmlsoap.org/ws/2004/09/policy">
        <wsp:Policy wsu:Id="AddNumbersAsynch_policy">
            <wsam:Addressing wsp:Optional="false" />
            <wsoma:OptimizedMimeSerialization/>
        </wsp:Policy>

And declare my message schemas for this operation as appropriate

        <complexType name="sendBulk">
            <sequence>
                <element name="data" type="base64Binary"
                    xmime:expectedContentTypes="application/xml"
                    xmlns:xmime="http://www.w3.org/2005/05/xmlmime"/>
            </sequence>
        </complexType>
        <element name="sendBulk" type="tns:sendBulk"/>

        <complexType name="sendBulkResponse">
            <sequence>
                <element name="return" type="xsd:string" />
            </sequence>
        </complexType>
        <element name="sendBulkResponse" type="tns:sendBulkResponse"/>

Setting up the binding as appropriate;

<binding name="AddNumbersBinding" type="tns:AddNumbersPortType">
<wsp:PolicyReference URI="#AddNumbersAsynch_policy"/>
    <soap:binding transport="http://schemas.xmlsoap.org/soap/http" style="document" />
    <operation name="sendBulk">
        <soap:operation soapAction="" />
        <input>
            <soap:body use="literal" />
        </input>
        <output>
            <soap:body use="literal" />
        </output>
    </operation>
</binding>

I generate the client and amend the code

port = new AddNumbersService().getAddNumbersPort(new SyncStartForAsyncFeature(), new MTOMFeature(1000));
  Source data = new StreamSource(new FileInputStream(file));
  String result = port.sendBulk(data);

and the http dump shows that the data is being sent as a multipart mime with the appropriate xop:include element in the 'message body'

---[HTTP request - http://localhost:8080/ProviderTest2/addnumbersAsynchMessage]---
Accept: text/xml, multipart/related
Content-Type: multipart/related;start="<rootpart*[email protected]>";type="application/xop+xml";boundary="uuid:daff762b-9651-40aa-ae2f-30d30a3c5e2e";start-info="text/xml"
SOAPAction: "http://message.asynch.duke.example.org/AddNumbersPortType/sendBulkRequest"
User-Agent: Metro/2.2 (branches/2.2-7015; 2012-02-20T20:31:25+0000) JAXWS-RI/2.2.6 JAXWS/2.2 svn-revision#unknown
--uuid:daff762b-9651-40aa-ae2f-30d30a3c5e2e
Content-Id: <rootpart*[email protected]>
Content-Type: application/xop+xml;charset=utf-8;type="text/xml"
Content-Transfer-Encoding: binary

<?xml version='1.0' encoding='UTF-8'?><S:Envelope xmlns:S="http://schemas.xmlsoap.org/soap/envelope/"><S:Header><To xmlns="http://www.w3.org/2005/08/addressing">http://localhost:8080/ProviderTest2/addnumbersAsynchMessage</To><Action xmlns="http://www.w3.org/2005/08/addressing" xmlns:S="http://schemas.xmlsoap.org/soap/envelope/" S:mustUnderstand="1">http://message.asynch.duke.example.org/AddNumbersPortType/sendBulkRequest</Action><ReplyTo xmlns="http://www.w3.org/2005/08/addressing">
    <Address>http://www.w3.org/2005/08/addressing/anonymous</Address>
</ReplyTo><FaultTo xmlns="http://www.w3.org/2005/08/addressing">
    <Address>http://www.w3.org/2005/08/addressing/anonymous</Address>
</FaultTo><MessageID xmlns="http://www.w3.org/2005/08/addressing">uuid:406ce4bc-7332-4849-b16b-69cec6ca21ea</MessageID></S:Header><S:Body><sendBulk xmlns="http://message.asynch.duke.example.org"><data><xop:Include xmlns:xop="http://www.w3.org/2004/08/xop/include" href="cid:[email protected]"/></data></sendBulk></S:Body></S:Envelope>
--uuid:daff762b-9651-40aa-ae2f-30d30a3c5e2e
Content-Id: <[email protected]>
Content-Type: application/xml; charset=UTF-8
Content-Transfer-Encoding: binary

....snip!

Inside the server I get an attachment;

public void invoke(Message source, AsyncProviderCallback<Message> cbak, WebServiceContext ctxt) {

        Map<String, DataHandler> attachments = (Map<String, DataHandler>) ctxt.getMessageContext().get(javax.xml.ws.handler.MessageContext.INBOUND_MESSAGE_ATTACHMENTS); 
        for(String attachmentKey: attachments.keySet()) {
            StreamingDataHandler handler = (StreamingDataHandler) attachments.get(attachmentKey);
            System.out.println("Got attachment " + attachmentKey + " of type " + attachments.get(attachmentKey));
        }
        if(attachments.isEmpty()) {
            System.out.println("Got No attachments");
        }

I then wish top parse the incoming message and get the context for the bulk attachment - for example the name of the element that encloses it - but not the bulk attachment itself, YET;

XMLStreamWriter sw = new MyXMLStreamWriter();
source.writeTo(sw);

Hopefully the attachment parser has not read all that part of the incoimuing HTTP stream yet, or if it has it has chunked it off into a file, where I can then read it chunk by chunk. The trouble is that I cannot find a way of parsing Message that does not convert the incoming into the base64 encoded data in the attachment. After I get the SAX/StaX event indicating the start of the element, it is followed by a characters event which is the base64 encoded data. Looking at the code for StreamMessage there is no way to "insert" any XMLReader or XMLStreamReader etc. 'in front of' the existing XMLStreamReader that the Message is already bound to ( not suprising as we are part way through message processing). But that XMLStreamReader implementation is a com.sun.xml.ws.encoding.MtomCodec$MtomXMLStreamReaderEx which always 'unpacks' attachments in line into memory. There is no way to paramterize the com.sun.xml.ws.encoding.MtomCodec class so that it does not first pass events to its inner MtomXMLStreamReaderEx class which unpacks the attachment inline.

I tried to read the attachment into a temporary file and then substitute the Message context attachments for a FileDataSource, but since Metro ignores these, that only had the effect of making the original DataSource throw a ReadOnce type error.

Map<String, DataHandler> attachments = (Map<String, DataHandler>) ctxt.getMessageContext().get(javax.xml.ws.handler.MessageContext.INBOUND_MESSAGE_ATTACHMENTS);
            HashMap<String, DataHandler> clonedAttachments = new HashMap<String, DataHandler>(attachments);
            for(String attachmentKey: attachments.keySet()) {
                StreamingDataHandler handler = (StreamingDataHandler) attachments.get(attachmentKey);
                File tempFile = File.createTempFile("Attachment" + attachmentKey, ".xmlb64", new File("C:\\Users\\Administrator\\workspaceTool1\\TomcatWorkingDir"));
                handler.moveTo(tempFile);
                FileDataSource newSource = new FileDataSource(tempFile);
                clonedAttachments.put(attachmentKey, new DataHandler(newSource));
                System.out.println("Got attachment " + attachmentKey + " of type " + attachments.get(attachmentKey));
            }
            if(attachments.isEmpty()) {
                System.out.println("Got No attachments");
            }
            else {
                ctxt.getMessageContext().put(javax.xml.ws.handler.MessageContext.INBOUND_MESSAGE_ATTACHMENTS, clonedAttachments);
            }

I tried also turning off MTOM in the provider by setting the annotation to disabled;

@MTOM(enabled=false) 

and then removing it and then the StreamingAttachment annotation to no avail. I also set the enable-mtom="false" attribute in the endpoint element for it in the sun-jaxws.xml file, again to no avail; it seems that xop:Include parsing is hardwired. I do not want to change the wsdl because that I do want clients to use MTOM if possible. ( but see my swaRef answer below )

I can make Metro do what I want if I use swaRef, but my understanding was that MTOM is the protocol that has gained support? With the swaRef I have to edit the WSDL ;

<types>


        <xsd:schema
            xmlns="http://www.w3.org/2001/XMLSchema"
            targetNamespace="http://swaRef.asynch.duke.example.org"
            elementFormDefault="qualified">

        <xsd:import namespace="http://ws-i.org/profiles/basic/1.1/xsd"  
                        schemaLocation="http://www.ws-i.org/profiles/basic/1.1/swaref.xsd"/>



            <complexType name="sendBulk">
                <sequence>
                    <element name="data" type="wsi:swaRef"
                        xmlns:wsi="http://ws-i.org/profiles/basic/1.1/xsd"/>
                </sequence>
            </complexType>
            <element name="sendBulk" type="tns:sendBulk"/>

I also have to generate a new client and call the sendBulk method differently;

FileDataSource newSource = new FileDataSource(file);
DataHandler data = new DataHandler(newSource);
String result = port.sendBulk(data);

But now when I parse the data element I get the corresponding cid which I can then use to retrieve the attachment as I wish;

DataHandler handler = attachments.get(dataText.substring("cid:".length()));
InputStream input = handler.getInputStream();

So, with MTOM and Metro though the attachments may have not been read up to the point where I parse the data element, or indeed streamed efficiently in chunks to a temporary file, I have no way to read the "rest" of the input message without reading the whole of the attachment into memory which seems self defeating? Apart from dumping Metro - suggestions re: Axis, CXF welcomed - is there any way around this anyone has discovered?

Logged a feature request.