micronaut httpclient is replacing ampersands and equal signs in my url, but I dont want it to

128 views Asked by At

Short Version

after reading the httpclient logs, I'm seeing that some characters are being url-encoded. I dont want that to happen.

Long Version

the url I'm calling is split into <base url>+<path>+<apitoken>+<query> + <channel>. The query is simply <tcins> then <store info>. there are = and & peppered through those, but these characters are encoded to %26 and %3D inconsistently through the url. for instance you'll note that calling the api key is fine to use ?key=..., but immediately afterwards, the &tcins=82691535... is encoded to %26tcins%3D82691535....

the get method is:

   @Get("/product_summary_with_fulfillment_v1?key=${redsky-api.token}&{tcins}&{targetStore}&channel=WEB")
    HttpResponse<List<?>> fetchProducts(String tcins, String targetStore);
Log excerpt
11:58:04.258 [default-nioEventLoopGroup-1-1] DEBUG io.micronaut.http.client.netty.DefaultHttpClient - Sending HTTP GET to  https://redsky.target.com/redsky_aggregations/v1/web/product_summary_with_fulfillment_v1?key=9f36aeafbe60771e321a7cc95a78140772ab3e96&%26tcins%3D82691535&store_id%3D1750%26zip%3D84014-1752%26state%3DUT%26latitude%3D40.91825%26longitude%3D-111.887%26scheduled_delivery_store_id%3D1750%26required_store_id%3D1750%26has_required_store_id%3Dtrue&channel=WEB
11:58:04.265 [default-nioEventLoopGroup-1-1] TRACE io.micronaut.http.client.netty.DefaultHttpClient - User-Agent: Micronaut HTTP Client
11:58:04.265 [default-nioEventLoopGroup-1-1] TRACE io.micronaut.http.client.netty.DefaultHttpClient - Accept: application/vnd.github.v3+json, application/json
11:58:04.266 [default-nioEventLoopGroup-1-1] TRACE io.micronaut.http.client.netty.DefaultHttpClient - host: redsky.target.com
Controller
package com.github.peanutbutter.unicorn.tarvester;

import com.github.peanutbutter.unicorn.tarvester.model.TargetStore;
import com.github.peanutbutter.unicorn.tarvester.model.TcinList;
import io.micronaut.core.async.annotation.SingleResult;
import io.micronaut.http.HttpResponse;
import io.micronaut.http.annotation.Controller;
import io.micronaut.http.annotation.Get;

import java.util.List;

@Controller("products")
public class TarvesterController {

    final TarvesterClient tarvesterClient;

    public TarvesterController(TarvesterClient tarvesterClient) {
        this.tarvesterClient = tarvesterClient;
    }

    @Get("by-tcin")
    @SingleResult
    HttpResponse<List<?>> fetchProducts(TcinList tcinlist, TargetStore targetStore) {
        return tarvesterClient.fetchProducts(tcinlist.toString(), targetStore.toString());
    }
}
Client
package com.github.peanutbutter.unicorn.tarvester;

import com.github.peanutbutter.unicorn.tarvester.model.Product;
import io.micronaut.http.HttpResponse;
import io.micronaut.http.annotation.Get;
import io.micronaut.http.annotation.Header;
import io.micronaut.http.client.annotation.Client;

import java.util.List;

import static io.micronaut.http.HttpHeaders.ACCEPT;
import static io.micronaut.http.HttpHeaders.USER_AGENT;

@Client(id = "redsky-api")
@Header(name = USER_AGENT, value = "Micronaut HTTP Client")
@Header(name = ACCEPT, value = "application/vnd.github.v3+json, application/json")
public interface TarvesterClient {

    @Get("/product_summary_with_fulfillment_v1?key=${redsky-api.token}&{targetStore}&{tcins}&channel=WEB")
    HttpResponse<List<?>> fetchProducts(String tcins, String targetStore);
}
ToString methods
    //tcins excerpt
    @Override
    public String toString() {
        return "&tcins=" + Arrays.stream(id)
                .mapToObj(String::valueOf)
                .collect(Collectors.joining("%2C"));
    }
...
    // TargetStore excerpt
    @Override
    public String toString() {
        return String.join("&", new String[]{
                "store_id=" + locationId,
                "zip=" + postalCode,
                "state=" + region,
                "latitude=" + latitude,
                "longitude=" + longitude,
                "scheduled_delivery_store_id=" + locationId,
                "required_store_id=" + locationId,
                "has_required_store_id=true"
        });
    }

application.properties
micronaut.application.name=Tarvester
micronaut.application.default-charset=UTF-8
micronaut.http.services.redsky-api.url=https://redsky.target.com/
micronaut.http.services.redsky-api.path=redsky_aggregations/v1/web
redsky-api.token=9f36aeafbe60771e321a7cc95a78140772ab3e96

Things I've tried

  • I've tried changing the controller and get annotations (separately) to include produces = MediaType.PLAINTEXT, but neither of those effected the url encoding issue.
  • I am swapping out where the java objects are converted to strings (which is where these characters are introduced) to be in the client or the controller, but i'm not hopeful this will work as the Get annotation includes the api key retrieval as well as the string interpolation for the java object string values.
  • I can't find any documentation on some kind of parameter for the httpclient to not encode the url
  • I read in the configuration reference that you can define the default application charset, so I changed that to UTF-8 to no avail

if you want to try this yourself, the project is https://github.com/PeanutButter-Unicorn/Tarvester

Part of me wonders if this is a bug since this isn't consisten behavoir across the URL?

0

There are 0 answers