How to replace a whole `anyOf` object with a custom field?

Question

I have a specific anyOf field in my form, and I need the whole field replaced with a custom field.

I can easily replace a whole plain object with a custom field like this [sandbox link]:

const MyCustomField = () => "I am a custom field!";

const schema = {
  type: "object",
  title: "first",
  properties: {
    item: { type: "string", enum: ["first"], default: "first" },
    firstProp: { type: "string" }
  }
};

const uiSchema = { "ui:field": "MyCustomField" };

This works very well. But when I try to apply the same idea to an anyOf instead of a plain object, it starts acting weird. The object's title gets replaced with widget, but the anyOf item select is still visible, and a second instance of the widgets appears below it! [Sandbox link]

const MyCustomField = () => "I am a custom field!";

const schema = {
  $id: "xxx",
  type: "object",
  anyOf: [
    {
      type: "object",
      title: "first",
      properties: {
        item: { type: "string", enum: ["first"], default: "first" },
        firstProp: { type: "string" }
      }
    },
    {
      type: "object",
      title: "second",
      properties: {
        item: { type: "string", enum: ["second"], default: "second" },
        secondProp: { type: "string" }
      }
    }
  ]
};

const uiSchema = { "ui:field": "MyCustomField" };

I managed to work around it simply by nesting the anyOf under a new object. But I don't want to alter my schema away from the structure already used by backend just to make this work.

I tried messing with different combinations of ui:widget, ui:field and "ui:title": "hidden", and combining it with the items keyword, but got nowhere.

Thank you very much