How to create a StackedBarChart with specific intervals, empty space and reoccurring Row Keys in JFreeChart?

Question

I want to create a StackedBarChart that represents Elements within a certain length. The Endresult should look something like this:

I have the data for each row (Row 1 -> A from 0,5 - 3 and B from 4,5 to 7 for example).

After playing around for a while and not finding a direct way to do this, as a workaround i tried to use a CategoryDataset with a StackedBarRenderer and "convert" my Data to the Format and just paint the empty the same as the Background.

This already is a problem, since it then tries to add both "empty" entries and just display it as one in the middle. I then simply made each "empty" entry unique and it worked fine that way. The Problem comes with the next "A" entry in Row "3" where it somehow switches the order i put the data into the dataset and displays "A" before "B" (i'm guessing to line it up with the A in the first row).

Current result of below Code.. as you can see, it's pretty close, but i guess actually very, very far away:

I don't really know what else to try right now. I have a feeling i missed something and there is a very easy way to achieve this - is there any way to do this with JFreeChart or should i use a different library?

Dataset:

private CategoryDataset createDataset() {
        DefaultCategoryDataset dataset = new DefaultCategoryDataset();

        //testdata
        dataset.addValue((Number) 0.5, "empty", 1);
        dataset.addValue((Number) 2.5, "A", 1);
        dataset.addValue((Number) 1.5, "empty2", 1);
        dataset.addValue((Number) 2.5, "B", 1);

        dataset.addValue((Number) 1.5, "empty3", 2);
        dataset.addValue((Number) 2.5, "C", 2);
        dataset.addValue((Number) 2, "empty4", 2);
        dataset.addValue((Number) 3, "D", 2);

        dataset.addValue((Number) 1, "empty5", 3);
        dataset.addValue((Number) 2.5, "B", 3);
        dataset.addValue((Number) 1.5, "empty6", 3);
        dataset.addValue((Number) 3, "A", 3);
        return dataset;
}

Chart:

        plot = new CategoryPlot();
        plot.setRangeAxisLocation(AxisLocation.TOP_OR_RIGHT);
        plot.setOrientation(PlotOrientation.HORIZONTAL);

        CategoryDataset dataset = createDataset();

        CategoryAxis catAxis = new CategoryAxis();
        NumberAxis numAxis = new NumberAxis();
        numAxis.setVerticalTickLabels(true);

        StackedBarRenderer rend = new StackedBarRenderer();

        for (int i = 0; i < dataset.getRowCount(); i++) {
            String currKey = (String) dataset.getRowKey(i);
            if (currKey.startsWith("empty")) {
                rend.setSeriesPaint(i, INVISIBLE);
                rend.setSeriesItemLabelFont(i, new Font("Verdana", Font.BOLD, 0));
            }
        }

        rend.setShadowVisible(false);
        rend.setBarPainter(new StandardBarPainter());
        rend.setMaximumBarWidth(HEIGHT);

        rend.setBaseItemLabelGenerator(new StandardCategoryItemLabelGenerator("{0}", NumberFormat.getInstance()));
        rend.setBaseItemLabelsVisible(true);

        plot.setDataset(dataset);
        plot.setDomainAxis(catAxis);
        plot.setRangeAxis(numAxis);
        plot.setDomainGridlinesVisible(false);
        plot.setRangeGridlinesVisible(false);
        plot.setRenderer(rend);
        plot.setBackgroundPaint(INVISIBLE);

        JFreeChart chart = new JFreeChart("", JFreeChart.DEFAULT_TITLE_FONT, plot, true);
        chart.getLegend().setVisible(false);