How to create a QVariant-based generic model?

Question

Quite often I find myself in need of some custom scheme model, mandating the implementation of more and more models, made even more tedious by the inability of QObject derived classes to be templates.

Qt has the QStandardItemModel but that seems a bit verbose and inconvenient to use, especially from the qml side, and total overkill for a basic list model.

There is also the basic qml ListModel, but that is limited and not elegant to use on the C++ side, and I do suspect a tad more bloated than it needs to be.

Qt has QVariant, which is what its model/view architecture uses internally, so it is surprising that the framework doesn't provide something as simple as:

// qml code
  VarMod {
    roles: ["name", "age", "weight"]
    Component.onCompleted: {
      insert(["Jack", 34, 88.5], -1) // qml doesn't support
      insert(["Mary", 26, 55.3], -1) // default arg values
    }
  }

// cpp code
VarMod vm { "name", "age", "weight" }; // member declaration
vm.insert({ "Jack", 34, 88.5 });
vm.insert({ "Mary", 26, 55.3 });

Answer 1

And here it is.

Note that you do have to be responsible with the parameters, as there is no type safety, in fact it has implicit analog to ListModel's dynamicRoles - that is, it will accept and work with any QVariant compatible value on every role slot.

As for memory efficiency, consider that QVariant has 8 bytes for data, plus 4 bytes for type id, plus another 4 bytes of padding, for a total of 16 bytes. That is not insignificant if you are using it for small data types, like say bool, so in case you have a data scheme that has a lot of small (1 - 4 bytes) fields and a scores of items, implementing a full model will still be the better option. It is still a lot better than the generic object model I am using, which has to carry the bloat of QObject, and even more significant in the case of qml objects.

Additionally, QVariant being 16 bytes, I opted to not use the convenience of QVariantList for data storage, which has an underlying QList, making the situation worse than it needs to be. Although that is fixed in Qt 6, which gets rid of QList as it is, and replaces it with an alias of QVector. Still, std::vector helps to avoid that in any case, plus it might actually be a tad faster, since it doesn't have to deal with COW and atomic ref counters. There are several auxiliary methods to help with pre-allocation and release of memory as well.

The model has a safeguard against the change the roles for obvious reasons, the latter is primarily intended to be initialized just once, but there is reset() that is intended to be used in a more dynamic qml context, making it possible to redefine the model schema on the fly and provide a compatible delegate. For the sake of certainty, the roles can only be redefined after the model has been explicitly reset.

There is a minute difference in inserting, on the c++ side, the parameter pack is passed wrapped in {}, in qml it is wrapped in [], both leveraging implicit conversion in the context specific way. Also, note that qml currently doesn't support omitting parameters with default values provided on the c++ side, so for appending you do have to provide an invalid index. Naturally, it would be trivial to add convenience methods for appending and prepending if needed.

In addition to the syntax example of the question, it is also possible to add multiple items at once, from "declarative-y" qml structure such as:

let v = [["Jack", 34, 88.5],
         ["Mary", 26, 55.3],
         ["Sue", 22, 69.6]]
vm.insertList(v, -1)

Finally, type safety is possible to implement if the scenario really calls for it, then each role can be specified with the expected type to go with it, such as:

VarMod vm {{"name", QMetaType::QString},
           {"age", QMetaType::Int},
           {"weight", QMetaType::QReal}};

and then iterating and making the necessary checks to ensure type safety when inserting.

Update: I also added serialization, and save/load from disk features, note that this will serialize the data together with the mode schema.

class VarMod : public QAbstractListModel {
    Q_OBJECT
    Q_PROPERTY(QVariantList roles READ roles WRITE setRoles NOTIFY rolesChanged)
    QVariantList vroles;
    QVariantList roles() const { return vroles; }
    QHash<int, QByteArray> _roles;
    std::vector<std::vector<QVariant>> _data;
    inline bool checkArgs(int rc) const {
      if (rc == _roles.size()) return true;
      qWarning() << "arg size mismatch, got / expected" << rc << _roles.size();
      return false;
    }
    inline bool inBounds(int i, bool ok = false) const {
      if (i > -1 && i < (int)_data.size()) return true;
      if (!ok) qWarning() << "out of bounds" << i; // do not warn if intentionally appending
      return false;
    }
    inline bool validRole(int r) const { return (r > -1 && r < _roles.size()); }
  protected:
    QHash<int, QByteArray> roleNames() const override { return _roles; }
    int rowCount(const QModelIndex &) const override { return _data.size(); }
    QVariant data(const QModelIndex &index, int r) const override {
      r = r - Qt::UserRole - 1;
      if (inBounds(index.row()) && validRole(r)) return _data[index.row()][r];
      return QVariant();
    }
  public:
    VarMod() {} // for qml
    VarMod(std::initializer_list<QByteArray> r) {
      int rc = Qt::UserRole + 1;
      for (const auto & ri : r) {
        _roles.insert(rc++, ri);
        vroles << QString::fromLatin1(ri);
      }
      rolesChanged();
    }
    inline void insert(std::initializer_list<QVariant> s, int i = -1) {
      if (!checkArgs(s.size())) return;
      insert(QVariantList(s), i);
    }
    inline bool setItem(int i, std::initializer_list<QVariant> s) {
      if (checkArgs(s.size())) return setItem(i, QVariantList(s));
      return false;
    }
    void setRoles(QVariantList r) {
      if (_roles.empty()) {
        int rc = Qt::UserRole + 1;
        for (const auto & vi : r) _roles.insert(rc++, vi.toByteArray());
        vroles = r;
        rolesChanged();
      } else qWarning() << "roles are already initialized";
    }
    void read(QDataStream & d) {
      reset();
      QVariantList vr;
      d >> vr;
      quint32 s;
      d >> s;
      _data.resize(s);
      for (uint i = 0; i < s; ++i) {
        _data[i].reserve(vr.size());
        for (int c = 0; c < vr.size(); ++c) {
          QVariant var;
          d >> var;
          _data[i].push_back(std::move(var));
        }
      }
      setRoles(vr);
      beginResetModel();
      endResetModel();
    }
    void write(QDataStream & d) const {
      d << vroles;
      d << (quint32)_data.size();
      for (const auto & v : _data) {
        for (const auto & i : v) d << i;
      }
    }
  public slots:
    void insert(QVariantList s, int i) {
      if (!inBounds(i, true)) i = _data.size();
      if (!checkArgs(s.size())) return;
      beginInsertRows(QModelIndex(), i, i);
      _data.insert(_data.begin() + i, { s.cbegin(), s.cend() });
      endInsertRows();
    }
    void insertList(QVariantList s, int i) {
      if (!inBounds(i, true)) i = _data.size();
      int added = 0;
      for (const auto & il : s) {
        QVariantList ll = il.value<QVariantList>();
        if (checkArgs(ll.size())) {
          _data.insert(_data.begin() + i + added++, { ll.cbegin(), ll.cend() });
        }
      }
      if (added) {
        beginInsertRows(QModelIndex(), i, i + added - 1);
        endInsertRows();
      }
    }
    bool setData(int i, int r, QVariant d) {
      if (!inBounds(i) || !validRole(r)) return false;
      _data[i][r] = d;
      dataChanged(index(i), index(i));
      return true;
    }
    bool setDataStr(int i, QString rs, QVariant d) { // a tad slower
      int r = _roles.key(rs.toLatin1()); // role is resolved in linear time
      if (r) return setData(i, r - Qt::UserRole - 1, d);
      qWarning() << "invalid role" << rs;
      return false;
    }
    bool setItem(int i, QVariantList d) {
      if (!inBounds(i) || !checkArgs(d.size())) return false;
      _data[i] = { d.cbegin(), d.cend() };
      dataChanged(index(i), index(i));
      return true;
    }
    QVariantList item(int i) const {
      if (!inBounds(i)) return QVariantList();
      const auto & v = _data[i];
      return { v.begin(), v.end() };
    }
    QVariant getData(int i, int r) const {
      if (inBounds(i) && validRole(r)) return _data[i][r];
      return QVariant();
    }
    QVariant getDataStr(int i, QString rs) const {
      int r = _roles.key(rs.toLatin1()); // role is resolved in linear time
      if (r) return getData(i, r);
      qWarning() << "invalid role" << rs;
      return QVariant();
    }
    QVariantList take(int i) {
      QVariantList res = item(i);
      if (res.size()) remove(i);
      return res;
    }
    bool swap(int i1, int i2) {
      if (!inBounds(i1) || !inBounds(i2)) return false;
      std::iter_swap(_data.begin() + i1, _data.begin() + i2);
      dataChanged(index(i1), index(i1));
      dataChanged(index(i2), index(i2));
      return true;
    }
    bool remove(int i) {
      if (!inBounds(i)) return false;
      beginRemoveRows(QModelIndex(), i, i);
      _data.erase(_data.begin() + i);
      endRemoveRows();
      return true;
    }
    void clear() {
      beginResetModel();
      _data.clear();
      _data.shrink_to_fit();
      endResetModel();
    }
    void reset() {
      clear();
      _roles.clear();
      vroles.clear();
      rolesChanged();
    }
    void reserve(int c) { _data.reserve(c); }
    int size() const { return _data.size(); }
    int capacity() const { return _data.capacity(); }
    void squeeze() { _data.shrink_to_fit(); }
    bool fromFile(QString path) {
      QFile f(path);
      if (!f.open(QIODevice::ReadOnly)) return false;
      QDataStream d(&f);
      read(d); // assumes correct data
      return true;
    }
    bool toFile(QString path) const {
      QFile f(path);
      if (!f.open(QIODevice::WriteOnly)) return false;
      QDataStream d(&f);
      write(d);
      return true;
    }
  signals:
    void rolesChanged();
};

I also created this sorting/filtering view to supplement the model:

class View : public QSortFilterProxyModel {
    Q_OBJECT
    Q_PROPERTY(QJSValue filter READ filter WRITE set_filter NOTIFY filterChanged)
    Q_PROPERTY(bool reverse READ reverse WRITE setReverse NOTIFY reverseChanged)
    bool reverse() const { return _reverse; }
    void setReverse(bool v) {
      if (v == _reverse) return;
      _reverse = v;
      reverseChanged();
      sort(0, (Qt::SortOrder)_reverse);
    }
    bool _reverse = false;
    mutable QJSValue m_filter;
    QJSValue & filter() const { return m_filter; }
    void set_filter(QJSValue & f) {
      if (!m_filter.equals(f))
        m_filter = f;
        filterChanged();
        invalidateFilter();
      }
    }
  public:
    View(QObject *parent = 0) : QSortFilterProxyModel(parent) { sort(0, (Qt::SortOrder)_reverse); }
  signals:
    void filterChanged();
    void reverseChanged();
  protected:
    bool filterAcceptsRow(int sourceRow, const QModelIndex &) const override {
      if (!m_filter.isCallable()) return true;
      VarMod * vm = qobject_cast<VarMod *>(sourceModel());
      if (!vm) {
        qWarning() << "model is not varmod";
        return true;
      }
      return m_filter.call({_engine->toScriptValue(vm->item(sourceRow))}).toBool();
    }
    bool lessThan(const QModelIndex &left, const QModelIndex &right) const override {
      VarMod * vm = qobject_cast<VarMod *>(sourceModel());
      if (!vm) {
        qWarning() << "model is not varmod";
        return false;
      }
      return vm->getData(left.row(), sortRole()) < vm->getData(right.row(), sortRole());
    }
};

For sorting, you just have to specify the sorting role, note that it is the index of the "column" rather than the int value from the roles hash. For filtering it works via a qml functor that receives the model item as a JS array, and expects to return a bool, a c++ functor can be easily added via std::function if needed. Also note that it needs a pointer to the actual qml engine.

  View {
    id: vv
    sourceModel: vm
    sortRole: sr.value
    reverse: rev.checked
    filter: { sa.value; o => o[1] < sa.value } // "capturing" sa.value to react to value changes
  }